Embedded Containers & Data
Mad Science
Reasonable people might consider this to be mad science. See this section for more discussion on that point, and if you're more interested in patterns and theory, then you might prefer to start here.
Sometimes it's useful to declare inlined containers and data alongside the same automation that is making use of those containers. You'll have to exercise some judgement about when and where that is a good idea, but since tool containers tend to be small and change infrequently, being self-contained reduces change-related friction and context-switching that's associated with external repositories, external registries, and external files.
Basic Examples are all on this page. Quick links: Inlined docker files, Extending inline docker files, Inlined compose files, Passing inlined data structures
Intermediate Examples are bigger external pages. Quick links: The Theorem Proving demo embeds a lean4 runtime, theorem, and code. The Custom Automation API demo wraps an Ansible runtime, then customizes it and exposes a different interface. And the Justfile demo wraps a just-runner and a just-file, then customizes it and exposes a different interface.
Advanced Examples are hard to come by, because for bigger projects you'll have to stop embedding stuff! However, the makeception demo might qualify, not based on length but based on weirdness.
Inlined Compose Files
#!/usr/bin/env -S make -f
# demos/inlined-composefile.mk:
# Demonstrates working with inlined compose-files via `_compose.import.string`,
# which works exactly like `compose.import`, but accepts embedded data instead of files.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
#
# USAGE: ./demos/inlined-composefile.mk
include compose.mk
# Look it's an embedded compose file. This defines services `alice` & `bob`
define inlined.services
services:
alice: &base
hostname: alice
build:
context: .
dockerfile_inline: |
FROM ${IMG_DEBIAN_BASE:-debian:bookworm-slim}
RUN apt-get update -qq && apt-get install -qq -y make procps
entrypoint: bash
working_dir: /workspace
volumes:
- ${PWD}:/workspace
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
bob:
<<: *base
hostname: bob
build:
context: .
dockerfile_inline: |
FROM ${IMG_ALPINE_BASE:-alpine:3.22}
RUN apk add -q --update --no-cache coreutils build-base bash procps-ng wget
# Download and compile make 4.3
RUN wget http://ftp.gnu.org/gnu/make/make-4.3.tar.gz
RUN tar -xzf make-4.3.tar.gz
#RUN cd make-4.3; ./configure && make && make install
endef
# After the inline, just call `compose.import.string` on it to
# build target scaffolding. See instead `compose.import` for
# using an external file.
$(call compose.import.string, def=inlined.services)
# Top level entrypoint, run the other individual demos.
__main__: demo.compose_verbs demo.dispatch
# Import has already modified the `inlined.services` namespace
# with familiar verbs from docker compose, like "build", "stop",
# "ps", etc. Now we can use them.
demo.compose_verbs: inlined.services.stop inlined.services.build
# Dispatch examples: run a task inside alice-container and bob-container.
# Note that docker build is implicit, on-demand, and cached unless forced.
demo.dispatch: \
alice.dispatch/self.internal_task \
bob.dispatch/self.internal_task
self.internal_task:
echo "Running inside `hostname`"
Inlined Dockerfile
#!/usr/bin/env -S make -f
# Demonstrates inlining a Dockerfile,
# building it, then working with the container.
#
# Part of the `compose.mk` repo, runs as part of the test-suite.
#
# USAGE: ./demos/inlined-dockerfile.mk
include compose.mk
# Minimal inlined dockerfile.
# You can install anything or nothing here, but let's
# have the minimal stuff that's required for using target dispatch.
define Dockerfile.demo_dockerfile
FROM ${IMG_ALPINE_BASE:-alpine:3.21.2}
RUN apk add -q --update --no-cache coreutils build-base bash procps-ng
endef
# After build, image is always at 'compose.mk:<def_name>'.
# This "absolute" name is expected by `docker.*` targets,
# but the prefix is implied for `mk.docker.*`.
inlined_img=compose.mk:demo_dockerfile
# Entrypoint. Ensures the container is built, then runs all the tests.
__main__: Dockerfile.build/demo_dockerfile flux.star/test
test.1.image_created_and_available:
$(call log.test, Image is created and available to docker)
docker image inspect ${inlined_img} > /dev/null
docker run --entrypoint sh ${inlined_img} -x -c "true" > /dev/null
test.2.mk.docker.dispatch:
$(call log.test, Omits image prefix & does target-dispatch)
img=demo_dockerfile ${make} mk.docker.dispatch/self.demo.dispatch
self.demo.dispatch:
printf "Running inside the inlined-container:\n"
uname -a
test.3.docker.dispatch:
$(call log.test, Expects image prefix & accepts targets)
img=${inlined_img} ${make} docker.dispatch/self.demo.dispatch
test.4.docker.run.sh:
$(call log.test, Low-level access to container)
entrypoint=sh cmd='-c "pwd"' \
img=${inlined_img} ${make} docker.run.sh
test.5.build.cache_busting:
$(call log.test, Caching by default. Pass force=1 to override)
force=1 ${make} Dockerfile.build/demo_dockerfile
test.6.quiet_build:
$(call log.test, Dockerfile.build silent by default. Pass quiet=0 to override)
quiet=0 force=1 ${make} Dockerfile.build/demo_dockerfile
test.7.docker.lambda.target:
$(call log.test, Builds/runs Dockerfile in 1 step with docker.lambda)
cmd='pwd' ${make} docker.lambda/demo_dockerfile
Extending Inlined Dockerfiles
Inlined containers can actually be extended with other inlines, but notice again the compose.mk:
prefix which is used as a repository.
#!/usr/bin/env -S make -f
# demos/extend-inlined-dockerfile.mk:
# Demonstrates extending an inlined container.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
#
# USAGE: ./demos/extend-inlined-dockerfile.mk
include demos/inlined-dockerfile.mk
# Inlined dockerfile, extending the one defined in `inlined-dockerfile.mk`
define Dockerfile.container_extension
FROM compose.mk:demo_dockerfile
RUN echo hello-docker
endef
# Ensures containers are built, then exercises them
__main__: Dockerfile.build/demo_dockerfile Dockerfile.build/container_extension
$(call log.test, Working with the image directly, note the 'compose.mk' prefix)
docker image inspect compose.mk:container_extension > /dev/null
docker run --entrypoint sh compose.mk:container_extension -x -c "true" > /dev/null
$(call log.test, Working with compose.mk builtins omits prefix \
and can dispatch targets to run inside the new image)
img=container_extension ${make} mk.docker.dispatch/self.demo_extension
$(call log.test, Add the prefix explicitly, and you can \
use `docker.run` instead of private `.docker.run`)
img=compose.mk:container_extension ${make} docker.dispatch/self.demo_extension
$(call log.test, Subsequent runs will use the cached image. \
Pass 'force' to work around this.)
force=1 ${make} Dockerfile.build/container_extension
self.demo_extension:
echo "Testing target from inside the extended container"
uname -a
Passing Inlined Data-Structures
For the related topic of general templating support, see instead this section of the polyglot docs.
Building on the Custom Automation APIs demo, let's embed an ansible playbook, then pass it into an embeddded container.
#!/usr/bin/env -S make -f
# demos/ansible-playbook.mk:
# Demonstrates passing embedded-data into an embedded-container.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# USAGE: ./demos/ansible-playbook.mk
include compose.mk
# Look, it's a container that has Ansible.
define Dockerfile.Ansible
FROM ${IMG_DEBIAN_BASE:-debian:bookworm-slim}
RUN apt-get update -qq && apt-get install -qq -y ansible make procps
endef
# Look, it's a simple Ansible playbook
define Ansible.playbook
- name: Example Playbook with Debug Task
hosts: localhost
gather_facts: no
tasks:
- name: Print a debug message
debug:
msg: "Hello, this is a debug message!"
endef
self.playbook_runner/%:
@# We only run on the `Ansible.playbook` def, but multiple playbooks are
@# possible so this target accepts the name of the define as a parameter.
@# Since runner runs inside the `Ansible` container described above, now
@# the ansible CLI is available for use. Next, write a tempfile for the
@# `Ansible.playbook` content, then call ansible, ensuring JSON oputput.
$(call log.io, ${dim_green} Running playbook:)
$(call io.mktemp) \
&& ${mk.def.read}/${*} | ${stream.peek} > $${tmpf} \
&& ANSIBLE_STDOUT_CALLBACK=json \
ansible-playbook -i localhost, -c local $${tmpf}
$(call log.io, ${dim_green} Playbook finished.)
# Main entrypoint for the playbook demo.
# The prerequisite `Dockerfile.build/Ansible` ensures the image is ready.
__main__: Dockerfile.build/Ansible
@# The next line specifies the target to run inside the container
img=compose.mk:Ansible ${make} \
docker.dispatch/self.playbook_runner/Ansible.playbook