Raw Docker Support
Info
Note that functionality here is use-case driven and so the docker.*
target api interface is not intended to be 1:1 with the docker CLI. Using the docker CLI directly is fine, so we're not looking to replace it. Instead, the offerings here are building on top of it, or making common interactions somewhat smoother, or exposing a pure-docker interface that's similar to the compose bridge.
Usually it's a good idea to separate task-definitions from the containers runtime, and using the docker compose support is really the recommended way to arrange things.
However.. compose.mk
sometimes needs more direct and low-level access to docker primitives for internal use, and this functionality is exposed for external use as well.
Tool-Container Defaults
Most of the low-level docker support in compose.mk
makes the assumption that you're working with tool containers, and there's two important defaults that come along with this assumption:
- The working directory, which is usually the project root, is volume-mounted by default.
- The docker-socket is mounted and shared by default also.
In the first case, without sharing the working directory, tools could not do any file IO, and only stream-based communication would be possible. And in the second case, the ability to access other dockerized tools from inside docker containers enables many other compose.mk
features, from structured io and stage stacks to the embedded TUI.
Dispatch Tasks in Containers
Target-dispatch for imported tool containers from compose files is explained in detail here, and other sections of this page show examples of dispatch used with an embedded image.
For unmodified stock images, the situation is similar, as long as that image actually ships with make
. One of the smallest images preconfigured with make
is debian/buildd
, so we'll use that here. As mentioned in the installation docs, lots of your daily-driver images probably already have make
by default, including images for stuff like gcc, golang, maven, etc.
The simplest way to use dispatch with existing targets is using docker.dispatch/<target> directly:
# Run the trivial `flux.ok` target inside the given image
$ img=debian/buildd:bookworm ./compose.mk docker.dispatch/flux.ok
≣ docker.run // img=debian/buildd:bookworm
Φ flux.ok // succeeding as requested!
Using docker.import
If you're doing something like the above frequently, then you can also use docker.image.import
to generate scaffolded targets that do the same thing:
#!/usr/bin/env -S make -f
# Demonstrates importing a docker image, then using scaffolded targets
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# USAGE: demos/import-image.mk
include compose.mk
$(call docker.import, namespace=debian img=debian/buildd:bookworm)
__main__: test.dispatch test.low_level_runner
test.dispatch: debian.dispatch/flux.ok
test.low_level_runner:
entrypoint=sh cmd='-c "echo hello-world"' ${make} debian
Besides the namespace.dispatch/..
helper and the main namespace
target that are created and demonstrated above, this also creates a namespace.shell
target that can be used to drop you into an interactive debugging shell for the container, and creates ${namespace.img}
that holds data for the image name. Argument reference follows.
Name | Required? | Description | Example |
---|---|---|---|
img |
✅ | Image to use. | img=debian/buildd:bookworm |
namespace |
✅ | Namespace to assign to image. | namespace=debian |
Inlined Dockerfiles
Working with inlined container definitions is mostly the same as working with stock images, but there are few things to keep in mind.
- Inlined Dockerfiles should be inside
define .. endef
blocks with a name likeDockerfile.my_container
- Containers must be built before you can use them, typically with something like Dockerfile.build/<my_container>.
- After build, containers are tagged as something like
compose.mk:my_container
Usually the compose.mk:..
prefix for image tags can be omitted, for example by using docker.dispatch/<target> instead of mk.docker.dispatch/<target>.
See the polyglot demos and matrioshka demos for many examples of working with embedded images. See also the equivalent idioms in CMK-lang. The rest of this section covers low-level support.
Low Level Support
Sometimes you might be interested in fine-grained control or more flexibility. Below you can see test-cases that are exercising low-level support for raw docker.
#!/usr/bin/env -S make -f
# Demonstrates inlining a Dockerfile,
# building it, then working with the container.
#
# Part of the `compose.mk` repo, runs as part of the test-suite.
#
# USAGE: ./demos/inlined-dockerfile.mk
include compose.mk
# Minimal inlined dockerfile.
# You can install anything or nothing here, but let's
# have the minimal stuff that's required for using target dispatch.
define Dockerfile.demo_dockerfile
FROM ${IMG_ALPINE_BASE:-alpine:3.21.2}
RUN apk add -q --update --no-cache coreutils build-base bash procps-ng
endef
# After build, image is always at 'compose.mk:<def_name>'.
# This "absolute" name is expected by `docker.*` targets,
# but the prefix is implied for `mk.docker.*`.
inlined_img=compose.mk:demo_dockerfile
# Entrypoint. Ensures the container is built, then runs all the tests.
__main__: Dockerfile.build/demo_dockerfile flux.star/test
test.1.image_created_and_available:
$(call log.test, Image is created and available to docker)
docker image inspect ${inlined_img} > /dev/null
docker run --entrypoint sh ${inlined_img} -x -c "true" > /dev/null
test.2.mk.docker.dispatch:
$(call log.test, Omits image prefix & does target-dispatch)
img=demo_dockerfile ${make} mk.docker.dispatch/self.demo.dispatch
self.demo.dispatch:
printf "Running inside the inlined-container:\n"
uname -a
test.3.docker.dispatch:
$(call log.test, Expects image prefix & accepts targets)
img=${inlined_img} ${make} docker.dispatch/self.demo.dispatch
test.4.docker.run.sh:
$(call log.test, Low-level access to container)
entrypoint=sh cmd='-c "pwd"' \
img=${inlined_img} ${make} docker.run.sh
test.5.build.cache_busting:
$(call log.test, Caching by default. Pass force=1 to override)
force=1 ${make} Dockerfile.build/demo_dockerfile
test.6.quiet_build:
$(call log.test, Dockerfile.build silent by default. Pass quiet=0 to override)
quiet=0 force=1 ${make} Dockerfile.build/demo_dockerfile
test.7.docker.lambda.target:
$(call log.test, Builds/runs Dockerfile in 1 step with docker.lambda)
cmd='pwd' ${make} docker.lambda/demo_dockerfile
Dispatch Scripts in Containers
Dispatching tasks is usually more composable, but working with scripts can be more convenient for one-offs. Script dispatch with stock-images looks like this:
#!/usr/bin/env -S make -f
# Demonstrating idioms for container-agnostic script dispatch with stock images.
# The target script always runs from the container, but does not care whether
# it's called from the host, or inside the container.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/container-dispatch
#
# USAGE: ./demos/script-dispatch-stock.mk
include compose.mk
img=debian/buildd:bookworm
__main__: script.sh wrapper_script
# Create target from the implied script and the given image
script.sh:; $(call docker.bind.script, img=${img})
define script.sh
echo hello `hostname` at `uname -a`
endef
# If target and script name differ, provide 2nd argument
wrapper_script:; $(call docker.bind.script, img=${img} def=script.sh)
For inlined-containers, the approach is similar:
#!/usr/bin/env -S make -f
# Demonstrating idioms for container-agnostic script dispatch with stock images.
# The target script always runs from the container, but does not care whether
# it's called from the host, or inside the container.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/container-dispatch
#
# USAGE: ./demos/script-dispatch-custom.mk
include compose.mk
__main__: my_script
# Look, it's a container definition
define Dockerfile.my_container
FROM debian/buildd:bookworm
# .. Other customization here ..
endef
# Look, it's a script to run in the container
define my_script
echo hello `hostname` at `uname -a`
endef
# Binds the given target to the given container + implied script
my_script:; $(call mk.docker.bind.script, img=my_container)
# If script and target names differ, provide `def` argument
script_wrapper:; $(call mk.docker.bind.script, img=my_container def=my_script)
Name | Required? | Description | Example |
---|---|---|---|
img |
✅ |
Image to use.
Defaults to 1st positional-arg if kwargs not present. |
img=... |
def |
✅ |
Name of code-block.
Implied for CMK ⨖-syntax |
def=.. |
entrypoint |
❌ |
Interpreter to use.
Defaults to bash. |
entrypoint=bash |
cmd |
❌ |
Arguments to pass to interpreter.
Defaults to empty string. Filename is post-fixed to command. |
cmd=-x |
env |
❌ |
Variables to pass through to container.
Defaults to value from environment. |
env='foo bar' |
Other Docker Utilities
Most of the use-cases for raw docker support are related to scripting with compose.mk
. Sometimes though the raw docker support has more of a stand-alone / tool mode vibe. For usage hints along those lines, see the test cases below.
#!/usr/bin/env -S bash -x -euo pipefail
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
#
# USAGE: bash -x tests/docker-utils.sh
# Shows docker version info
./compose.mk docker.init
# Shows docker status
./compose.mk docker.stat
# Shows newline-separated size details per repo / image.
./compose.mk docker.size.summary
# Create container from URL
url="https://github.com/alpine-docker/git.git#1.0.38:." \
tag="alpine-git" ./compose.mk docker.from.url
# Construct URL from attributes
user=alpine-docker repo=git tag="1.0.38" ./compose.mk docker.from.github
# Build a Dockerfile to a tag
tag=compose.mk:testing ./compose.mk docker.from.file/demos/data/Dockerfile
# Failure if no tag is provided
! ./compose.mk docker.from.file/demos/data/Dockerfile
# Failure if file does not exist
! ./compose.mk docker.from.file/demos/data/Dockerfile.missing
# Run an image with a command
img=debian/buildd:bookworm entrypoint=sh cmd='-c ls' ./compose.mk docker.run.sh
# Run a target inside an image
img=debian/buildd:bookworm ./compose.mk docker.dispatch/flux.ok
# Show help for the docker.images target
./compose.mk help docker.images
# Show images that are managed by compose.mk
./compose.mk docker.images
# Show all docker images
./compose.mk docker.images.all
# Like docker ps, but always returns JSON
./compose.mk docker.ps
Full Docker API
Below, you can find quick links for the docker / Dockerfile / mk.docker sections of the main API.
Related API |
---|
Dockerfile.build/<arg> |
Dockerfile.from.fs/<arg> |
mk.docker |
mk.docker.dispatch/<arg> |
mk.docker.image/<arg> |
mk.docker.prune |
mk.docker.run.sh |
mk.docker/<arg> |