Raw Docker Support


Info

Note that functionality here is use-case driven and so the docker.* target api interface is not intended to be 1:1 with the docker CLI. Using the docker CLI directly is fine, so we're not looking to replace it. Instead, the offerings here are building on top of it, or making common interactions somewhat smoother, or exposing a pure-docker interface that's similar to the compose bridge.

Usually it's a good idea to separate task-definitions from the containers runtime, and using the docker compose support is really the recommended way to arrange things.

However.. compose.mk sometimes needs more direct and low-level access to docker primitives for internal use, and this functionality is exposed for external use as well.

Tool-Container Defaults


Most of the low-level docker support in compose.mk makes the assumption that you're working with tool containers, and there's two important defaults that come along with this assumption:

  1. The working directory, which is usually the project root, is volume-mounted by default.
  2. The docker-socket is mounted and shared by default also.

In the first case, without sharing the working directory, tools could not do any file IO, and only stream-based communication would be possible. And in the second case, the ability to access other dockerized tools from inside docker containers enables many other compose.mk features, from structured io and stage stacks to the embedded TUI.

Dispatch Tasks in Containers


Target-dispatch for imported tool containers from compose files is explained in detail here, and other sections of this page show examples of dispatch used with an embedded image.

For unmodified stock images, the situation is similar, as long as that image actually ships with make. One of the smallest images preconfigured with make is debian/buildd, so we'll use that here. As mentioned in the installation docs, lots of your daily-driver images probably already have make by default, including images for stuff like gcc, golang, maven, etc.

The simplest way to use dispatch with existing targets is using docker.dispatch/<target> directly:

# Run the trivial `flux.ok` target inside the given image
$ img=debian/buildd:bookworm ./compose.mk docker.dispatch/flux.ok
 docker.run // img=debian/buildd:bookworm 
Φ flux.ok // succeeding as requested! 

Using docker.import


If you're doing something like the above frequently, then you can also use docker.image.import to generate scaffolded targets that do the same thing:

Summary
#!/usr/bin/env -S make -f
# Demonstrates importing a docker image, then using scaffolded targets
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# USAGE: demos/import-image.mk

include compose.mk

$(call docker.import, namespace=debian img=debian/buildd:bookworm)

__main__: test.dispatch test.low_level_runner

test.dispatch: debian.dispatch/flux.ok

test.low_level_runner:
    entrypoint=sh cmd='-c "echo hello-world"' ${make} debian

Besides the namespace.dispatch/.. helper and the main namespace target that are created and demonstrated above, this also creates a namespace.shell target that can be used to drop you into an interactive debugging shell for the container, and creates ${namespace.img} that holds data for the image name. Argument reference follows.

Arguments for `docker.import`
Name Required? Description Example
img Image to use. img=debian/buildd:bookworm
namespace Namespace to assign to image. namespace=debian

Inlined Dockerfiles


Working with inlined container definitions is mostly the same as working with stock images, but there are few things to keep in mind.

  1. Inlined Dockerfiles should be inside define .. endef blocks with a name like Dockerfile.my_container
  2. Containers must be built before you can use them, typically with something like Dockerfile.build/<my_container>.
  3. After build, containers are tagged as something like compose.mk:my_container

Usually the compose.mk:.. prefix for image tags can be omitted, for example by using docker.dispatch/<target> instead of mk.docker.dispatch/<target>.

See the polyglot demos and matrioshka demos for many examples of working with embedded images. See also the equivalent idioms in CMK-lang. The rest of this section covers low-level support.

Low Level Support


Sometimes you might be interested in fine-grained control or more flexibility. Below you can see test-cases that are exercising low-level support for raw docker.

Summary
#!/usr/bin/env -S make -f
# Demonstrates inlining a Dockerfile, 
# building it, then working with the container.
#
# Part of the `compose.mk` repo, runs as part of the test-suite.
#
# USAGE: ./demos/inlined-dockerfile.mk

include compose.mk

# Minimal inlined dockerfile.  
# You can install anything or nothing here, but let's 
# have the minimal stuff that's required for using target dispatch.
define Dockerfile.demo_dockerfile
FROM ${IMG_ALPINE_BASE:-alpine:3.21.2}
RUN apk add -q --update --no-cache coreutils build-base bash procps-ng
endef

# After build, image is always at 'compose.mk:<def_name>'.
# This "absolute" name is expected by `docker.*` targets, 
# but the prefix is implied for `mk.docker.*`.
inlined_img=compose.mk:demo_dockerfile

# Entrypoint.  Ensures the container is built, then runs all the tests.
__main__: Dockerfile.build/demo_dockerfile flux.star/test

test.1.image_created_and_available:
    $(call log.test, Image is created and available to docker)
    docker image inspect ${inlined_img} > /dev/null
    docker run --entrypoint sh ${inlined_img} -x -c "true" > /dev/null

test.2.mk.docker.dispatch:
    $(call log.test, Omits image prefix & does target-dispatch)
    img=demo_dockerfile ${make} mk.docker.dispatch/self.demo.dispatch
self.demo.dispatch:
    printf "Running inside the inlined-container:\n"
    uname -a

test.3.docker.dispatch:
    $(call log.test, Expects image prefix & accepts targets)
    img=${inlined_img} ${make} docker.dispatch/self.demo.dispatch

test.4.docker.run.sh:
    $(call log.test, Low-level access to container)
    entrypoint=sh cmd='-c "pwd"' \
        img=${inlined_img} ${make} docker.run.sh 

test.5.build.cache_busting:
    $(call log.test, Caching by default. Pass force=1 to override)
    force=1 ${make} Dockerfile.build/demo_dockerfile

test.6.quiet_build:
    $(call log.test, Dockerfile.build silent by default. Pass quiet=0 to override)
    quiet=0 force=1 ${make} Dockerfile.build/demo_dockerfile

test.7.docker.lambda.target:
    $(call log.test, Builds/runs Dockerfile in 1 step with docker.lambda)
    cmd='pwd' ${make} docker.lambda/demo_dockerfile

Dispatch Scripts in Containers


Dispatching tasks is usually more composable, but working with scripts can be more convenient for one-offs. Script dispatch with stock-images looks like this:

Summary
#!/usr/bin/env -S make -f
# Demonstrating idioms for container-agnostic script dispatch with stock images.
# The target script always runs from the container, but does not care whether 
# it's called from the host, or inside the container.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.  
# See also: http://robot-wranglers.github.io/compose.mk/container-dispatch
#
# USAGE: ./demos/script-dispatch-stock.mk

include compose.mk

img=debian/buildd:bookworm

__main__: script.sh wrapper_script

# Create target from the implied script and the given image
script.sh:; $(call docker.bind.script, img=${img})
define script.sh
echo hello `hostname` at `uname -a`
endef

# If target and script name differ, provide 2nd argument
wrapper_script:; $(call docker.bind.script, img=${img} def=script.sh)

For inlined-containers, the approach is similar:

Summary
#!/usr/bin/env -S make -f
# Demonstrating idioms for container-agnostic script dispatch with stock images.
# The target script always runs from the container, but does not care whether 
# it's called from the host, or inside the container.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.  
# See also: http://robot-wranglers.github.io/compose.mk/container-dispatch
#
# USAGE: ./demos/script-dispatch-custom.mk

include compose.mk
__main__: my_script 

# Look, it's a container definition 
define Dockerfile.my_container
FROM debian/buildd:bookworm
# .. Other customization here .. 
endef

# Look, it's a script to run in the container 
define my_script
echo hello `hostname` at `uname -a`
endef

# Binds the given target to the given container + implied script
my_script:; $(call mk.docker.bind.script, img=my_container)

# If script and target names differ, provide `def` argument
script_wrapper:; $(call mk.docker.bind.script, img=my_container def=my_script)

Arguments for docker.bind.script & mk.docker.bind.script
Name Required? Description Example
img Image to use.
Defaults to 1st positional-arg if kwargs not present.
img=...
def Name of code-block.
Implied for CMK ⨖-syntax
def=..
entrypoint Interpreter to use.
Defaults to bash.
entrypoint=bash
cmd Arguments to pass to interpreter.
Defaults to empty string.
Filename is post-fixed to command.
cmd=-x
env Variables to pass through to container.
Defaults to value from environment.
env='foo bar'

Other Docker Utilities


Most of the use-cases for raw docker support are related to scripting with compose.mk. Sometimes though the raw docker support has more of a stand-alone / tool mode vibe. For usage hints along those lines, see the test cases below.

Summary
#!/usr/bin/env -S bash -x -euo pipefail
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
#
# USAGE: bash -x tests/docker-utils.sh

# Shows docker version info
./compose.mk docker.init

# Shows docker status
./compose.mk docker.stat

# Shows newline-separated size details per repo / image.
./compose.mk docker.size.summary

# Create container from URL
url="https://github.com/alpine-docker/git.git#1.0.38:." \
    tag="alpine-git" ./compose.mk docker.from.url

# Construct URL from attributes
user=alpine-docker repo=git tag="1.0.38" ./compose.mk docker.from.github

# Build a Dockerfile to a tag
tag=compose.mk:testing ./compose.mk docker.from.file/demos/data/Dockerfile

# Failure if no tag is provided
! ./compose.mk docker.from.file/demos/data/Dockerfile

# Failure if file does not exist
! ./compose.mk docker.from.file/demos/data/Dockerfile.missing

# Run an image with a command
img=debian/buildd:bookworm  entrypoint=sh cmd='-c ls' ./compose.mk docker.run.sh

# Run a target inside an image
img=debian/buildd:bookworm ./compose.mk docker.dispatch/flux.ok

# Show help for the docker.images target
./compose.mk help docker.images

# Show images that are managed by compose.mk
./compose.mk docker.images 

# Show all docker images
./compose.mk docker.images.all

# Like docker ps, but always returns JSON
./compose.mk docker.ps

Full Docker API


Below, you can find quick links for the docker / Dockerfile / mk.docker sections of the main API.