Quickstart

Basic Installation

Just copy the compose.mk file and make it executable:

$ cd myproject 

# Download the compose.mk automation lib
$ curl -sL \
  https://raw.githubusercontent.com/Robot-Wranglers/compose.mk/master/compose.mk \
    > compose.mk

# Make it executable
$ chmod +x compose.mk

# Now equivalent to `make -f compose.mk ...`
$ ./compose.mk ...

If you have make and docker, that's all the setup you'll need for stand-alone tool mode usage. While it's not really necessary or recommended.. you can always copy it to a bin folder too if you want. (See the forks and versioning section for more details.)

Note that since various containers are only pulled on demand, initial usage can be slow, but subsequent runs can take advantage of cache. Read on for details about tighter integration with projects.

Project Integration

If you're looking to integrate with existing automation or you brought your own tool containers, you'll probably be interested in setting up the Make/Compose Bridge or preparing for Container Dispatch. See that full documentation for more discussion, or if you already know what you're doing, read on for the copy/paste stuff.

Makefile Integration

A quick example of what your project Makefile should look like:

# Include compose.mk for access to target API and macros.
include compose.mk

# If you brought tool containers, you'll want to import them.
# Call the bridge-building macro to get targets for each service.
$(call compose.import, file=docker-compose.yml)

# At this point, scaffolded targets are defined for 
# the tool containers mentioned in the compose file.
# Now you can dispatch any task to any tool container.
test: tool_container_name.dispatch/self.test

self.test:
    echo hello world from container `uname -n -v`

Docker Compose Integration

Working with docker compose is optional if you're just interested in stand-alone mode or the standard library offerings. And if you are using docker-compose, then simple access to tools-containers from your compose files will just work for many use-cases, via the Makefile integration in the last section and via the scaffolded targets.

However.. more complicated use-cases will need a volume-mount for your project directory so that tool containers can read/write files, and, using container dispatch means that your tool containers need make and bash.

Here is some fairly complete boilerplate that's usually a good starting place:

##
# demos/data/docker-compose.yml: 
#  Typical compose file with container-bases that are preconfigured to 
#  work with target dispatch, default workspaces, volume sharing for
#  tools, yaml inheritance, etc
services:
  debian: &base
    hostname: debian
    entrypoint: bash
    working_dir: /workspace
    volumes:
      # optional, but required for docker-in-docker
      - ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
      # standard, so tools can access project files
      - ${PWD}:/workspace
    build:
      context: .
      dockerfile_inline: |
        FROM ${IMG_DEBIAN_BASE:-debian:bookworm}
        RUN apt-get update -qq && apt-get install -qq -y make procps 

  ubuntu: 
    <<: *base
    hostname: ubuntu
    build:
      context: .
      dockerfile_inline: |
        FROM ${IMG_UBUNTU_BASE:-ubuntu:noble}
        RUN apt-get update -qq && apt-get install -y -qq make procps

  alpine:
    <<: *base
    hostname: alpine
    build:
      context: .
      dockerfile_inline: |
        FROM ${IMG_ALPINE_BASE:-alpine:3.21.2} 
        RUN apk add -q --update --no-cache coreutils build-base bash procps-ng

This example above is typical, but not necessarily minimal, so let's discuss a few things about it.

Regarding base images, there's a good chance your tool-container already includes make, and you don't need to add packages at all. But if you're looking for reasonable base-containers that include make and will help you avoid initial package updates with apt or apk.. you might consider things like golang containers, gcc containers, or debian/buildd:bookworm.

Next, a few tips and tricks about the docker-compose config details:

1. Use defaults-with-overrides in FROM lines to stay versioned but still keep some flexibility.
2. Use dockerfile_inline to quickly extend a base-container, without adding an extra Dockerfile to your repository. Tool-containers rarely change once established, and having a separate file and separate git/docker repos for everything can get tedious.
3. Use yaml inheritance with anchors, i.e. &base and <<: *base. This is a good trick to keep compose files DRY and as simple as possible.
4. Provide a few optional keys just for clarity. Things like hostname clean up logging, and even if entrypoint isn't used by software it can help humans who are trying to debug.
5. Correct config keys for working_dir and volumes are crucial for project file IO, which might include reading your project Makefile if you're using container dispatch.
6. It's optional, but there are cases where you'll want docker to be available inside tool-containers and you also might want to share the hosts docker-socket.¹ The boilerplate above goes half way, defaulting to using a volume mount but stopping short of actually installing docker.

Compatibility Notes

Platforms used in development for compose.mk include modern docker (say 25+), recentish make (3.8+) and bash (~5) on both Linux and MacOS. Testing done in CI only uses Linux, and won't try every possible combination of versions. Still, the goal is to support most things you'll encounter in the wild, including OSX, out of the box. But with MacOS you may see some of the usual problems with certain invocations to non-gnu OSX default sed / ps / xargs, etc. Please report issues!

Simple usage of compose.mk should work without problems even using BSD tools instead of GNU ones, but for some advanced usage you'll probably encounter problems with MacOS's default awk.

# Install a GNU awk
$ brew install gawk
$ cp /usr/local/bin/gawk /usr/local/bin/awk

Typically local-bin comes before /bin in PATH, so this should leave your base installation untouched but still prefer gawk by default. Opting in to GNU tools on MacOS everywhere is a matter of taste, but might be worthwhile. See documentation for example here and here.

For Developers

For development on compose.mk itself, here's how you can clone the project, build related containers, and run the test suites.

# Clone with SSH
$ git clone git@github.com:Robot-Wranglers/compose.mk.git

# Or clone with HTTP
$ git clone https://github.com/Robot-Wranglers/compose.mk.git

# Pull tool containers to do some early caching
$ make clean build 

# Run all demos (in Makefile)
$ make demos

# Run only CMK-lang demos
$ make demos/cmk

Forks & Versioning

Given the advice above to simply embed compose.mk directly into your project repositories, some people might be reading this and thinking what if compose.mk changes upstream?

Eventually compose.mk will probably get a pypi installer option ², although of course that won't help much if you're not already in the Python ecosystem, and it does undermine the goal of minimalism.

For now, if you're really worried about this, you can always setup the compose.mk repository (or your fork of it) as a git submodule inside your project folder, but there's probably no need. Fork-and-forget is a reasonable strategy here, and removing dependency problems rather than causing new ones is the goal. For more discussion, see the section on Contributing.

References