Overview

---

Meet compose.mk, a tool / library / framework for Makefile-based automation, scripting, and lightweight orchestration. Support for docker, docker-compose, workflow primitives, TUI elements, and more, all provided by a single file with no dependencies beyond what's already in your development environment.

Typical use-cases include general project automation, especially decoupling your CI/CD from different kinds of platform lock-in. Other superpowers include the ability to quickly incorporate foreign tools and foreign code as first-class objects, which provides unique and powerful capabilities for quickly assembling console applications, systems prototyping, and component-oriented design experiments in general. Definitely not the Makefiles of your ancestors.

---

Introduction

---

Whether compose.mk is a tool, a library, a language, or a framework depends on how you decide to use it.

For programmatic usage,

you can think of compose.mk as an attempt to provide a "standard library" that extends vanilla make. Besides advanced stuff like container support and workflows, it also offers basic stuff like clean, colored logging, and structured IO.

As a framework,

you can combine compose.mk with docker-compose file(s) or inlined container descriptions, and very quickly expose powerful, domain-specific automation APIs in your project Makefile to orchestrate tasks that might involve multiple containers. Dispatching make-targets inside these containers turns out to be an especially powerful technique, and is supported for most containers.

As a stand-alone tool,

no project-integration is required to use compose.mk, and it still does lots of practical stuff with no dependencies beyond make and docker. Because make targets always do double-duty as CLI entrypoints and as reusable task units, the "internal API" is automatically published as a CLI interface.

As a language,

there is support for working with foreign languages, and while it remains completely backwards compatible with classical Makefile, it also extends make, generalizes it, and adds new kinds of idioms and patterns. Additionally compose.mk can function as both a Makefile interpreter and as a transpiler— this allows you to potentially write code in CMK-lang— a syntactically distinct language that exposes the compose.mk library and primitives for containers and polyglots, and compiles to Makefile on the backend.

Not the Makefiles of Your Ancestors

---

Working with compose.mk makes make hit different. The demo above shows an example of combining docker support with TUI elements, using 3 different containers as "widgets".   (Under the hood, tmux is used as a geometry manager, but since it's also dockerized, it won't become a host dependency.)

Capabilities like interactivity, online help, and working with TUIs and GUIs all show a major break with traditional use-cases and create some attention-grabbing demos. You may be asking yourself: can it run DOOM? And it really kind of does. But making weird new kinds of application development easy is just one of the places where compose.mk shines, and average use-cases will probably look more like automation.

Taken together, the facilities for working with containers and polyglots suggest that compose.mk is best thought of as a new kind of meta-language. One that makes it easier to aggressively reuse existing tools, and to think of groups of tools as a single logical unit or cohesive ensemble.

Easy access to containers, multiple programming languages, and workflows are things that we usually associate with microservices and distributed systems like Kubernetes, Airflow, or Jenkins. What happens if it is suddenly very easy to design undistributed applications, tools, and pipelines with similar architectures? ¹

Features & Design Goals

---

Radically minimal, in terms of dependencies and conceptual overhead.

Out of the box, make is expressive but has relatively few core concepts, it's ubiquitous, and it's fast. Building on this, compose.mk adds just enough extra concepts to get a lot of extra power and flexibility.

Seamless and simple idioms for working with containers in a way that's platform agnostic.

Code that automates tasks in project management should not be locked inside Jenkinsfiles or Github Actions. Using compose.mk tends to help projects to decouple and run elsewhere, but afterwards running from CI/CD still works.

Generic and easy to extend foundation.

A major focus for compose.mk is what might be summarized as mapping, dispatch, and interoperability. For example you can map docker containers to make targets, or targets to containers, or map containers/targets into tmux panes. This makes it easy to quickly put together domain-specific toolkits, regardless of whether that domain happens to be wrangling development tools, documentation tools, or infracode tools.

Clean & carefully curated output

Beyond providing the missing facilities for logging with make, the general goal is to be readable and human-friendly on stderr, while still remaining machine-friendly for downstream processing on stdout.

Basic workflow support is also available.

Tasks and DAGs are built-ins for make, and compose.mk adds support for other primitives, including parallel tasks and task retries, conditional execution, pipelining, etc.

Hacker friendly.

The single-file approach used with compose.mk makes it easy to embed alongside existing projects. The project itself uses the unlicense, and the goal is for the base API to eventually become completely frozen. Rip out the parts you don't need, contribute upstream if you can, but if you don't want to track upstream or if your customizations become too specific to your project, no need to ever look back.

Choose Your Path

---

🚀 If you're interested and prefer to learn by example.. just dive into some.

🎨 If you're interested but prefer to have your overviews focus more on theory, patterns, and paradigms.. you might want to start with the language overview and the matrioshka automata docs.

🤨 If you are understandably skeptical about getting involved with the biggest, baddest, and most highly-powered mutant Makefile the world has ever seen, you might want to focus on stand-alone tool mode, or see how CMK as a language can help you move further away from classic Makefile syntax. You can also read long-format discussion about motivations and design philosophy.

🔎 If you are withholding judgement on just how insane this project is until you see concrete larger-scale use-cases.. skip directly to some of the larger demos. You can also check out a preview of the sibling project at k8s-tools ¹, which is still a work in progress, but more domain specific, and covers automation specifically for kubernetes cluster lifecycles.

---