Compiler & Dialects
To avoid freaking out people who are just interested in tools or libraries, the fact that compose.mk is actually a programming language isn't emphasized in most docs, and the shocking secret truth is safe to ignore. If you are interested though, this section is hands-on with the language itself, and enough details to extend it.
Quick Version
- The
compose.mk
language is called CMK or CMK-lang. - Files use the
.cmk
extension by convention. - CMK is a superset of Makefile, compiled into legal classic Makefile.
- Makefile is always valid CMK; the reverse might not be true.
- The compiler is contained inside
compose.mk
, and there are no other tools. - The compiler is extensible, enabling new CMK dialects with user-defined config.
You could say that CMK = Makefile + compose.mk standard library + optional extra syntax
.
Crucially, writing code in the CMK language does not add new core capabilities to the stuff that compose.mk
already does without CMK. At a high level, the point of CMK is to eliminate lots of pesky dollar-signs and curly brackets in pure Makefile, while it also elevates various existing idioms for containers and polyglots to native syntax.
Whether you love make
or passionately hate it, you'll probably agree that default syntax can be very annoying. Since compose.mk
fixes most of the other problems with practical general-purpose usage, syntax is the main the problem left! This actually matters a lot, not just for adoption or usage as a daily driver, but because notation is part of how we think.1
About CMK Demos
All examples written in CMK rather than pure Makefile can be found in the demos/cmk folder. Most CMK demos are direct ports from existing pure-Makefile demos, i.e. given demos/cmk/foo.cmk
there's a corresponding vanilla Makefile demo at demos/foo.mk
.
# Run some "foo" demo directly, using CMK interpreter via shebang
$ ./demos/cmk/foo.cmk
# Run demo explicitly using interpreter
$ ./compose.mk mk.interpret! ./demos/cmk/foo.cmk
# Test the compiler without running the result
$ cat ./demos/cmk/foo.cmk | ./compose.mk mk.compiler
Basic Example
Let's revisit the JSON emit/consume example that's part of the structured IO docs. The original code was simple, but rewriting this in CMK is much more pretty:
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of demos/structured-io.mk
emit:
🡄 key=val
consume:
🡆 .key
__main__:
this.emit | this.consume
cmk.log.target(hello world!)
We'll get to the odd arrow syntax at the end of this section, but start with most basic stuff first.
- New Interpreter:
- This example introduces mk.interpret! in a shebang. Not to be confused with mk.interpret, mk.interpret! includes additional steps for CMK->Makefile transpilation, allowing us to extend the base syntax and use the more exotic stuff you see in the rest of the file. As with other demos, this one is executable directly as
./demos/cmk/structured-io.cmk
. Unlike other demos, we can't run it directly withmake -f ..
, and would use instead./compose.mk mk.interpret! ..
- Implied Include + Standard Library:
- When using the interpreter, a line like
include compose.mk
is always implied, and of course this brings with it full access to the standard library. - Function Calls Rewritten:
- Awkward stuff like
$(call log.target, ...)
can be written ascmk.log.target(..)
instead. Additionally, although we don't use it here, compose.import statements previously written as$(call ..)
can be rewritten as simplycompose.import(..)
- Recursive Calls Rewritten:
- Awkward stuff like
${make} foo
can be written asthis.foo
instead. Recursive invocation of the current context is common, and absolutely critical for code-reuse.3 The choice ofthis
for a new keyword is actually configurable too.. we'll get to that in the next section.
Finally, about that arrow syntax? These operators just help to make JSON IO feel more native, and are sugar to safely use jq
and jb
for parsing and pushing JSON, respectively, regardless of whether those tools are available on the host. If you glanced at the original code that's "ported" to CMK-lang here, then you might notice that the only code-transformation rules we really need to get from CMK -> Makefile in this case are just very simple string substititions*.
The next section explains the specific substitions that enable the code above at the same time as explaining "dialects" in general.
About Dialects
A dialect is just an array of paired substitition rules, and during compilation these rules take effect everywhere except for inside define-blocks. Using the reflection capabilities, we can show some info about the default dialect that the compiler supports like this:
$ ./compose.mk mk.def.read/cmk.default.dialect
[
["🡆", "${stream.stdin} | ${jq} -r"],
["🡄", "${jb}"],
["this.", "${make} "]
...
]
Matching/substitution for dialects is strict and simple. Regexes are not supported, and a preference for unicode symbols is an easy way to guarantee that new keywords will not collide with any other usage. But note that since substitution does not apply inside define-blocks, a new keyword like this
cannot not effect polyglots, even if they also use the same keyword. You only need to think carefully about collision with shell-commands and their arguments that appear inside normal targets.
Full dialect details are available in the appendix, but the rules shown above are the ones relevant to the basic example we started with: all we've done so far is introduce new syntax for common stuff with jq
and jb
and calling make
recursively.
It might not be obvious how useful and powerful that simple substition actually is here, so let's take a brief detour to peel back the layers. Above is the dialect literal, but below you can see a first approximation of variable expansion.
As discussed in structured IO docs, we try to use local tools if available, and fall back to docker.
$ ./compose.mk mk.get/cmk.default.dialect
[
[
"🡆",
"cat /dev/stdin | /usr/bin/jq -r"
],
[
"🡄",
"docker container run --rm ghcr.io/h4l/json.bash/jb:${JB_CLI_VERSION:-0.2.2}",
],
[
"this.",
"make -rs --warn-undefined-variables -f ./compose.mk "
],
...
]
One layer of expansion reveals that fallback works as advertised; compose.mk
knows that a local copy of jq
is available already, but jb
is missing and will have to use docker.
It also reveals that there is actually yet another layer of expansion, where JB_CLI_VERSION
has a default value for tool versions that supports user-override. Therefore besides being visually easy to parse and clarifying intent, the usage of the arrow operators results in portable code with no extra tool installation.
Compose File Example
Revisiting the code from the inlined compose-file demo, here's what it looks like ported to CMK.
Like the original demo, this embeds a compose file, creates target-scaffolding for the containers involved, and demonstrates task-dispatch for them. But this time we introduce the ⋘ name ..body.. ⋙ syntax for compose files. (Unicode ahead: ⋘
isn't <<<
!)
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of demos/inlined-composefile.mk
# USAGE: ./demos/cmk/inlined-composefile.cmk
⋘ inlined.services
services:
alice: &base
hostname: alice
build:
context: .
dockerfile_inline: |
FROM ${IMG_DEBIAN_BASE:-debian:bookworm-slim}
RUN apt-get update -qq && apt-get install -qq -y make procps
entrypoint: bash
working_dir: /workspace
volumes:
- ${PWD}:/workspace
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
bob:
<<: *base
hostname: bob
build:
context: .
dockerfile_inline: |
FROM ${IMG_ALPINE_BASE:-alpine:3.21.2}
RUN apk add -q --update --no-cache \
coreutils build-base bash procps-ng
⋙
# A private target, intended to run inside containers
self.internal_task:; cmk.log(Running inside ${red}`hostname`)
# Dispatch demo:
# Runs the private target inside containers, using
# syntax that looks like function-invocation.
__main__: \
alice.dispatch(self.internal_task) \
bob.dispatch(self.internal_task)
alternate_style1:
@# Equivalent to above, using target-body instead of prereqs;
@# For this scenario, using the `this.` prefix is required.
this.alice.dispatch(self.internal_task)
this.bob.dispatch(self.internal_task)
# More "classic" style in pure Makefile.
# Equivalent to __main__, no target body required.
alternate_style2: \
alice.dispatch/self.internal_task \
bob.dispatch/self.internal_task
# More "classic" style in pure Makefile.
# Equivalent to alternate_style1, using target body.
alternate_style3:
@# Equivalent to style1, just trading `${make}` for `this`.
${make} alice.dispatch/self.internal_task
${make} bob.dispatch/self.internal_task
Most of the differences with the original demo are pretty obvious, and alternate equivalents are shown to make what's happening more clear, but let's walk through it.
Closer Look
- As in the basic example,
include compose.mk
is implied. - We skip the
compose.import.string(..)
the original code used to create target scaffolding for services - After declaration of services with
⋘ name ..body.. ⋙
target handles are available immediately. - We replaced a more naive "echo ..." command with a call to
cmk.log
that's using color and stderr.
Most interestingly though, <container_name>.dispatch(<target_name>)
now looks like a normal function invocation. Also notice that alternate_style1
shows that double-rewrite rules for transpiler substition works the way you'd expect.
Compose-Context & Bind-Script
The last example inlines a compose file, and then dispatches targets. How about using an external compose file, then running script-dispatch with a tool-container? This example introduces two new idioms that can help. (Unicode ahead: ᝏ
isn't @
!)
Below, examples for script1.sh
and script2.sh
are equivalent, but use two different idioms:
#!/usr/bin/env -S ./compose.mk mk.interpret!
#
# CMK-lang equivalent of demos/bind-script.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/cmk/bind-script.cmk
compose.import(file=demos/data/docker-compose.yml)
# Script-in-container using With/As-idiom
⨖ script1.sh
echo hello container `hostname`
⨖ with alpine as compose_context
# Script-in-container using Decorator-Idiom
script2.sh: ᝏcompose.bind.script(svc=debian)
define script2.sh
echo hello container `hostname`
endef
# Target scripts always run from the container, regardless of
# whether they are called from the host or from inside the container.
__main__: \
script1.sh alpine.dispatch/script1.sh \
script2.sh debian.dispatch/script2.sh
So why have two idioms if they are equivalent? Well, both of these turn out to be pretty generic. Since we'll encounter other variants later, it makes sense to introduce them side by side for some compare and contrast.
-
The decorator-style idiom with
compose.bind.script
is most explicit, and one of several possible bind declarations that CMK supports. This actually looks like a traditional Makefile target, but also adds a piece that looks like a python decorator. -
The ⨖ / with / as idiom is a more generic form. In this case since we're working with containers defined by compose, we use
compose_context
for the as-clause. But⨖
will come up again, because depending on the contents of the with-clause, it can work with any script, use any interpretter, and run with containers that aren't managed by docker compose.
Although we've switched to an external compose file rather than embedded one, both forms work with inlined services too.
Choosing between bind-declarations vs with / as is sometimes just a matter of taste, but there are some practical considerations too. One advantage of using bind-declarations is that they can be stacked. Meanwhile, the with/as idiom is easier to refactor later, for example if you're moving from compose-backed containers in external files to inlined-Dockerfiles, etc.
Supported arguments are the same for both the decorator style and the with/as style. Both forms optionally support forwarding environment variables, sending output to stderr instead of stdout, etc. See here 5 for additional explanation re: env
and quiet
.
Name | Required? | Description | Example |
---|---|---|---|
svc |
✅ | Service / container to use. | debian |
entrypoint |
❌ |
Interpreter to use.
Defaults to bash. |
entrypoint=bash |
env |
❌ |
Variables to pass through to container.
Defaults to value from environment |
env='foo bar' |
quiet |
❌ |
Whether to silence announcements re: compose-file and service name.
Defaults to value from environment, or 0. |
quiet=1 |
See this demo for a concrete example using multiple arguments.
Compose-Context & Bind-Target
Quick-and-dirty scripting is often convenient, but working with targets makes it much easier to leverage composition. For this you can use compose.bind.target
.
This idiom binds together a given container, a "public" target and a "private" target. The public-target is the main interface; the "private" target has the same name but uses a "self." prefix and always runs inside the named container.
#!/usr/bin/env -S ./compose.mk mk.interpret!
#
# CMK-lang equivalent of demos/bind-target.mk & demos/bind-script.mk,
# demonstrating some idioms similar to function-decorators.
#
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/container-dispatch
#
# USAGE: ./demos/bind-target-2.mk
# Create scaffolding for debian container
compose.import(file=demos/data/docker-compose.yml)
# Decorator Idiom: Target-in-container
target_dispatch: ᝏcompose.bind.target(debian)
self.target_dispatch:
echo hello container `hostname`
# Main entrypoint: Drive the test-cases, ensuring
# that target still works whether it is called from
# the host or from a container.
__main__: target_dispatch debian.dispatch/target_dispatch
Again we import services here instead of inlining them, but compose.bind.target
works in either case.
Supported arguments for compose.bind.target
include the ability to optionally forward environment variables, sending output to stderr instead of stdout, etc. See here 5 for additional explanation re: env
and quiet
.
Name | Required? | Description | Example |
---|---|---|---|
1st pos arg | ✅ | Service-name / Where to run target | debian |
prefix |
❌ | Prefix used to find private target. Defaults to "self." |
prefix=. |
env |
❌ |
Variables to pass through to container.
Defaults to value from environment |
env='foo bar' |
quiet |
❌ | Defaults to value from environment, or 0 | quiet=1 |
About Sugar
Under the hood, examples using with/as involve something slightly more complicated than the dialects we saw earlier, and this is called "sugar".
Sugar is an array of triples representing [sugar_start, sugar_end, sugar_template]
. Similar to dialects, we can dump the default sugar for the compiler by using mk.def.read
. A complete reference for sugar is in the appendix, but the part that's relevant for inlined-compose files is this:
$ ./compose.mk mk.def.read/cmk.default.sugar
[
["⋘","⋙","$(call compose.import.string, def=__NAME__ import_to_root=TRUE)"],
...
]
Sugar transforms CMK source towards legal Makefile define-blocks while giving those blocks additional semantics. Since define-blocks have names, every sugar also requires a name.
For the sugar_template
element of the triple, we render it and add it after the define-block. See the table below for an explanation of the variables that may appear inside this template.
Name | Description |
---|---|
__NAME__ |
The sugar-block's name |
__WITH__ |
The contents of the with-clause, if one was used. |
__AS__ |
The contents of the as-clause, if one was used. |
__REST__ |
Everything that comes after `sugar_end` up to newline [^4] |
Polyglot Examples
CMK particularly shines at cleaning up the idioms you can see in the polyglot index, and almost every example is more clear and concise.
Foreign Code Objects
The example below uses another form of with .. as and introduces different syntax for bound vs unbound code, i.e. whether or not an interpreter has been declared.
Bound code uses the syntax ⟦ name ..body.. ⟧ with .. as .., whereas unbound code uses 🞹 name ..body.. 🞹
. Note that both syntaxes use unicode specials, and should not be confused with normal square-brackets or asterisks!
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of demos/code-objects.mk
# USAGE: ./demos/cmk/code-objects.cmk
# Main entrypoint, showing how to preview and run embedded code.
__main__: \
hello_world1.preview hello_world1.run \
hello_world2.run \
hello_world3.run \
hello_world4.preview
# Code bound to a containerized interpreter using literals.
elixir.img=elixir:otp-27-alpine
elixir.interpreter=elixir
⟦ hello_world1
import IO, only: [puts: 1]
puts("elixir World!")
System.halt(0)
⟧ with ${elixir.img}, ${elixir.interpreter} as container
# Code bound to a containerized interpreter using variables.
# In this case, you can accept user-overrides from env-vars.
export python_interpreter=python
export python_img?=python:3.11-slim-bookworm
⟦ hello_world3
print('hello world 2')
⟧ with $${python_img}, $${python_interpreter} as container
# Code bound to a target-as-interpreter.
# Target must be parametric, then do something with the parameter!
⟦ hello_world2
hello world2 !
⟧ with my_interpreter as target
# A custom interpreter, as used above.
# This always takes a file argument as a parameter:
# we do nothing except for displaying the file-contents.
my_interpreter/%:; cat ${*}
# Unbound code block: only `.preview` target is available
# unless you bind something else to `.interpreter` later
🞹 hello_world4
print('hello world 4')
🞹
The eagle-eyed reader will have noticed that our friend ⨖ / with / as is generic enough to already handle the use-case of working with polyglots by passing img=.. entrypoint=..
. Why add another syntax?
There are some differences, including the scaffolded preview
target. But the main difference is that this approach to polyglots is intentionally pretty minimal.
-
Using
.. as target
tends to result in designs that could use a container, but doesn't strictly require one, and usually makes things more friendly for composition. And for example this allows you to use host tools if available and fall-back to containers, or make dynamic decisions about the image to use. -
Using
.. as container
only supports container/interpreter arguments. Amongst other things, this disables support for environment-variable passthrough, and therefore encourages only pipe-based I/O if any is required.
Files as Targets with polyglot.import.file
For larger scripts, you'll want to use external files. Binding a file, container, and interpreter is a typical use-case.
#!/usr/bin/env -S ./compose.mk mk.interpret!
#
# CMK-lang equivalent of demos/bind-script.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/cmk/bind-file.cmk
kwargs=\
file=demos/data/test-file.py \
img=python:3.11-slim-bookworm \
entrypoint=python
__main__: ᝏpolyglot.bind.file(${kwargs})
As an alternative to binding to an existing target, you can use an import statement, and create a target more dynamically. This example also demonstrates passing a few more of the supported arguments:
#!/usr/bin/env -S ./compose.mk mk.interpret!
#
# CMK-lang equivalent of demos/import-file.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/cmk/import-file.cmk
export foo=val1
export bar=val2
kwargs= \
file=demos/data/test-file.py \
img=python:3.11-slim-bookworm \
entrypoint=python cmd='-O' \
namespace=testing \
env='foo bar'
polyglot.import.file(${kwargs})
__main__: testing
Supported arguments for the bind-declarations and the import statements are mostly the same, but as you can see above, import requires an additional namespace
argument to determine the name of the target that will be created.
Name | Required? | Description | Example |
---|---|---|---|
file |
✅ | Filename | relative/path/to/file.py |
namespace |
✅ | Prefix used to find private target. Defaults to "self." |
prefix=. |
img |
✅ | Defaults to value from environment, or 0 | quiet=1 |
entrypoint |
✅ | Defaults to value from environment, or 0 | quiet=1 |
env |
❌ |
Variables to pass through to container.
Defaults to value from environment |
env='foo bar' |
cmd |
❌ |
Variables to pass through to container.
Defaults to value from environment |
env='foo bar' |
Host-context with compose.import.script
The approach for running a shell script on the host is slightly different, and compose.import.script
becomes built-in:
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang equivalent of demos/script-dispatch-host.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/cmk/script-dispatch-host.cmk
# Look, here's a simple shell script
define script.sh
set -x
printf "multiline stuff\n"
for i in $(seq 2); do
echo "Iteration $i"
done
endef
# Directly imports script to target of same name.
compose.import.script(def=script.sh)
__main__: script.sh
Host scripts inherit the parent environment automatically. This includes any variables passed or exported from the command line, as well as any variables exported globally in the main script.
Dockerfile Example
Let's revisit the code from the inlined Dockerfile demo, and port it to CMK. (Unicode ahead: ⫻
is not ///
!)
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of demos/inlined-dockerfile.mk
# USAGE: ./demos/cmk/inlined-dockerfile.cmk
⫻ Dockerfile.my_container
FROM alpine:3.21.2
RUN apk add -q --update --no-cache coreutils build-base bash procps-ng
⫻
__main__: test.scaffold test.runner test.raw
test.scaffold: \
my_container.build \
my_container.dispatch(self.dispatch_test)
self.dispatch_test:
cmk.log(Testing target from inside the inlined-container)
cmk.log(host details: `uname -a`)
test.runner:
cmk.log.test_case(Low level access to run commands)
entrypoint=sh cmd='-c "pwd"' this.my_container.run
test.raw:
cmk.log.test_case(Low-level unmediated access to image)
docker image inspect ${my_container.img} > /dev/null
docker run --entrypoint sh ${my_container.img} \
-x -c "true" > /dev/null
It's much cleaner than the plain Makefile now. Helper targets for dispatch and build are created automatically, and data like my_container.img
is also scaffolded.
Tip
Notice that the Dockerfile.
prefix as part of the block-name is required for declaration, but afterwards reference to the image/container won't use it.
Local-Context & Embedded Containers
The last example uses some raw docker commands just to show it's possible, but in most cases you'll want tighter integration that feels more "native". For this we can leverage the generic ⨖ / with / as idiom again, this time using with local_context for the as-clause.
Here's two examples, showing minimal usage first, then demonstrating passing some extra arguments.
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang equivalent of demos/script-dispatch-custom.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/script-dispatch-custom.mk
⫻ Dockerfile.my_container
FROM debian/buildd:bookworm
# .. other customization here ..
⫻
⨖ example-1.sh
echo hello `hostname` at `uname -a`
⨖ with my_container as local_context
export var1=hello-world
⨖ example-2.sh
echo variable exported: ${var1:-not set}
⨖ with img=my_container env='var1' quiet=1 as local_context
__main__: example-1.sh example-2.sh
Supported arguments for local_context
are described below. See here 5 for additional explanation re: env
and quiet
.
Name | Required? | Description | Example |
---|---|---|---|
img |
✅ |
Image to use.
Defaults to 1st positional-arg if kwargs not present. |
img=... |
def |
✅ |
Name of code-block.
Implied for CMK ⨖-syntax |
def=.. |
entrypoint |
❌ |
Interpreter to use.
Defaults to bash. |
entrypoint=bash |
cmd |
❌ |
Arguments to pass to interpreter.
Defaults to empty string. Filename is post-fixed to command. |
cmd=-x |
env |
❌ |
Variables to pass through to container.
Defaults to value from environment. |
env='foo bar' |
Docker-Context & Stock Containers
Using stock images instead of inlined ones is similar to the last section, except that it uses docker_context
in the as-clause.
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang equivalent of demos/script-dispatch-stock.mk.
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# See also: http://robot-wranglers.github.io/compose.mk/compiler
#
# USAGE: ./demos/cmk/script-dispatch-stock.cmk
⨖ my_dockerized_script
echo hello `hostname` at `uname -a`
⨖ with img=debian/buildd:bookworm as docker_context
__main__: my_dockerized_script
Supported arguments for docker_context
are exactly the same as the ones we saw earlier with local_context
.
User-Defined Extensions
Both dialects and sugar can be defined at runtime on a per-file basis, basically allowing the code itself to describe aspects of the compiler that will be used on it.
Info
File hints must occur after the shebang, before the first non-comment line, and must be legal JSON that specifies the full dialect/sugar to use. This overwrites the default dialect/sugar, and does not append it!
Minified JSON is optional, and everything between the first and second :::
will be used, so multi-line JSON is allowed, but must remain comment-prefixed.
User-Defined Dialects
User-defined dialects require a hint at the top of the file. Building on the basic example, here's an example that changes the default jq/jb JSON operators to use different characters.
#!/usr/bin/env -S ./compose.mk mk.interpret!
# Demonstrating user-defined extensions for the CMK -> Makefile compiler.
#
# cmk_dialect ::: [
# ["⏩️", "${stream.stdin} | ${jq} -r"],
# ["⏪️", "${jb}"], ["this.", "${make} "] ] :::
#
emit:
⏪️ key=val
consume:
⏩️ .key
__main__:
this.emit | this.consume
To get the full current default dialect in a minified form that's ready for your own modifications, use ./compose.mk mk.def.read/cmk.default.dialect| jq -c
User-Defined Sugar
User-defined sugar requires a hint at the top of the file. Building on the compose file example, here's an example that changes that default syntax to use different characters.
#!/usr/bin/env -S ./compose.mk mk.interpret!
# Showing user-defined rules for CMK -> Makefile transpilation.
# You can define cmk_dialect and cmk_sugar individually or together.
#
# cmk_dialect ::: [
# ["cmk.io.","${make} io."],
# ["this.","${make} "]
# ] :::
# cmk_sugar ::: [
# ["𒀹","𒀺","$(call compose.import.string, def=__NAME__ import_to_root=TRUE)"]
# ] :::
#
# USAGE: ./demos/cmk/user-sugar.cmk
𒀹 inlined.services
services:
alice: &base
hostname: alice
build:
context: .
dockerfile_inline: |
FROM ${IMG_DEBIAN_BASE:-debian:bookworm-slim}
RUN apt-get update -qq && apt-get install -qq -y make procps
working_dir: /workspace
volumes: ['${PWD}:/workspace']
bob:
<<: *base
hostname: bob
build:
context: .
dockerfile_inline: |
FROM ${IMG_ALPINE_BASE:-alpine:3.21.2}
RUN apk add -q --update --no-cache \
coreutils build-base bash procps-ng
𒀺
# A task that runs inside the containers.
self.internal_task:; cmk.log(Running inside ${red}`hostname`)
__main__:
cmk.log(Testing dialect substitutions)
cmk.io.print.banner/testing_dialect_substitution
cmk.log(Testing sugar and dispatching task to containers)
this.alice.dispatch(self.internal_task)
this.bob.dispatch(self.internal_task)
To get the full current default sugar in a minified form that's ready for your own modifications, use ./compose.mk mk.def.read/cmk.default.sugar | jq -c
Bind Declarations
Bind-declarations were introduced in the compose file example, but they show up in a few places. You can think of them as a type of "target decorator", similar to function-decorators in python. Bind declarations in CMK-Lang use ᝏ
(not @
!) for the usual reason of avoiding collisions with other languages and to simplify the compiler.
One use-case of bind-declarations is to bind variables to values, basically providing support for keyword-arguments, as seen elsewhere.
The equivalent idiom is much more readable in CMK-lang:
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of ./demos/kwarg-parsing.mk
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# USAGE: ./demos/cmk/kwarg-parsing.cmk
emit:
🡄 shape=triangle color=red
consume: ᝏargs.from_json(shape color=blue name=default)
printf "shape=$${shape} color=$${color} name=$${name}\n"
__main__:; this.emit | this.consume
Stacking Bind Declarations
Bind-declarations can be stacked. For example, loading some variables from JSON and some from the environment looks like this:
#!/usr/bin/env -S ./compose.mk mk.interpret!
# CMK-lang translation of ./demos/kwarg-parsing.mk
# Part of the `compose.mk` repo. This file runs as part of the test-suite.
# USAGE: ./demos/cmk/kwarg-parsing.cmk
export color=green
emit:
🡄 shape=square
consume: \
ᝏargs.from_json(shape=default1) \
ᝏargs.from_env(color=default name=Bob)
@# Comments in the middle are ok.
printf "shape=$${shape} color=$${color} name=$${name}\n"
__main__: flux.pipeline/emit,consume
Status Info, Limitations, & Other Advice
Compilation and interpreted CMK is even more experimental than the rest of the project! However if you're nervous about upstream changes due to depending on the default dialect, one option is to just make it the user-defined default.
That said, everything that's documented here is stable and unlikely to change. Mainly experimental just means that testing so far is limited, so please report bugs, but not until you've confirmed you're using GNU awk!.
Per the rest of the design philosophy, the goal is to freeze CMK after it's useful and capable of being extended and most future changes are probably optimizations rather than extensions. See also the docs re: Forks & Versioning and Contributing.
Appendix
Other related documentation includes: The more basic interpreter usage, which allows for inheritance of the entire compose.mk
standard library and signals, but skips compilation and does not allow syntactic extensions. The packaging support generates executables, but isn't compilation. And there's also a few thoughts about style.
Data below is rendered from the latest source code.
Appendix: Default Dialect
$ ./compose.mk mk.def.read/cmk.default.dialect
[
["ᝏ","; cmk.bind."], ["ᝏ ","; cmk.bind."],
["⧐", ".dispatch/"],
["🡆", "${stream.stdin} | ${jq} -r"],
["🡄", "${jb}"],
["this.", "${make} "]
]
Appendix: Default Sugar
$ ./compose.mk mk.def.read/cmk.default.sugar
[
["⋘", "⋙", "$(call compose.import.string, def=__NAME__ import_to_root=TRUE)"],
["⫻", "⫻", "$(call dockerfile.import.string, def=__NAME__)"],
["⟦", "⟧", "$(call polyglot.__import__.__AS__,__NAME__,__WITH__)"],
["🞹", "🞹", "$(call compose.import.code, def=__NAME__)"],
["⨖", "⨖", "__NAME__:; $(call __AS__,__WITH__)"]
]
References
-
Note that during compilation for CMK code, this isn't just a naive a find/replace, because we do have to leave define-blocks alone ↩
-
Actually
compose.mk
takes recursive invocation so seriously that a significant effort is invested in fixing problems with${MAKE}
. This results in the less shouty usage of${make}
, which is also more robust, reliable, and predictable. Incompose.mk
source code, there are more than 250 usages of${make}
, but extensions can avoid the pain. Literally ifself
was${self}
andthis
was${this}
everywhere, then the resulting RSI would be a public health crisis! ↩ -
Line-feeds are allowed. ↩
-
For multiple values with
env
, items must be single-quoted and space-delimited. As forquiet
, pass 1 or 0 to control whether the target will run silently or announce context (like the compose filename/service, or the docker image being used). ↩↩↩