Language Overview

You might already be aware that compose.mk offers a standard library for working with make, a sort of stand-alone tool mode, and a TUI framework, but now you've reached the page that will finally reveal the shocking secret truth: compose.mk is actually a programming language! 😮 More specifically, compose.mk is a matrioshka language, and maybe the first one that's actually intended to be useful.

The rest of this page focuses on other language-related features and serves as an alternate to the main landing page. If you prefer overviews to focus less on applications and more on theory, architecture, and patterns, you've come to the right place.

Related Links: See here, here, here if you're looking for concrete examples. For a discussion more focused on patterns, see the matrioshka automata page to learn about packing programs inside of programs.

Language Properties

As a language, you could say that compose.mk is multi-paradigm and has the following properties:

  1. Inherits macros, declarative style, incremental computing1, and basic DAGs2 from make
  2. Inherits tacit programming3 and a preference for pipes & streams from the unix shell
  3. Very untyped, but some support for structured data and file and stream kinds
  4. Supports extensible programming5
  5. Supports functional programming6 and data flow programming7
  6. Supports reflective programming4 and lightweight aspect-oriented programming10
  7. Inclines towards concatenative style8 and stack-oriented programming9
  8. Supports modules and something like OOP implicitly
    • via container layers, or via yaml inheritance in composefiles
    • via make-includes, via target-chaining, via namespacing conventions for targets, etc

Further, you could say that compose.mk is an interpreted language that is implemented in make, but in some sense it also runs on make by hijacking it as a kind of gadget.

Also related but harder to categorize, the packaging support is almost a language feature in that it effectively allow us to release frozen artifacts. For a simpler approach to packaging that doesn't require archives, see instead the documentation for guests and payloads. These capabilities are inspired by the idea of image-based persistence11.

Homoiconic?

Let's introduce a somewhat surprising fact: compose.mk is designed to be homoiconic on make targets, at the same time as it generalizes what targets can be. Thus among other things, a compose.mk target can:

  1. Execute directly (default behaviour for make)
  2. Group other tasks into DAGs (default behaviour for make)
  3. Serve as a handle for a container's default entrypoint (via compose.import)
  4. Execute inside of containers (via container dispatch)
  5. Execute later, conditionally, or repeatedly etc (via flux)
  6. Describe data-flow and/or control-flow across other tasks (again via flux)
  7. Execute arbitrary tasks in foreign languages (via polyglots)

In the workflow docs, the flux.* module is introduced as a target algebra, and compared to more familiar shell process-algebra. But as a simple example of "targets all the way down" consider this:

  • flux.wrap/<targets> is a n-ary target that accepts nullary targets as arguments.
  • Meanwhile.. flux.ok and flux.fail are examples of nullary targets.
  • But invocation, like say flux.or/flux.ok,flux.fail, is itself also a (nullary) target.

Of course, no matter how easily you can incorporate foreign tools.. at some point, you might also need to write task implementations rather than just glue, i.e. actual target bodies in shell, or polyglots in foreign languages. Still.. you'll probably be surprised by how infrequently that's actually necessary, and when it is, how small the task-implementations you need actually are. For a quick example of how much you can accomplish with no target bodies at all, see also the docs for low config TUIs.

Interpreters & Transpilation

By the way, since the claim is that something like flux.wrap/flux.ok,flux.fail is actually a program in a "language" though, it's worth pointing out that there are many ways to execute it beyond just mentioning it in a Makefile as a target-prerequisite. The following are all equivalent, but the last example demonstrates what you can think of as the compose.mk interpreter.

# program as target
$ make -f compose.mk flux.or/flux.ok,flux.fail

# program as subcommand
$ ./compose.mk flux.or/flux.ok,flux.fail

# program as code
$ echo flux.or/flux.ok,flux.fail | ./compose.mk mk.kernel

This example is simplistic, but there are other ways that compose.mk can act as an interpreter, and even support transpilation for a syntactically different language that transforms to classical Makefile. For more details, see the Compiler & Dialects docs.

References

  1. https://en.wikipedia.org/wiki/Incremental_computing ↩

  2. https://en.wikipedia.org/wiki/Directed_acyclic_graph ↩

  3. https://en.wikipedia.org/wiki/Tacit_programming ↩

  4. https://en.wikipedia.org/wiki/Reflective_programming ↩

  5. https://en.wikipedia.org/wiki/Extensible_programming ↩

  6. https://en.wikipedia.org/wiki/Functional_programming ↩

  7. https://en.wikipedia.org/wiki/Dataflow_programming ↩

  8. https://en.wikipedia.org/wiki/Concatenative_programming_language ↩

  9. https://en.wikipedia.org/wiki/Stack-oriented_programming ↩

  10. https://en.wikipedia.org/wiki/Aspect-oriented_programming ↩

  11. https://en.wikipedia.org/wiki/Smalltalk#Image-based_persistence ↩