The hostnames are already declared, once, in the fabio route tags. A
domains.txt would be a second source of truth free to disagree with what is
actually served, so myos cert reads the tags instead and decides on its own
which name needs a wildcard: one is asked for where a tag uses one, and it
absorbs the concrete names it covers. A wildcard covers a single label, so
a.b.example.org keeps its own certificate.
dehydrated issues them: a shell script, no python, which fits a tool that has
to install on any server. It answers http-01 itself on a port bound to the
loopback and routed by fabio, and delegates dns-01 to a provider hook. The
deploy hook writes the two file names fabio watches for, through a temporary
name so fabio never reads half a certificate.
Fixed on the way: the port parser wanted six spaces of indentation and the
catalogue writes four, so every stack that binds its ports was reported
unbound.
The variable declared an intent and changed nothing: it fed the report and
never the binding, so it could say public while the file published on the
loopback, or the reverse. A declaration that cannot be wrong is better than one
that has to be kept in step.
The scope is now which binding the file asks for, read from the compose files
as written: ${MYOS_BIND_PUBLIC} is public, a hand-written address is pinned,
and a bare '- 80' or '9000:9000' is unbound, meaning docker opens it on every
address and nobody chose that. --strict fails on those.
Reading the resolved configuration instead would lose the distinction: every
form ends up as a plain address, and an unbound port looks exactly like a
deliberate public one.
What is left to configure per host is the address of each scope, which is the
part that belongs to the host rather than to the stack.
The catalogue publishes 51 compose files with the bare form 'ports: [80]',
which binds a random host port on 0.0.0.0: every service answers the internet.
ufw-docker existed to take that back afterwards, as root, on linux only,
because docker writes its own firewall rules and ufw never sees those ports.
Publishing where you mean to solves it at the source. Verified against the
daemon: '- 80' gives 0.0.0.0:32768, '127.0.0.1::80' gives 127.0.0.1:32769.
Same on macOS and on linux, no privilege, and visible in docker ps.
A stack binds with ${MYOS_BIND_PRIVATE|PUBLIC|MESH} and declares what it
means with <PREFIX>_SERVICE[_<port>]_EXPOSE. myos expose reads the resolved
compose configuration and reports what would be opened; --strict fails when a
port faces the world without declaring it, which is what an agent runs against
a server it did not set up.
The usage listed recreate but the dispatcher did not know it: myos recreate
exited 2 saying the command was unknown. status was missing too, and -H was
parsed into a variable nothing ever reads, so it was accepted and ignored. It
now refuses rather than pretending.
myos export prints every setting the requested stacks declare, in one process:
68 variables in half a second, where asking for them one at a time costs a
process each and about ten seconds.
The shim reads that once and includes it, so a .mk target sees
HOST_FABIO_VERSION and the computed fabio tag without make knowing anything
about how they are produced. --make quotes the values so make neither expands
a $ nor starts a comment at a #.
- the wrapper followed one absolute symlink only, and fed the config file to
env(1) as-is, so a comment or a blank line in /etc/conf.d/myos made every
command fail with 'env: #comment: No such file or directory'
- verlt called verlte, which does not exist, and used return outside a
function: it never compared anything
- stat asked for the access time on macOS and the modification time elsewhere,
so newer/older did not mean the same thing depending on the machine
- setup-docker-group called ansible-user-add-groups, removed with ansible, then
announced that the user had been added to the docker group. It now runs
usermod or addgroup, and says so when neither works
- the ssh targets looped over AWS_INSTANCE_IP, which nothing defines since
make/apps/aws was dropped: they exited 0 having done nothing. They take
SSH_HOSTS and fail when it is empty
- patsublist left a space before the comma joining two fabio routes
- the JWT macro splits a payload on its commas; documented at the macro, since
fixing it would change every key it has already produced
- hooks load the _stack files of every directory between the stack path root
and the stack, outermost first: make included both $(dir)/*.mk and
$(dir)/*/*.mk, so a stack in a subdirectory saw its parent's settings
- a group may be declared in <group>/<group>.env, where the stack lives
- MYOS_STACK_DIR lets a hook read a file it ships next to itself
- myos_filter no longer confuses a literal * in a make pattern with a wildcard,
and the list helpers no longer let the shell expand a * into filenames
- MACHINE, SYSTEM, HOST and DOMAINNAME join the framework variables a hook sees
- the make shim gains $(call myos-var,NAME), so a .mk target can read a
setting that now lives in a hook, and it picks up the .mk of every stack
directory rather than only the project's
lib/str.sh gains firstword, lastword, or, patsubst, filter, filter_out,
addprefix, addsuffix and jwt: what the catalogue .mk files are written in.
The make JWT macro split on the commas of its payload; this one does not.
lib/context.sh registers COMPOSE_PROJECT_NAME, APP, DOCKER_NETWORK_* and the
rest as lazy defaults, so a converted .mk keeps reading them as it did.
Hooks also load a directory-level _stack.env and _stack.sh, for the .mk files
of the catalogue that hold settings for several stacks at once.
myos build up logs host/fabio runs the three in order and stops at the first
failure, the way make build up logs STACK=host/fabio did. Leading words that
name commands are commands; the first word that is not one starts the stacks.
A stack found in several directories of the stack path is now merged rather
than shadowed, least specific first, so a project drops
stack/postgres/postgres.local.yml next to the catalogue's postgres.yml and
refines it. Settings hooks follow the same order, so a project can redefine a
default the catalogue ships. Neither engine did this before: the project
directory simply hid the catalogue one.
An unknown command now says so and suggests the command to type, instead of
printing the whole usage.
Two mechanisms, matching what the make engine actually did:
Lazy defaults. A stack setting is a function myos_default_<VAR>, called only
when the variable has no value, and called again at every reference. That is
exactly a recursive ?=: an explicit value wins, and the default follows a
DOMAIN that a .env changes later. The prefix is what makes it safe; the first
version used a bare function named after the variable, and the test suite
caught it running /usr/bin/host for a stack group called host.
Templates. myos env-update fills a .env from the .env.dist files, expanding
${VAR} against the current values and running $(command), forward references
included.
Also fixed: the project .env now wins over /etc/conf.d/myos, which is what the
documentation claimed and the code did not.
share/make/shim.mk lets a project keep make as a front end: every myos command
becomes a target that shells out to bin/myos, and the project keeps its own
targets and its stack .mk files. It sits outside make/ because the legacy
engine globs every .mk in there.
A stack can now ship <name>.env and <name>.sh next to its compose files. The
hook is sourced with the tag helpers available, which is what the computing
.mk files of the catalogue were using make for: 29 of its 44 .mk files only
exist to build variables like the fabio tags.
Converting stack/host/fabio.mk by hand gives byte-identical output for the
route tag, and drops a trailing comma the make version left in the listener
list.
Also: APP_HOST and APP_URI are computed (the tag helpers build on them),
--color controls the escape codes rather than always emitting them, and
make test-portability runs the CLI under busybox ash and dash.
Setting IFS to a newline to pass the environment also stopped the command
line from splitting, so '-f a -f b' went out as a single argument and the two
words 'docker compose' were looked up as one program name. Every real command
was broken; only --dry-run, which prints a string, looked right.
The variables are now exported in a subshell instead of going through env(1),
which keeps values with spaces intact and leaves the command line to normal
word splitting.
The tests only checked what --dry-run printed, which is why they missed it.
spec/unit/compose_spec.sh now asserts the argv that is actually executed, for
both the plugin and the standalone binary.
An install under a custom prefix (--prefix ~/.local) could not find the
catalogue it had just cloned into <prefix>/share/myos: only the three system
paths were searched.
The skill documents how to use myos: what to check before touching a host
stack, how a reference resolves, what the three kinds of stack mean, and where
each failure comes from. Every command it shows was run against the fixtures
before being written down, which is how exec, run and scale turned out to be
missing.
AGENTS.md covers the other side: how to change myos without breaking the
deployments that already run it.
holcommon's empty .env made the loader evaluate one blank line, which
tripped set -u. Verified afterwards on holcommon's real host stack: the CLI
and the make engine resolve the same five files and render a byte-identical
241-line compose config.
Both engines now go through the same 68 cases. 31 produce byte-identical
output; the rest have a recorded CLI expectation and a reason in DELTAS.md,
the bulk of it being that the CLI does not shell out to a recursive make and
calls compose once per project.
Fixed while comparing: the CLI was missing the COMPOSE_FILE_* defaults, so
overlays such as supabase.labels.yml would not have been loaded at all.
A stack is a directory of compose files; the current directory is a stack when
it holds one. The same command works for a project, a catalogue stack, a group
and a host singleton, and stacks sharing a project are now a single compose
call.
Two traps of the make engine are closed on the way: an unknown command is an
error instead of a silent success, and a group is only expanded when its name
is lowercase, so an environment variable can no longer be mistaken for one.
That guard spells out its character class because a-z matches uppercase too
under a dictionary collation.
config: dotenv files are parsed, never sourced, so a value can hold a # or a
$(...) without breaking or executing. The make engine included .env with
make syntax, which neither allowed.
compose: one call per project instead of one per sub-stack.
Replaces the 500-character stack_path expression of
make/apps/def.docker.mk:159 and its leaked $(stackz) loop variable with
three readable functions, covered by 20 assertions.
Ported from make/utils.mk, make/def.mk, make/apps/def.docker.mk and
make/apps/def.mk, with the examples that sat as dead comments in
make/apps/def.mk turned into actual assertions.
Two deviations from the make macros are pinned and documented in DELTAS.md;
both only show up on code paths nothing uses.
stack/ and docker/ now live in the myos-stacks project (extracted with their
history). What the framework itself needs stays here:
- stack/myos/*.yml -> share/compose/
- docker/myos/ -> share/docker/myos/
- docker/compose/ -> dropped, docker compose >= 2.24.4 is now required
Residues fixed along the way: DOCKER_IMAGES scanned a hardcoded ./docker,
include.mk filtered a hardcoded stack/*.mk, and docker-image-myos expanded an
undefined MYOS_DOCKER_IMAGES on every up/build.
README and CHANGELOG rewritten: they still documented make host,
host-certbot-* and user-config, all removed when the catalogue was split out.
- spec/golden/cases.txt: 69 black-box cases (CLI wrapper mode and make
include mode) run in a hermetic sandbox with a mocked docker
- spec/golden/expected/*.txt recorded from this engine (baseline)
- make test / test-golden / golden-record / lint dev targets
181a3e9 wrapped both $(or) alternatives in a single $(realpath …), which
turned the argument separator into a literal comma glued to the last path.
host/<svc> stacks were only resolved when the project stack dir happened to
sort before the other STACK_DIR entries.
Yann Autissier