Compare commits

29 Commits
Author SHA1 Message Date
Yann Autissier 803dcd1e10 add the restart protocol
The rewrite starts over from the make engine: fix it, record its behaviour
target by target as golden tests, then rewrite in red/green. The first
attempt in lib/ is tagged attempt-1-lib and kept for its ideas only.
2026-09-05 15:45:53 +02:00
Yann Autissier ed3c5a0c7c add the engine benchmark: make, sh, just and a Go prototype
Same work on each, median of five. The numbers separate three costs that the
earlier measurements mixed up: the engine itself (go 23 ms flat, sh 177 ms
plus 63 per stack, just 204 plus 32, make 792 plus 700), the shell hooks
(about 40 ms per computed setting whatever the engine, since Go runs the same
sh), and bin/myos loading a directory's hooks once per stack reference rather
than once per directory, which doubles the hook cost for a group.

Memoising the lazy defaults changes nothing: the cost is the command
substitutions inside each tag helper, not repeated lookups.
2026-09-05 15:07:53 +02:00
Yann Autissier f429b8c38d derive the certificates a server needs from what it routes
The hostnames are already declared, once, in the fabio route tags. A
domains.txt would be a second source of truth free to disagree with what is
actually served, so myos cert reads the tags instead and decides on its own
which name needs a wildcard: one is asked for where a tag uses one, and it
absorbs the concrete names it covers. A wildcard covers a single label, so
a.b.example.org keeps its own certificate.

dehydrated issues them: a shell script, no python, which fits a tool that has
to install on any server. It answers http-01 itself on a port bound to the
loopback and routed by fabio, and delegates dns-01 to a provider hook. The
deploy hook writes the two file names fabio watches for, through a temporary
name so fabio never reads half a certificate.

Fixed on the way: the port parser wanted six spaces of indentation and the
catalogue writes four, so every stack that binds its ports was reported
unbound.
2026-09-05 13:41:16 +02:00
Yann Autissier be777fc9e6 read the exposure scope from the compose file, drop the _EXPOSE variable
The variable declared an intent and changed nothing: it fed the report and
never the binding, so it could say public while the file published on the
loopback, or the reverse. A declaration that cannot be wrong is better than one
that has to be kept in step.

The scope is now which binding the file asks for, read from the compose files
as written: ${MYOS_BIND_PUBLIC} is public, a hand-written address is pinned,
and a bare '- 80' or '9000:9000' is unbound, meaning docker opens it on every
address and nobody chose that. --strict fails on those.

Reading the resolved configuration instead would lose the distinction: every
form ends up as a plain address, and an unbound port looks exactly like a
deliberate public one.

What is left to configure per host is the address of each scope, which is the
part that belongs to the host rather than to the stack.
2026-09-05 13:22:04 +02:00
Yann Autissier 90bb97cca8 replace the linux-only firewall patching with a portable exposure model
The catalogue publishes 51 compose files with the bare form 'ports: [80]',
which binds a random host port on 0.0.0.0: every service answers the internet.
ufw-docker existed to take that back afterwards, as root, on linux only,
because docker writes its own firewall rules and ufw never sees those ports.

Publishing where you mean to solves it at the source. Verified against the
daemon: '- 80' gives 0.0.0.0:32768, '127.0.0.1::80' gives 127.0.0.1:32769.
Same on macOS and on linux, no privilege, and visible in docker ps.

A stack binds with ${MYOS_BIND_PRIVATE|PUBLIC|MESH} and declares what it
means with <PREFIX>_SERVICE[_<port>]_EXPOSE. myos expose reads the resolved
compose configuration and reports what would be opened; --strict fails when a
port faces the world without declaring it, which is what an agent runs against
a server it did not set up.
2026-09-05 13:07:13 +02:00
Yann Autissier 60668fc80a implement recreate, reload and status, and stop advertising -H
The usage listed recreate but the dispatcher did not know it: myos recreate
exited 2 saying the command was unknown. status was missing too, and -H was
parsed into a variable nothing ever reads, so it was accepted and ignored. It
now refuses rather than pretending.
2026-09-05 12:15:05 +02:00
Yann Autissier f2bcbc6857 give make a single call for the settings the CLI resolves
myos export prints every setting the requested stacks declare, in one process:
68 variables in half a second, where asking for them one at a time costs a
process each and about ten seconds.

The shim reads that once and includes it, so a .mk target sees
HOST_FABIO_VERSION and the computed fabio tag without make knowing anything
about how they are produced. --make quotes the values so make neither expands
a $ nor starts a comment at a #.
2026-09-05 12:10:32 +02:00
Yann Autissier ca7338bcb6 fix the bugs the rewrite uncovered, in the make engine too
- the wrapper followed one absolute symlink only, and fed the config file to
  env(1) as-is, so a comment or a blank line in /etc/conf.d/myos made every
  command fail with 'env: #comment: No such file or directory'
- verlt called verlte, which does not exist, and used return outside a
  function: it never compared anything
- stat asked for the access time on macOS and the modification time elsewhere,
  so newer/older did not mean the same thing depending on the machine
- setup-docker-group called ansible-user-add-groups, removed with ansible, then
  announced that the user had been added to the docker group. It now runs
  usermod or addgroup, and says so when neither works
- the ssh targets looped over AWS_INSTANCE_IP, which nothing defines since
  make/apps/aws was dropped: they exited 0 having done nothing. They take
  SSH_HOSTS and fail when it is empty
- patsublist left a space before the comma joining two fabio routes
- the JWT macro splits a payload on its commas; documented at the macro, since
  fixing it would change every key it has already produced
2026-09-05 12:03:53 +02:00
Yann Autissier 7163c844c9 note the catalogue conversion in the changelog 2026-09-03 22:33:29 +02:00
Yann Autissier 084a25c627 make the hooks usable by the whole catalogue
- hooks load the _stack files of every directory between the stack path root
  and the stack, outermost first: make included both $(dir)/*.mk and
  $(dir)/*/*.mk, so a stack in a subdirectory saw its parent's settings
- a group may be declared in <group>/<group>.env, where the stack lives
- MYOS_STACK_DIR lets a hook read a file it ships next to itself
- myos_filter no longer confuses a literal * in a make pattern with a wildcard,
  and the list helpers no longer let the shell expand a * into filenames
- MACHINE, SYSTEM, HOST and DOMAINNAME join the framework variables a hook sees
- the make shim gains $(call myos-var,NAME), so a .mk target can read a
  setting that now lives in a hook, and it picks up the .mk of every stack
  directory rather than only the project's
2026-09-03 22:30:01 +02:00
Yann Autissier 6192d73cbe port the make list functions and JWT, and expose the framework variables to hooks
lib/str.sh gains firstword, lastword, or, patsubst, filter, filter_out,
addprefix, addsuffix and jwt: what the catalogue .mk files are written in.
The make JWT macro split on the commas of its payload; this one does not.

lib/context.sh registers COMPOSE_PROJECT_NAME, APP, DOCKER_NETWORK_* and the
rest as lazy defaults, so a converted .mk keeps reading them as it did.

Hooks also load a directory-level _stack.env and _stack.sh, for the .mk files
of the catalogue that hold settings for several stacks at once.
2026-09-03 21:58:22 +02:00
Yann Autissier 3e55cdcd14 chain commands, and let a project refine a catalogue stack
myos build up logs host/fabio runs the three in order and stops at the first
failure, the way make build up logs STACK=host/fabio did. Leading words that
name commands are commands; the first word that is not one starts the stacks.

A stack found in several directories of the stack path is now merged rather
than shadowed, least specific first, so a project drops
stack/postgres/postgres.local.yml next to the catalogue's postgres.yml and
refines it. Settings hooks follow the same order, so a project can redefine a
default the catalogue ships. Neither engine did this before: the project
directory simply hid the catalogue one.

An unknown command now says so and suggests the command to type, instead of
printing the whole usage.
2026-09-03 21:17:17 +02:00
Yann Autissier 55fae625d6 keep the dynamism of make in pure shell
Two mechanisms, matching what the make engine actually did:

Lazy defaults. A stack setting is a function myos_default_<VAR>, called only
when the variable has no value, and called again at every reference. That is
exactly a recursive ?=: an explicit value wins, and the default follows a
DOMAIN that a .env changes later. The prefix is what makes it safe; the first
version used a bare function named after the variable, and the test suite
caught it running /usr/bin/host for a stack group called host.

Templates. myos env-update fills a .env from the .env.dist files, expanding
${VAR} against the current values and running $(command), forward references
included.

Also fixed: the project .env now wins over /etc/conf.d/myos, which is what the
documentation claimed and the code did not.

share/make/shim.mk lets a project keep make as a front end: every myos command
becomes a target that shells out to bin/myos, and the project keeps its own
targets and its stack .mk files. It sits outside make/ because the legacy
engine globs every .mk in there.
2026-09-03 20:46:28 +02:00
Yann Autissier f541ca418b let a stack compute its settings without make
A stack can now ship <name>.env and <name>.sh next to its compose files. The
hook is sourced with the tag helpers available, which is what the computing
.mk files of the catalogue were using make for: 29 of its 44 .mk files only
exist to build variables like the fabio tags.

Converting stack/host/fabio.mk by hand gives byte-identical output for the
route tag, and drops a trailing comma the make version left in the listener
list.

Also: APP_HOST and APP_URI are computed (the tag helpers build on them),
--color controls the escape codes rather than always emitting them, and
make test-portability runs the CLI under busybox ash and dash.
2026-09-03 20:22:52 +02:00
Yann Autissier 234739e531 fix the compose call: it never worked outside --dry-run
Setting IFS to a newline to pass the environment also stopped the command
line from splitting, so '-f a -f b' went out as a single argument and the two
words 'docker compose' were looked up as one program name. Every real command
was broken; only --dry-run, which prints a string, looked right.

The variables are now exported in a subshell instead of going through env(1),
which keeps values with spaces intact and leaves the command line to normal
word splitting.

The tests only checked what --dry-run printed, which is why they missed it.
spec/unit/compose_spec.sh now asserts the argv that is actually executed, for
both the plugin and the standalone binary.
2026-09-03 20:11:37 +02:00
Yann Autissier ac41e6e5f9 add the installer, and document the 2.0 CLI
install.sh handles a system or a user prefix, clones the catalogue on demand
and writes the machine config in the right place for the distribution.
2026-09-03 18:50:49 +02:00
Yann Autissier c08c379565 look for the catalogue beside the installation too
An install under a custom prefix (--prefix ~/.local) could not find the
catalogue it had just cloned into <prefix>/share/myos: only the three system
paths were searched.
2026-09-03 18:49:22 +02:00
Yann Autissier 32e2624245 add exec, run and scale, and the agent skill
The skill documents how to use myos: what to check before touching a host
stack, how a reference resolves, what the three kinds of stack mean, and where
each failure comes from. Every command it shows was run against the fixtures
before being written down, which is how exec, run and scale turned out to be
missing.

AGENTS.md covers the other side: how to change myos without breaking the
deployments that already run it.
2026-09-03 18:46:00 +02:00
Yann Autissier 653a3c9415 harden the config reader against an empty or malformed .env
holcommon's empty .env made the loader evaluate one blank line, which
tripped set -u. Verified afterwards on holcommon's real host stack: the CLI
and the make engine resolve the same five files and render a byte-identical
241-line compose config.
2026-09-03 18:40:31 +02:00
Yann Autissier 35999574bd run the golden suite against the CLI too, and pin every delta
Both engines now go through the same 68 cases. 31 produce byte-identical
output; the rest have a recorded CLI expectation and a reason in DELTAS.md,
the bulk of it being that the CLI does not shell out to a recursive make and
calls compose once per project.

Fixed while comparing: the CLI was missing the COMPOSE_FILE_* defaults, so
overlays such as supabase.labels.yml would not have been loaded at all.
2026-09-03 18:39:23 +02:00
Yann Autissier a346d4f4e1 add the myos CLI: one model for both modes
A stack is a directory of compose files; the current directory is a stack when
it holds one. The same command works for a project, a catalogue stack, a group
and a host singleton, and stacks sharing a project are now a single compose
call.

Two traps of the make engine are closed on the way: an unknown command is an
error instead of a silent success, and a group is only expanded when its name
is lowercase, so an environment variable can no longer be mistaken for one.
That guard spells out its character class because a-z matches uppercase too
under a dictionary collation.
2026-09-03 18:33:47 +02:00
Yann Autissier 990b99f0c0 add the config and compose modules
config: dotenv files are parsed, never sourced, so a value can hold a # or a
$(...) without breaking or executing. The make engine included .env with
make syntax, which neither allowed.
compose: one call per project instead of one per sub-stack.
2026-09-03 18:29:36 +02:00
Yann Autissier e7eab505e9 add stack resolution: search path, references, overlays, groups
Replaces the 500-character stack_path expression of
make/apps/def.docker.mk:159 and its leaked $(stackz) loop variable with
three readable functions, covered by 20 assertions.
2026-09-03 18:27:29 +02:00
Yann Autissier 3437b58078 annotate the cross-file variables for shellcheck 2026-09-03 18:26:14 +02:00
Yann Autissier 5b4db64114 add the pure bash core: core, str, naming and tags
Ported from make/utils.mk, make/def.mk, make/apps/def.docker.mk and
make/apps/def.mk, with the examples that sat as dead comments in
make/apps/def.mk turned into actual assertions.

Two deviations from the make macros are pinned and documented in DELTAS.md;
both only show up on code paths nothing uses.
2026-09-03 18:26:00 +02:00
Yann Autissier 6c97f99f87 run the whole spec dir by default, gate docker-backed specs on a variable
shellspec --tag ~integration filtered out untagged examples too, which
silently emptied the suite.
2026-09-03 18:22:40 +02:00
Yann Autissier 3a64c47260 move the stack catalogue out of the framework
stack/ and docker/ now live in the myos-stacks project (extracted with their
history). What the framework itself needs stays here:
- stack/myos/*.yml            -> share/compose/
- docker/myos/                -> share/docker/myos/
- docker/compose/             -> dropped, docker compose >= 2.24.4 is now required

Residues fixed along the way: DOCKER_IMAGES scanned a hardcoded ./docker,
include.mk filtered a hardcoded stack/*.mk, and docker-image-myos expanded an
undefined MYOS_DOCKER_IMAGES on every up/build.

README and CHANGELOG rewritten: they still documented make host,
host-certbot-* and user-config, all removed when the catalogue was split out.
2026-09-03 18:21:21 +02:00
Yann Autissier 1d44a2ff32 add shellspec golden harness for the legacy make engine
- spec/golden/cases.txt: 69 black-box cases (CLI wrapper mode and make
  include mode) run in a hermetic sandbox with a mocked docker
- spec/golden/expected/*.txt recorded from this engine (baseline)
- make test / test-golden / golden-record / lint dev targets
2026-09-03 14:35:58 +02:00
Yann Autissier e2e34d813e fix stack_path: keep realpath inside each $(or) branch
181a3e9 wrapped both $(or) alternatives in a single $(realpath …), which
turned the argument separator into a literal comma glued to the last path.
host/<svc> stacks were only resolved when the project stack dir happened to
sort before the other STACK_DIR entries.
2026-09-03 14:35:57 +02:00
274 changed files with 5781 additions and 220 deletions
+4
View File
@@ -0,0 +1,4 @@
--require spec_helper
--shell bash
--default-path spec
--pattern "*_spec.sh"
+41
View File
@@ -0,0 +1,41 @@
# myos, for agents working on this repository
myos runs docker compose stacks. `skills/myos/SKILL.md` explains how to *use*
it; this file is about changing it.
## Check your work
```sh
make test # unit + golden, both engines, docker is mocked
make lint # shellcheck
```
Golden tests compare the output of the CLI **and** of the legacy make engine
against recordings made at the tag `legacy-1.0-beta`. If a change moves an
output on purpose, record the CLI expectation in `spec/golden/expected.cli/`
and write down why in `spec/golden/DELTAS.md`. Never edit
`spec/golden/expected/` by hand: it is the behaviour of the old engine.
## Layout
See `skills/myos/references/authoring.md`. In short: `bin/myos` parses and
dispatches, `lib/*.sh` holds one concern each, `lib/cmd/<name>.sh` holds one
command each, `share/compose/` holds the two overlays the framework itself
provides, and the stacks live in another repository, `myos-stacks`.
## Shell constraints
POSIX shell only: this runs on the bash 3.2 of macOS and on Alpine. No `local`,
no arrays, no `[[`. Two traps already paid for:
- `[a-z]` in a `case` pattern also matches uppercase under a `fr_FR` collation;
use `[:lower:]`.
- a function called inside `$( )` cannot return anything through a global.
## Do not
- Reintroduce a `stack/` or `docker/` directory here: they belong to
`myos-stacks`.
- Make an unknown command or an unknown stack succeed silently.
- Change a default project name without an entry in `DELTAS.md` and a note in
the skill: it renames the containers and volumes of every deployment.
+48
View File
@@ -1,5 +1,53 @@
# CHANGELOG # CHANGELOG
## v2.0.0-dev - 2026-09-03
- new bash CLI (`bin/myos`, `lib/`): one model for a project directory, a
catalogue stack, a group and a host singleton
- unknown command or unknown stack now fails, instead of succeeding silently
- default compose project is `<user>-<env>-<app>`; set
`MYOS_PROJECT_FORMAT=user-app-env` on deployments created before this
- `myos ls`, `myos env`, `myos doctor` to inspect an installation
- `install.sh`, and the catalogue is looked up beside the installation
- agent skill in `skills/myos/`, contributor notes in `AGENTS.md`
- stacks carry their settings in `<name>.env` and `<name>.sh` hooks, so the
catalogue no longer needs make to be installed
- lazy defaults (`myos_default_<VAR>` functions) give the recursive `?=` of
make in pure shell: an explicit value wins, and the default is recomputed
at each reference
- `myos env-update` generates a `.env` from the `.env.dist` templates,
expanding `${VAR}` and `$(command)`, including forward references
- the project `.env` now wins over `/etc/conf.d/myos`, as documented;
`MYOS_CONF_PRIORITY=system` restores the previous order
- `share/make/shim.mk`: make as an optional front end over the same shell code
- `myos expose` reports what each stack publishes and to whom, and `--strict`
fails when a port faces the world without saying so. `MYOS_BIND_PUBLIC`,
`_PRIVATE` and `_MESH` let a stack bind its published ports, which replaces
the linux-only ufw-docker patching with something that behaves the same on
macOS and needs no privilege. The scope is read from the compose file rather
than declared beside it, so it cannot disagree with what is published
- `myos cert` derives the certificates a server needs from the route tags its
stacks publish, and asks dehydrated for them: a wildcard where a tag uses one,
a certificate per name otherwise. The `host/dehydrated` stack answers http-01
itself and delegates dns-01 to a provider hook
- commands chain: `myos build up logs host/fabio`, as make targets did
- the stack catalogue no longer needs make at all: its settings are hooks, and
only six stacks keep a .mk, for targets
- a stack found in several directories of the stack path is merged, project
last, so a project refines a catalogue stack instead of replacing it
- `--color always|never|auto`, and no colour when the output is piped
- verified under the /bin/sh of Alpine (busybox) and Debian (dash)
- the make engine still works and is still covered by the golden tests
## v1.1 - 2026-09-03
- move the stack catalogue and the docker build contexts to the myos-stacks project
- keep the framework infra compose files in share/compose and the myos tool image in share/docker
- fix stack_path resolution: host/<svc> stacks were only found when the project
stack directory sorted first
- drop the docker/compose image fallback: docker compose >= 2.24.4 or docker-compose is required
- add a shellspec golden test harness (make test)
## v1.0-beta - 2026-07-29 ## v1.0-beta - 2026-07-29
* split make files in `myos` project and docker files in `stack` project * split make files in `myos` project and docker files in `stack` project
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+34
View File
@@ -1 +1,35 @@
DEV_TARGETS := test test-unit test-golden test-integration test-portability lint golden-record
ifneq ($(filter $(DEV_TARGETS),$(MAKECMDGOALS)),)
SHELLSPEC ?= shellspec
SHELLCHECK ?= shellcheck
.PHONY: $(DEV_TARGETS)
test: ## Run unit + golden tests against both engines
$(SHELLSPEC)
MYOS_ENGINE=cli $(SHELLSPEC) spec/golden
test-unit: ## Run unit tests only
$(SHELLSPEC) spec/unit
test-golden: ## Run golden tests only (MYOS_ENGINE=legacy|cli)
$(SHELLSPEC) spec/golden
test-integration: ## Run tests needing a real docker daemon
MYOS_INTEGRATION=1 $(SHELLSPEC) spec/integration
golden-record: ## Re-record golden expectations from the legacy engine
spec/golden/record.sh $(CASES)
test-portability: ## Run the CLI under the /bin/sh of Alpine and Debian
@for img in alpine:3.20 debian:13-slim; do \
printf '%s: ' "$$img"; \
tar cf - --exclude .git . | docker run -i --rm "$$img" /bin/sh -c \
'mkdir -p /myos && tar xf - -C /myos && cd /tmp && \
export PATH=/myos/spec/support/bin:$$PATH && \
/myos/bin/myos version >/dev/null && \
/myos/bin/myos -C /myos/spec/fixtures/host-project -n up host >/dev/null && \
echo ok'; \
done
lint: ## shellcheck all shell sources
$(SHELLCHECK) -s bash myos spec/golden/record.sh spec/support/run.sh spec/support/bin/* $(wildcard bin/* lib/*.sh lib/cmd/*.sh install.sh)
else
include make/include.mk include make/include.mk
endif
+114 -161
View File
@@ -1,6 +1,12 @@
# myos - Make Your Own Stack # myos - Make Your Own Stack
Make Your Own Stack provides common make targets to build and run docker projects. myos runs docker compose stacks: on a server, in a project directory, for a user.
It is a thin layer over `docker compose` that resolves which compose files to load,
under which project name, with which environment variables.
The framework itself ships no stack. Ready-to-use stacks (consul, fabio, registrator,
postgres, supabase, drone, …) live in a separate catalogue,
[myos-stacks](https://github.com/aya/myos-stacks).
## Disclaimer ## Disclaimer
@@ -8,197 +14,144 @@ This is beta software, use it at your own risks.
## Requirements ## Requirements
You need `docker`, `git` and `make`. `docker` (with the `docker compose` plugin >= 2.24.4, or a `docker-compose` binary),
`git` and `make`.
## Install ## Install
* Include MYOS file `make/include.mk` adding the following lines to your project file `Makefile`. ### As a command
```sh
sudo git clone https://github.com/aya/myos /usr/local/lib/myos
sudo ln -s /usr/local/lib/myos/myos /usr/local/bin/myos
``` ```
MYOS ?= ../myos
MYOS_REPOSITORY ?= $(patsubst %/$(THIS),%/myos,$(THIS_REPOSITORY)) Optionally pin per-machine settings in `/etc/conf.d/myos` (or `/etc/default/myos`),
THIS ?= $(lastword $(subst /, ,$(THIS_REPOSITORY))) one `VAR=value` per line, no comments:
THIS_REPOSITORY ?= $(shell git config --get remote.origin.url 2>/dev/null)
$(MYOS): ```sh
-@git clone $(MYOS_REPOSITORY) $(MYOS) DOMAIN=example.org
ENV=master
```
`myos` runs from the current directory: it passes it as `WORKDIR`, so stacks and
`.env` are looked up there. A `WORKDIR` set in the config file wins over the current
directory, which pins a machine to its deployment directory.
### As a make include
Add to your project `Makefile`:
```make
MYOS ?= /usr/local/lib/myos
-include $(MYOS)/make/include.mk -include $(MYOS)/make/include.mk
``` ```
* Call the `make help` command to show available targets. Then `make help` lists the available targets.
``` ### Stack catalogue
$ make help
Usage:
make [target]
Targets: myos looks for stacks in `./stack`, `../stack`, `~/.local/share/myos/stack`,
help This help `/usr/local/share/myos/stack` and `/usr/share/myos/stack`:
[...]
```sh
sudo git clone https://github.com/aya/myos-stacks /usr/local/share/myos
``` ```
## Usage ## Usage
### Examples ```sh
myos doctor # check the installation first
* Configure myos for domain `domain.tld` and stack `default` myos ls # what stacks are reachable
myos -n up host # print what it would run
```shell myos up # the stack of the current directory
$ make bootstrap DOMAIN=domain.tld STACK=default myos up STACK=host # a group of stacks, see stack/host/host.mk
myos up STACK=host/fabio # a single stack
myos up STACK=postgres:9.6 # a versioned stack
myos ps
myos logs
myos config # rendered compose file
myos down
myos shutdown # every stack: app, host and user
``` ```
* Start myos stack `host` ### How a stack is resolved
```shell A stack is a directory of compose files. For `STACK=<name>` in environment `ENV`,
$ make host myos loads, in order, whichever of these exist:
```
`make host` starts the stack `host` with docker host services :
- consul (service discovery) on host port 8500
- fabio (load balancer) on host ports 80 and 443
- registrator (docker/consul bridge)
* Stop myos
```shell
$ make shutdown
```
### Variables
* DEBUG
Show executed commands.
```shell
$ make up DEBUG=true
```
* DRYRUN
Do nothing, show commands instead of executing it.
```shell
$ make up DRYRUN=true
```
* VERBOSE
Show called functions.
```shell
$ make up VERBOSE=true
```
* Show variable USER
```shell
$ make print-USER
```
#### Setup
* SETUP_LETSENCRYPT
Generate ${DOMAIN} certificate files with letsencrypt.
By default, myos generates invalid ${DOMAIN} certificate files with openssl.
You can use letsencrypt instead, to generate valid wildcard certificate files.
To achieve this, you must add following DNS entries to domain ${DOMAIN} to prove you own it:
``` ```
_acme-challenge.${DOMAIN} IN CNAME ${DOMAIN}.acme.${DOMAIN}. <name>.yml <name>.<ENV>.yml <ENV>/<name>.yml
acme.${DOMAIN}. IN NS certbot.${DOMAIN}. <name>.<suffix>.yml <name>.<suffix>.<ENV>.yml <name>.<version>.yml
certbot.${DOMAIN}. IN A ${DOCKER_HOST_INET4}
``` ```
In this config, DOCKER_HOST_INET4 should be the external IP address of the server running certbot. `<suffix>` comes from the `COMPOSE_FILE_*` variables that are not `false`
Port 53 of this IP address must be reachable from internet and point to this server. (`app`, `labels`, `networks`, `ssh`, `volumes` by default; add e.g.
`COMPOSE_FILE_WWW=true` to also load `<name>.www.yml`). The framework always
appends its own `share/compose/networks.yml`.
If you want a simple DNS configuration to host all your services on the same server, you can setup following DNS config: ### Project names and networks
``` | stack | compose project | meaning |
@ IN A ${DOCKER_HOST_INET4} |---|---|---|
*.${DOMAIN}. IN CNAME ${DOMAIN}. | `host/*` | `$(HOSTNAME)` | one instance per machine: ports 80/443, consul, certbot |
_acme-challenge.${DOMAIN} IN CNAME ${DOMAIN}.acme.${DOMAIN}. | `User/*` | user identity | one instance per user |
acme.${DOMAIN}. IN NS ${DOMAIN}. | anything else | `<user>-<app>-<env>` | many instances per machine |
Networks: `default` = `_<project>` (private to the project), `private` =
`<user>-<env>` and `public` = `<hostname>`, both external and created on demand.
## Variables
| variable | effect |
|---|---|
| `DEBUG=true` | show executed commands |
| `DRYRUN=true` | print commands instead of running them |
| `VERBOSE=true` | show called functions |
| `ENV=<env>` | environment: selects `.env.<env>` and the `<name>.<env>.yml` overlays |
| `STACK=<refs>` | stacks to act on |
| `SERVICE=<name>` | target one compose service (`exec`, `run`, `logs`, `scale`) |
| `MYOS_PROJECT_FORMAT` | `user-env-app` (default) or `user-app-env` for deployments made before myos 2.0 |
```sh
myos env COMPOSE_FILE # show a variable
myos env COMPOSE_PROJECT_NAME
myos config <stack> # the rendered compose file
``` ```
This will point domain ${DOMAIN} to the IP address ${DOCKER_HOST_INET4} of this server, and point all subdomains *.{DOMAIN} to the ip address pointed by ${DOMAIN}. The make targets keep working: `print-VAR`, `stack-<stack>-<command>`,
`<command>@<env>`, and a project `Makefile` that includes `make/include.mk`.
At this point, you should be able to generate a valid certificate for *.${DOMAIN} using certbot [dns standalone](https://github.com/siilike/certbot-dns-standalone) plugin. `SETUP_UFW=true` enables the ufw/ufw-docker integration (`myos setup-ufw`).
This task is done automatically when creating the host stack if SETUP_LETSENCRYPT variable is not empty.
If you already launched myos host stack before, the ${DOMAIN} certificates has been automatically generated by openssl and you should remove them before trying to generate them with letsencrypt. ## With make
``` A project that would rather drive make can include the shim, which turns every
$ make host-down myos command into a make target while leaving its own targets alone:
$ docker volume rm $(hostname)
```make
MYOS ?= /usr/local/lib/myos
include $(MYOS)/share/make/shim.mk
``` ```
You can then test the letsencrypt certificate generation using DEBUG mode that force to use the letsencrypt staging server. `make up STACK=host` then runs exactly what `myos up host` runs: the shim only
forwards. make is not needed otherwise, and the CLI never calls it.
``` ## For agents
$ make host SETUP_LETSENCRYPT=true DEBUG=true
`skills/myos/SKILL.md` is a skill describing how to drive myos, with
references on the conventions, the commands and the failure modes.
`AGENTS.md` covers changing myos itself.
## Tests
```sh
make test # shellspec: unit + golden (a mocked docker, no daemon needed)
make test-golden # golden only
make golden-record # re-record the golden expectations
make lint # shellcheck
``` ```
If letsencrypt certificate generation fails, you can retry the generation of a staging certificate. ## License
```
$ make host-certbot-staging
```
Once the certificate generation is working, you can ask for a valid certificate.
```
$ make host-down
$ docker volume rm $(hostname)
$ make host SETUP_LETSENCRYPT=true
```
* SETUP_UFW
Control linux firewall rules with ufw.
```
$ echo SETUP_UFW=true >> .env
$ make setup-ufw
```
### Debug
* Show docker compose yaml config
```shell
$ make config
```
`make config` show docker compose yaml config for stack `STACK`
`make host-config` show docker compose yaml config for stack `host`
`make user-config` show docker compose yaml config for stack `User`
`make stack-elastic-config` show docker compose yaml config for stack `elastic`
* Show debug variables
```shell
$ make debug
```
* Generate self documentation
```shell
$ make doc
```
* Show env args
```shell
$ make print-env_args
```
* Show user mail
```shell
$ make print-MAIL
```
GPL-3.0, see LICENSE.
+40
View File
@@ -0,0 +1,40 @@
# Restart protocol (2026-09-05)
The rewrite starts over from the make engine, not from `lib/`.
1. **Reference = `make/*` as it is**, bugs fixed one by one, each with a test
that goes red then green. Known bugs and their locations are listed in the
planning notes and in `git log --grep=fix` on this branch.
2. **Tests first, against make**: every target in scope gets a functional case
in `spec/golden/cases.txt`, recorded from the make engine
(`spec/golden/record.sh`, `spec/support/run.sh` engine `legacy`). That
recording is the historical behaviour, defects included.
3. **Rewrite target by target in red/green**: the new implementation (`just`
as the interface with line recipes, POSIX sh as the logic) must turn each
case green under `MYOS_ENGINE=just`. A deliberate departure from the
historical behaviour is written down in `spec/golden/DELTAS.md`.
4. The first attempt (`bin/myos`, `lib/`, `lib/cmd/`, `share/make/shim.mk`,
`spec/golden/expected.cli/`, `spec/unit/`) is tagged `attempt-1-lib` and is
not a base for the rewrite. What is worth keeping from it is ideas: port
exposure by bind address (`expose --strict`), certificates derived from the
route tags (dehydrated), `env-update` with forward references, stack
directories merged along the search path, lazy defaults, command chaining,
typed exit codes, the agent skill, the installer.
5. The catalogue readable by make is `myos-stacks@7289b83` (or `github/develop`
here): the later hooks (`_stack.sh`) are not read by make.
6. Scope from real fleet usage (~15 targets): up down build config logs ps
restart status, the `host` group, print-VAR, docker-build-<image>,
setup-ufw, install bootstrap clean, apps-install. Never used: release,
subrepo, git-*, deploy, ssh-*.
7. Keep and reuse: `spec/support/run.sh`, the docker mocks, the fixtures, the
golden cases, `spec/bench/` (make 312 ms fixed + ~700 ms per stack; just
line recipe 19 ms, shebang recipe 160 ms; shell hooks with command
substitutions ~40 ms per computed setting on any engine).
Traps already paid for in POSIX sh, do not pay them again: `IFS=$'\n'` stops
argument splitting; `for w in $list` globs a `*` (use `set -f`); `[a-z]`
matches uppercase under fr_FR (use `[:lower:]`); a function called inside
`$( )` cannot return through a global; an environment variable must never be
taken for a stack group (lowercase names only); an unprefixed lazy default
named `host` runs `/usr/bin/host`; zsh does not split unquoted variables, so
test scripts run under `sh`.
Executable
+314
View File
@@ -0,0 +1,314 @@
#!/bin/sh
# myos - Make Your Own Stack
#
# shellcheck disable=SC2034 # most globals here are read by lib/ and lib/cmd/
# shellcheck disable=SC1091 # lib files are sourced by path at runtime
#
# Runs docker compose stacks: on a host, in a project directory, for a user.
# See README.md, or `myos help`.
set -u
MYOS_VERSION=2.0.0-dev
# --- locate the installation ---------------------------------------------
_self=$0
while [ -L "$_self" ]; do
_link=$(readlink "$_self")
case $_link in /*) _self=$_link ;; *) _self=$(dirname "$_self")/$_link ;; esac
done
MYOS_ROOT=$(cd "$(dirname "$_self")/.." && pwd -P)
export MYOS_ROOT
for _m in core str var tags naming stack config compose hooks context expose cert; do
# shellcheck source=/dev/null
. "$MYOS_ROOT/lib/$_m.sh"
done
# myos_is_command WORD true when WORD names a command rather than a stack
myos_is_command() {
case $1 in
up|down|start|stop|restart|ps|logs|config|build|pull|create|kill|top|images) return 0 ;;
version|help|export) return 0 ;;
print-*|stack-*-*) return 0 ;;
*@*) myos_is_command "${1%@*}"; return $? ;;
esac
[ -f "$MYOS_ROOT/lib/cmd/$1.sh" ]
}
# --- command line ---------------------------------------------------------
# These are read by the lib/cmd/* files sourced further down.
# shellcheck disable=SC2034
{
MYOS_CMDS=
MYOS_REFS=
MYOS_REFS_RAW=
MYOS_VARS=
MYOS_ARGS=
MYOS_COLOR=${MYOS_COLOR:-auto}
VERBOSE=${VERBOSE:-}
DEBUG=${DEBUG:-}
}
usage() {
cat <<'USAGE'
Usage: myos [options] <command> [stack...] [VAR=value...] [-- args...]
Options:
-C DIR work in DIR instead of the current directory
-e ENV environment (default: local, or ENV from the config)
-n, --dry-run print the commands instead of running them
--color WHEN always, never or auto (default: colour when on a terminal)
-v, --verbose show what myos does
-d, --debug show every command
-h, --help this help
Commands:
up down start stop restart recreate manage the containers of a stack
ps status logs config exec run inspect and enter them
ls [--groups] list the stacks myos can see
env [VAR...] show resolved variables
export every setting of the stacks, as KEY=value
env-update fill .env from the .env.dist templates
expose [--strict] what the stacks publish, and to whom
cert list|issue|renew|show certificates, derived from the routes
doctor check the installation
version print the myos version
Stacks:
myos up the stack of the current directory
myos up host a group, expanded from host=... in a .env or .mk
myos up host/fabio a single stack
myos up postgres:9.6 a versioned stack
USAGE
}
while [ $# -gt 0 ]; do
case $1 in
-C) WORKDIR=$2; shift 2 ;;
-e) ENV=$2; shift 2 ;;
# running on remote hosts is not implemented yet; the flag is refused
# rather than silently ignored
-H) myos_die "$MYOS_E_USAGE" "-H is not implemented yet: run myos on the host itself" ;;
-n|--dry-run) DRYRUN=true; shift ;;
--color) MYOS_COLOR=$2; shift 2 ;;
--color=*) MYOS_COLOR=${1#--color=}; shift ;;
-v|--verbose) VERBOSE=true; shift ;;
-d|--debug) DEBUG=true; shift ;;
-h|--help) usage; exit 0 ;;
--) shift; MYOS_ARGS="$*"; break ;;
-*)
if [ -n "$MYOS_CMDS" ]; then
MYOS_ARGS="${MYOS_ARGS:+$MYOS_ARGS }$1"; shift
else
myos_error "unknown option: $1"; usage >&2; exit "$MYOS_E_USAGE"
fi ;;
*=*) eval "${1%%=*}=\${1#*=}"; export "${1%%=*}"; shift ;;
*)
# Leading words that name commands are commands, the rest are stacks:
# `myos build up logs host/fabio` runs three commands on one stack, the
# way `make build up logs STACK=host/fabio` did.
if [ -z "$MYOS_REFS" ] && myos_is_command "$1"; then
MYOS_CMDS="${MYOS_CMDS:+$MYOS_CMDS }$1"
else
MYOS_REFS="${MYOS_REFS:+$MYOS_REFS }$1"
fi
shift ;;
esac
done
if [ -z "$MYOS_CMDS" ]; then
if [ -n "$MYOS_REFS" ]; then
# the first word was meant as a command; name it rather than dump the usage
myos_error "unknown command: ${MYOS_REFS%% *}"
myos_error "to act on a stack of that name, say what to do: myos up ${MYOS_REFS%% *}"
exit "$MYOS_E_USAGE"
fi
usage
exit "$MYOS_E_USAGE"
fi
MYOS_REFS_RAW=$MYOS_REFS
# Targets of the make engine keep working: print-VAR, stack-<stack>-<command>
# and <command>@<env> each translate to a command of the CLI.
_cmds=
for _c in $MYOS_CMDS; do
case $_c in
*@*) ENV=${_c#*@}; _c=${_c%@*} ;;
esac
case $_c in
print-*)
MYOS_VARS="${MYOS_VARS:+$MYOS_VARS }${_c#print-}"
# shellcheck disable=SC2209 # the literal string "env", not the command
_c=env ;;
stack-*-*)
_rest=${_c#stack-}
_c=${_rest##*-}
MYOS_REFS="${_rest%-*}${MYOS_REFS:+ $MYOS_REFS}" ;;
esac
_cmds="${_cmds:+$_cmds }$_c"
done
MYOS_CMDS=$_cmds
# cert takes a subcommand where the others take only stacks
case $MYOS_CMDS in
*cert*)
case ${MYOS_REFS%% *} in
list|domains|issue|renew|show)
MYOS_VARS=${MYOS_REFS%% *}
case $MYOS_REFS in *' '*) MYOS_REFS=${MYOS_REFS#* } ;; *) MYOS_REFS= ;; esac ;;
esac ;;
esac
# env, ls and doctor take variable names where the others take stacks
case $MYOS_CMDS in
env|ls|doctor)
[ -n "$MYOS_VARS" ] || MYOS_VARS=$MYOS_REFS
MYOS_REFS= ;;
esac
# --- configuration --------------------------------------------------------
WORKDIR=${WORKDIR:-$PWD}
WORKDIR=$(cd "$WORKDIR" 2>/dev/null && pwd -P) || myos_die "$MYOS_E_USAGE" "no such directory: $WORKDIR"
# ENV decides which .env.<env> to read, so it is resolved first, from the most
# specific source that names it.
if [ -z "${ENV:-}" ]; then
for _f in "$WORKDIR/.env" "${HOME:-}/.config/myos/config" $(myos_conf_files); do
ENV=$(myos_dotenv_parse "$_f" | sed -n 's/^ENV=//p' | tail -1)
[ -n "$ENV" ] && break
done
fi
ENV=${ENV:-local}
# The layers, most specific first: the loader never overwrites a value, so the
# order below is the order of precedence. The environment and the VAR=value
# arguments are already set, and therefore win over every file.
# MYOS_CONF_PRIORITY=system puts the machine files first, as the make engine did.
myos_config_layers() {
if [ "${MYOS_CONF_PRIORITY:-}" = system ]; then
myos_conf_files
printf '%s\n' "${HOME:-}/.config/myos/config" "$WORKDIR/.env.$ENV" "$WORKDIR/.env"
else
printf '%s\n' "$WORKDIR/.env.$ENV" "$WORKDIR/.env" "${HOME:-}/.config/myos/config"
myos_conf_files
fi
}
for _f in $(myos_config_layers); do myos_dotenv_load "$_f"; done
# Overlay switches. Their names drive which <stack>.<suffix>.yml files load,
# so these defaults decide that e.g. supabase.labels.yml is picked up.
COMPOSE_FILE_APP=${COMPOSE_FILE_APP:-true}
COMPOSE_FILE_LABELS=${COMPOSE_FILE_LABELS:-true}
COMPOSE_FILE_NETWORKS=${COMPOSE_FILE_NETWORKS:-true}
COMPOSE_FILE_SSH=${COMPOSE_FILE_SSH:-true}
COMPOSE_FILE_VOLUMES=${COMPOSE_FILE_VOLUMES:-true}
COMPOSE_FILE_DEBUG=${COMPOSE_FILE_DEBUG:-${DEBUG:+true}}
USER=${USER:-$(id -nu 2>/dev/null)}
HOSTNAME=${HOSTNAME:-$(hostname 2>/dev/null | sed 's/\..*//')}
HOSTNAME=$(myos_lower "$HOSTNAME")
DOMAIN=${DOMAIN:-localhost}
DOMAINNAME=${DOMAINNAME:-${DOMAIN%% *}}
MAIL=${MAIL:-$(git config user.email 2>/dev/null || printf '%s@%s' "$USER" "$DOMAINNAME")}
DRYRUN=${DRYRUN:-false}
myos_colors
# --- stack references -----------------------------------------------------
# No reference given: the configured STACK, else the current directory when it
# holds a compose file. An explicit STACK always wins, so a project that pins
# its stacks in .env keeps working from inside its own directory.
if [ -z "$MYOS_REFS" ]; then
if [ -n "${STACK:-}" ]; then
MYOS_REFS=$STACK
elif [ -f "$WORKDIR/docker-compose.yml" ] || [ -f "$WORKDIR/compose.yml" ] ||
[ -f "$WORKDIR/docker/docker-compose.yml" ]; then
MYOS_REFS=./
fi
fi
if [ -z "$MYOS_REFS" ]; then
# these commands describe the installation rather than act on a stack
for _c in $MYOS_CMDS; do
case $_c in
env|env-update|export|expose|ls|doctor|version|help) ;;
*) myos_die "$MYOS_E_USAGE" "no stack given, and no compose file in $WORKDIR" ;;
esac
done
fi
# shellcheck disable=SC2086 # a list of references
MYOS_STACKS=$(myos_group_expand $MYOS_REFS)
myos_context_defaults
# The uri a stack is served on: the tag helpers build on it, so it has to be
# known before the hooks run.
MYOS_SCOPE_FIRST=$(for _r in $MYOS_STACKS; do myos_scope "$_r"; break; done)
MYOS_APP_FIRST=$(for _r in $MYOS_STACKS; do myos_stack_name "$_r"; break; done)
APP_HOST=${APP_HOST:-$(myos_app_host "$MYOS_SCOPE_FIRST" "$USER" "$ENV" "$MYOS_APP_FIRST" "$DOMAINNAME" "$HOSTNAME")}
APP_URI=${APP_URI:-$(myos_app_uri "$APP_HOST" "${APP_PATH:-}")}
APP_SCHEME=${APP_SCHEME:-http}
# Per-stack hooks must run in this shell: everything downstream reads the
# variables they set, and a command substitution would throw them away.
for _ref in $MYOS_STACKS; do
for _hdir in $(myos_stack_dirs "$_ref"); do
myos_stack_hooks "$_hdir" "$(myos_stack_name "$_ref")"
done
done
# The two functions below are called from the lib/cmd/* files sourced later,
# which shellcheck cannot see.
# shellcheck disable=SC2329
# myos_framework_compose_files the networks and volumes overlays myos itself
# provides; they always come last so a stack can rely on them being there.
myos_framework_compose_files() {
myos_compose_files "$MYOS_ROOT/share/compose" "networks volumes" "$(myos_compose_suffixes)" "$ENV"
}
# shellcheck disable=SC2329
# myos_stack_compose_files REF the ordered compose files of one reference
myos_stack_compose_files() {
_dirs=$(myos_stack_dirs "$1")
[ -n "$_dirs" ] || { myos_stack_resolve "$1" >/dev/null; return $?; }
_name=$(myos_stack_name "$1")
_suffixes="$(myos_compose_suffixes) $(myos_stack_version "$1")"
for _dir in $_dirs; do
case $1 in
.|./*|/*|../*)
myos_compose_files "$_dir" "docker-compose compose" "$_suffixes" "$ENV"
myos_compose_files "$_dir/docker" "docker-compose compose" "$_suffixes" "$ENV" ;;
*)
myos_compose_files "$_dir" "docker-compose $_name" "$_suffixes" "$ENV" ;;
esac
done
}
# --- dispatch -------------------------------------------------------------
# myos_dispatch COMMAND run one command
myos_dispatch() {
case $1 in
version) printf 'myos %s\n' "$MYOS_VERSION"; return 0 ;;
help) usage; return 0 ;;
esac
if [ -f "$MYOS_ROOT/lib/cmd/$1.sh" ]; then
# shellcheck source=/dev/null
. "$MYOS_ROOT/lib/cmd/$1.sh"
"myos_cmd_$(printf '%s' "$1" | tr '-' '_')"
return $?
fi
# commands that map straight onto docker compose
case $1 in
up|down|start|stop|restart|ps|logs|config|build|pull|create|kill|top|images) ;;
*) myos_error "unknown command: $1"; usage >&2; return "$MYOS_E_USAGE" ;;
esac
# shellcheck source=/dev/null
. "$MYOS_ROOT/lib/cmd/_compose.sh"
myos_cmd_compose "$1"
}
# Several commands run in order and stop at the first failure, as make did.
_rc=0
for MYOS_CMD in $MYOS_CMDS; do
myos_dispatch "$MYOS_CMD" || { _rc=$?; break; }
done
exit "$_rc"
-20
View File
@@ -1,20 +0,0 @@
FROM alpine:latest as dist
LABEL maintainer aynic.os <support+docker@asycn.io>
ARG DOCKER_BUILD_DIR
ARG COMPOSE_REMOTE=https://github.com/docker/compose
ARG COMPOSE_VERSION=2.5.0
ARG DOCKER_MACHINE=x86_64
ARG DOCKER_SYSTEM=Linux
RUN apk update \
&& apk add --no-cache ca-certificates \
&& OS="$(echo ${DOCKER_SYSTEM} |awk '{print tolower($0)}')" \
&& ARCH="$(echo ${DOCKER_MACHINE} |awk '{print /armv7l/ ? "armv7" : $0}')" \
&& wget -qO /usr/bin/docker-compose ${COMPOSE_REMOTE}/releases/download/v${COMPOSE_VERSION}/docker-compose-${OS}-${ARCH} \
&& chmod +x /usr/bin/docker-compose
ENTRYPOINT ["/usr/bin/docker-compose"]
FROM dist as master
ARG DOCKER_BUILD_DIR
Executable
+106
View File
@@ -0,0 +1,106 @@
#!/bin/sh
# myos installer.
#
# curl -fsSL https://raw.githubusercontent.com/aya/myos/lightning/install.sh | sh
# ... | sh -s -- --prefix ~/.local --with-stacks
#
# Installs the framework into <prefix>/lib/myos, links <prefix>/bin/myos, and
# optionally clones the stack catalogue into <prefix>/share/myos.
set -eu
MYOS_REPOSITORY=${MYOS_REPOSITORY:-https://github.com/aya/myos}
STACKS_REPOSITORY=${STACKS_REPOSITORY:-https://github.com/aya/myos-stacks}
REF=${MYOS_REF:-lightning}
PREFIX=
WITH_STACKS=false
WRITE_CONF=false
say() { printf '%s\n' "$*"; }
warn() { printf 'warning: %s\n' "$*" >&2; }
die() { printf 'error: %s\n' "$*" >&2; exit 1; }
have() { command -v "$1" >/dev/null 2>&1; }
while [ $# -gt 0 ]; do
case $1 in
--prefix) PREFIX=$2; shift 2 ;;
--ref) REF=$2; shift 2 ;;
--repository) MYOS_REPOSITORY=$2; shift 2 ;;
--with-stacks) WITH_STACKS=true; shift ;;
--conf) WRITE_CONF=true; shift ;;
-h|--help)
sed -n '2,9p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) die "unknown option: $1" ;;
esac
done
# Default prefix: system wide when we can write there, user local otherwise.
if [ -z "$PREFIX" ]; then
if [ "$(id -u)" = 0 ] || [ -w /usr/local/lib ]; then PREFIX=/usr/local; else PREFIX=$HOME/.local; fi
fi
# --- requirements ---------------------------------------------------------
have git || die "git is required"
have docker || warn "docker not found: myos will not be able to run anything"
if have docker && docker compose version >/dev/null 2>&1; then :
elif have docker-compose; then :
else warn "no docker compose found: install the compose plugin, or docker-compose >= 2.24.4"
fi
# --- install --------------------------------------------------------------
LIB=$PREFIX/lib/myos
BIN=$PREFIX/bin
if [ -d "$LIB/.git" ]; then
say "updating $LIB"
git -C "$LIB" fetch --quiet origin "$REF"
git -C "$LIB" checkout --quiet FETCH_HEAD
else
say "installing myos into $LIB"
mkdir -p "$(dirname "$LIB")"
git clone --quiet --branch "$REF" "$MYOS_REPOSITORY" "$LIB"
fi
mkdir -p "$BIN"
ln -sf "$LIB/bin/myos" "$BIN/myos"
say "linked $BIN/myos"
if [ "$WITH_STACKS" = true ]; then
SHARE=$PREFIX/share/myos
if [ -d "$SHARE/.git" ]; then
say "updating the stack catalogue in $SHARE"
git -C "$SHARE" pull --quiet --ff-only
else
say "installing the stack catalogue into $SHARE"
mkdir -p "$(dirname "$SHARE")"
git clone --quiet "$STACKS_REPOSITORY" "$SHARE"
fi
fi
# --- machine configuration ------------------------------------------------
# /etc/conf.d on Alpine and other OpenRC systems, /etc/default elsewhere.
if [ "$WRITE_CONF" = true ]; then
if [ -d /etc/conf.d ]; then CONF=/etc/conf.d/myos; else CONF=/etc/default/myos; fi
if [ "$(id -u)" != 0 ]; then CONF=$HOME/.config/myos/config; mkdir -p "$(dirname "$CONF")"; fi
if [ -f "$CONF" ]; then
say "keeping the existing $CONF"
else
cat > "$CONF" <<CONFEOF
# myos machine settings, one KEY=value per line.
# ENV=master
# DOMAIN=$(hostname -d 2>/dev/null || echo example.org)
# WORKDIR=/srv/myos
# a deployment created before myos 2.0 must keep the old project names:
# MYOS_PROJECT_FORMAT=user-app-env
CONFEOF
say "wrote $CONF"
fi
fi
case :$PATH: in
*:$BIN:*) ;;
*) warn "$BIN is not on your PATH" ;;
esac
say ""
say "myos $("$BIN/myos" version 2>/dev/null | awk '{print $2}') installed"
say "next: myos doctor"
+98
View File
@@ -0,0 +1,98 @@
#shellcheck shell=sh
# cert: which certificates a server needs, derived from what its stacks route.
#
# The hostnames are already declared, once, in the fabio route tags a stack
# publishes: urlprefix-<host>/<path>. Asking for them a second time in a
# domains.txt would be a second source of truth, free to disagree with what is
# actually served. They are read from the resolved compose configuration
# instead.
#
# A name written *.example.org needs a wildcard, which ACME only issues over
# dns-01; a concrete name can be had over http-01. That is the whole of
# "per-site or wildcard according to need": the tags say which.
# myos_cert_names the hostnames the requested stacks route, one per line
myos_cert_names() {
for _ref in $MYOS_STACKS; do
_files=$(myos_stack_compose_files "$_ref" 2>/dev/null) || continue
[ -n "$_files" ] || continue
_fw=$(myos_framework_compose_files)
[ -n "$_fw" ] && _files="$_files
$_fw"
_app=$(myos_stack_name "$_ref")
_project=$(myos_project_name "$(myos_scope "$_ref")" "$USER" "$ENV" "$_app")
DRYRUN=false myos_compose "$_project" "$_files" -- config 2>/dev/null
done | myos_cert_parse
}
# myos_cert_parse (compose config on stdin) -> hostnames
# A tag is urlprefix-<host>[:<port>]/<path> with options after a space; the
# bare "*" is fabio's catch-all and names nothing.
myos_cert_parse() {
grep -oE 'urlprefix-[^",[:space:]]*' 2>/dev/null |
sed -e 's/^urlprefix-//' -e 's|/.*||' -e 's/:[0-9]*$//' |
grep -vE '^\*?$' |
sort -u
}
# myos_cert_covers WILDCARD_PARENT NAME does *.parent cover this name?
# A wildcard matches one label, so *.example.org covers a.example.org but
# neither example.org nor a.b.example.org.
myos_cert_covers() {
case $2 in
*".$1")
_head=${2%".$1"}
case $_head in *.*|'') return 1 ;; *) return 0 ;; esac ;;
*) return 1 ;;
esac
}
# myos_cert_groups the certificates to ask for, one per line, in the shape
# dehydrated reads: the common name first, then its subject alternative names.
#
# MYOS_CERT_MODE:
# auto a wildcard where the tags use one, a certificate per name otherwise
# wildcard one wildcard per domain, whether or not a tag asked for it
# per-site never a wildcard: one certificate per name, dns-01 not required
myos_cert_groups() {
_names=$(myos_cert_names)
[ -n "$_names" ] || return 0
_mode=${MYOS_CERT_MODE:-auto}
# the parents a wildcard is wanted for
_wild=
for _n in $_names; do
case $_n in
\*.*) [ "$_mode" = per-site ] || _wild="$_wild ${_n#\*.}" ;;
esac
done
if [ "$_mode" = wildcard ]; then
for _n in $_names; do
case $_n in
\*.*) ;;
*.*.*) _wild="$_wild ${_n#*.}" ;;
esac
done
fi
_wild=$(printf '%s' "$_wild" | tr ' ' '\n' | sed '/^$/d' | sort -u)
# one line per wildcard, the parent first so it is the common name
for _p in $_wild; do
printf '%s *.%s\n' "$_p" "$_p"
done
# the concrete names a wildcard does not already cover
for _n in $_names; do
case $_n in \*.*) continue ;; esac
_covered=no
for _p in $_wild; do
[ "$_n" = "$_p" ] && { _covered=yes; break; }
myos_cert_covers "$_p" "$_n" && { _covered=yes; break; }
done
[ "$_covered" = no ] && printf '%s\n' "$_n"
done
return 0
}
# myos_cert_needs_dns true when any certificate asked for is a wildcard
myos_cert_needs_dns() { myos_cert_groups | grep -q '\*\.'; }
+75
View File
@@ -0,0 +1,75 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# The commands that map straight onto docker compose.
#
# Stacks are grouped by compose project: every host stack shares the project of
# the machine, so `myos up host` is a single compose call with every file, the
# way the stack was meant to be described.
# myos_cmd_compose COMMAND
myos_cmd_compose() {
_cmd=$1
_projects=
_rc=0
for _ref in $MYOS_STACKS; do
_files=$(myos_stack_compose_files "$_ref") || { _rc=$MYOS_E_NOSTACK; continue; }
[ -n "$_files" ] || { myos_warning "no compose file for stack $_ref"; continue; }
_scope=$(myos_scope "$_ref")
_app=$(myos_stack_name "$_ref")
case $_ref in .|./*|/*|../*) _app=$(basename "$(myos_stack_resolve "$_ref")") ;; esac
_project=$(myos_project_name "$_scope" "$USER" "$ENV" "$_app")
# accumulate the files of every stack sharing a project, keeping the order
_projects=$(printf '%s\n%s\t%s' "$_projects" "$_project" "$(printf '%s' "$_files" | tr '\n' ' ')")
done
[ "$_rc" = 0 ] || return "$_rc"
for _project in $(printf '%s' "$_projects" | sed '/^$/d' | cut -f1 | awk '!seen[$0]++'); do
_files=$(printf '%s' "$_projects" | sed '/^$/d' | awk -F'\t' -v p="$_project" '$1==p {print $2}' | tr ' ' '\n' | sed '/^$/d' | awk '!seen[$0]++')
# the framework overlays always come last, as the make engine did
_fw=$(myos_framework_compose_files)
[ -n "$_fw" ] && _files="$_files
$_fw"
COMPOSE_PROJECT_NAME=$_project
COMPOSE_SERVICE_NAME=$(myos_service_name "$_project")
DOCKER_NETWORK_DEFAULT=${DOCKER_NETWORK_DEFAULT:-$(myos_network_default "$_project")}
DOCKER_NETWORK_PRIVATE=$(myos_network_private "$USER" "$ENV")
# shellcheck disable=SC3028 # HOSTNAME is set by bin/myos, not by the shell
DOCKER_NETWORK_PUBLIC=$(myos_network_public "$HOSTNAME")
export COMPOSE_PROJECT_NAME COMPOSE_SERVICE_NAME
export DOCKER_NETWORK_DEFAULT DOCKER_NETWORK_PRIVATE DOCKER_NETWORK_PUBLIC
case $_cmd in
up) myos_network_ensure "$DOCKER_NETWORK_PRIVATE" "$DOCKER_NETWORK_PUBLIC" ;;
esac
# shellcheck disable=SC2086,SC2046 # options and MYOS_ARGS are word lists
myos_compose "$_project" "$_files" -- "$_cmd" $(myos_compose_options "$_cmd") ${MYOS_ARGS:-} || _rc=$?
done
return "$_rc"
}
# myos_compose_options COMMAND the default options of each compose command
myos_compose_options() {
case $1 in
up) printf '%s' "${DOCKER_COMPOSE_UP_OPTIONS:--d}" ;;
logs) printf '%s' "${DOCKER_COMPOSE_LOGS_OPTIONS:---follow --tail=100}" ;;
down) printf '%s' "${DOCKER_COMPOSE_DOWN_OPTIONS:-}" ;;
*) printf '' ;;
esac
}
# myos_network_ensure NAME... create the external networks if they are missing
myos_network_ensure() {
for _n in "$@"; do
[ -n "$_n" ] || continue
if [ "${DRYRUN:-false}" = true ]; then
printf 'docker network create %s\n' "$_n"
else
docker network inspect "$_n" >/dev/null 2>&1 || myos_run docker network create "$_n" >/dev/null
fi
done
return 0
}
+91
View File
@@ -0,0 +1,91 @@
#shellcheck shell=sh
# shellcheck disable=SC1091 # lib/cmd files are sourced by path at run time
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# myos cert <list|domains|issue|renew|show> the certificates a server needs
#
# The hostnames come from the route tags of the stacks, so a site gets a
# certificate by being routed, not by being written down a second time.
myos_cmd_cert() {
_sub=$(myos_firstword "${MYOS_VARS:-}${MYOS_ARGS:+ $MYOS_ARGS}")
[ -n "$_sub" ] || _sub=list
case $_sub in
list) myos_cert_list ;;
domains) myos_cert_write_domains ;;
issue) myos_cert_run "" ;;
renew) myos_cert_run "--cron" ;;
show) myos_cert_show ;;
*) myos_die "$MYOS_E_USAGE" "myos cert <list|domains|issue|renew|show>" ;;
esac
}
# myos_cert_list the certificates that would be asked for, and how
myos_cert_list() {
_groups=$(myos_cert_groups)
[ -n "$_groups" ] || {
printf 'no routed hostname: nothing to certify\n'
return 0
}
printf '%s%-46s %-9s %s%s\n' "$MYOS_C_HIGHLIGHT" CERTIFICATE CHALLENGE NAMES "$MYOS_C_RESET"
printf '%s\n' "$_groups" | while IFS= read -r _line; do
_cn=$(myos_firstword "$_line")
case $_line in
*'*.'*) _ch=dns-01 ;;
*) _ch=http-01 ;;
esac
printf '%-46s %-9s %s\n' "$_cn" "$_ch" "$_line"
done
myos_cert_needs_dns &&
myos_info "a wildcard is asked for: dns-01 needs MYOS_CERT_HOOK to talk to your dns provider"
return 0
}
# myos_cert_write_domains the domains.txt dehydrated reads
myos_cert_write_domains() {
_dir=${MYOS_CERT_DIR:-$WORKDIR/.myos/dehydrated}
_file=$_dir/domains.txt
_groups=$(myos_cert_groups)
[ -n "$_groups" ] || { myos_warning "no routed hostname: not writing $_file"; return 0; }
myos_run mkdir -p "$_dir"
if [ "${DRYRUN:-false}" = true ]; then
printf 'would write %s:\n%s\n' "$_file" "$_groups"
else
printf '%s\n' "$_groups" > "$_file"
printf '%s\n' "$_file"
fi
}
# myos_cert_run ARGS run dehydrated in the host stack, on the domains derived
myos_cert_run() {
myos_cert_write_domains >/dev/null || return $?
_args=$1
[ -n "${MYOS_CERT_STAGING:-}" ] && _args="$_args --staging"
case ${MYOS_ARGS:-} in
*--staging*) _args="$_args --staging" ;;
esac
case ${MYOS_ARGS:-} in
*--force*) _args="$_args --force" ;;
esac
MYOS_ARGS="$_args" SERVICE=${SERVICE:-dehydrated} \
MYOS_STACKS="host/dehydrated" myos_cert_exec
}
myos_cert_exec() {
# shellcheck source=lib/cmd/exec.sh
. "$MYOS_ROOT/lib/cmd/exec.sh"
myos_cmd_exec
}
# myos_cert_show the certificates that exist, and when they expire
myos_cert_show() {
_vol=${HOST_DOCKER_VOLUME:-${HOSTNAME:-localhost}}
# shellcheck disable=SC2016 # the script runs in the container, not here
myos_run docker run --rm -v "$_vol:/host" alpine:3.20 sh -c '
apk add -q openssl 2>/dev/null
for c in /host/certs/*-cert.pem; do
[ -f "$c" ] || continue
n=$(basename "$c" -cert.pem)
e=$(openssl x509 -in "$c" -noout -enddate 2>/dev/null | sed "s/notAfter=//")
i=$(openssl x509 -in "$c" -noout -issuer 2>/dev/null | sed "s/.*CN *= *//;s/,.*//")
printf "%-46s %-28s %s\n" "$n" "$e" "$i"
done'
}
+39
View File
@@ -0,0 +1,39 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# myos doctor check that this installation can actually run a stack
myos_cmd_doctor() {
_rc=0
_ok() { printf ' %-28s %s\n' "$1" "$2"; }
_bad() { printf ' %-28s %s%s%s\n' "$1" "$MYOS_C_ERROR" "$2" "$MYOS_C_RESET"; _rc=$MYOS_E_NOREQ; }
printf 'myos %s at %s\n' "$MYOS_VERSION" "$MYOS_ROOT"
printf 'requirements:\n'
if myos_have docker; then _ok docker "$(docker version --format '{{.Client.Version}}' 2>/dev/null || echo present)"
else _bad docker "not found"; fi
if _c=$(myos_compose_bin 2>/dev/null); then _ok "compose" "$_c"; else _bad compose "docker compose >= $MYOS_COMPOSE_MIN_VERSION not found"; fi
if [ "${DRYRUN:-false}" != true ]; then
if docker info >/dev/null 2>&1; then _ok "docker daemon" "reachable${DOCKER_HOST:+ via $DOCKER_HOST}"
else _bad "docker daemon" "unreachable${DOCKER_HOST:+ ($DOCKER_HOST)}"; fi
fi
printf 'configuration:\n'
for _f in $(myos_conf_files); do _ok "$_f" "read"; done
[ -f "${HOME:-}/.config/myos/config" ] && _ok "${HOME:-}/.config/myos/config" "read"
[ -f "$WORKDIR/.env" ] && _ok "$WORKDIR/.env" "read"
_ok ENV "$ENV"
_ok USER "$USER"
_ok HOSTNAME "$HOSTNAME"
_ok DOMAIN "$DOMAIN"
_ok "project format" "${MYOS_PROJECT_FORMAT:-user-env-app}"
printf 'stacks:\n'
_p=$(myos_path)
if [ -n "$_p" ]; then printf '%s\n' "$_p" | tr ':' '\n' | sed 's/^/ /'
else _bad "stack path" "empty"; fi
# a .env written for the make engine can hold values the shell reads differently
if [ -f "$WORKDIR/.env" ] && grep -qE '\$\(|\$\{[a-z]' "$WORKDIR/.env"; then
printf ' %-28s %s%s%s\n' "$WORKDIR/.env" "$MYOS_C_WARN" "contains make expansions" "$MYOS_C_RESET"
fi
return "$_rc"
}
+26
View File
@@ -0,0 +1,26 @@
#shellcheck shell=sh
# myos env-update fill the .env of the workdir from the .env.dist it finds
#
# Templates are read from the workdir and from every requested stack, so a
# stack can ship the variables it expects and their defaults.
myos_cmd_env_update() {
_target=${ENV_FILE:-$WORKDIR/.env}
_dists=$MYOS_ARGS
if [ -z "$_dists" ]; then
for _ref in $MYOS_STACKS; do
_d=$(myos_stack_resolve "$_ref" 2>/dev/null) || continue
for _c in "$_d/.env.dist" "$_d/$(myos_stack_name "$_ref").env.dist"; do
[ -f "$_c" ] && _dists="${_dists:+$_dists }$_c"
done
done
for _c in "$WORKDIR/.env.dist" "$WORKDIR/.env.example" "$WORKDIR/.env.sample"; do
[ -f "$_c" ] && _dists="${_dists:+$_dists }$_c"
done
fi
[ -n "$_dists" ] || { myos_warning "no .env.dist found for $MYOS_STACKS"; return 0; }
for _dist in $_dists; do
myos_info "env-update $_target from $_dist"
myos_env_update "$_target" "$_dist" "$WORKDIR/.env.$ENV"
done
printf '%s\n' "$_target"
}
+48
View File
@@ -0,0 +1,48 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# myos env [VAR...] show resolved variables (replaces the make print-VAR target)
# myos_env_print NAME VALUE
# Same shape as the make print-<VAR> target: the name padded to 37 columns in
# the highlight colour, then the value in the value colour.
myos_env_print() {
printf '%s%-37s%s%s%s%s\n' \
"$MYOS_C_HIGHLIGHT" "$1" "$MYOS_C_RESET" "$MYOS_C_VALUE" "$2" "$MYOS_C_RESET"
}
myos_cmd_env() {
_vars=${MYOS_VARS:-}
[ -n "$_vars" ] || _vars=$MYOS_ARGS
[ -n "$_vars" ] || _vars=$MYOS_REFS_RAW
if [ -z "$_vars" ]; then
_vars="ENV USER HOSTNAME DOMAIN WORKDIR MYOS_PATH SCOPE STACK COMPOSE_PROJECT_NAME COMPOSE_FILE"
fi
for _v in $_vars; do
case $_v in
MYOS_PATH) myos_env_print "$_v" "$(myos_path)" ;;
COMPOSE_FILE) myos_env_print "$_v" "$(myos_all_compose_files | tr '\n' ' ' | sed 's/ $//')" ;;
COMPOSE_PROJECT_NAME) myos_env_print "$_v" "$(myos_first_project)" ;;
COMPOSE_SERVICE_NAME) myos_env_print "$_v" "$(myos_service_name "$(myos_first_project)")" ;;
COMPOSE_FILE_SUFFIX) myos_env_print "$_v" "$(myos_compose_suffixes)" ;;
STACK) myos_env_print "$_v" "$(printf '%s' "$MYOS_STACKS" | tr '\n' ' ' | sed 's/ $//')" ;;
SCOPE) myos_env_print "$_v" "$(myos_first_scope)" ;;
APP|APP_NAME) myos_env_print "$_v" "$(myos_first_app)" ;;
DOCKER_REPOSITORY) myos_env_print "$_v" "$(printf '%s' "$(myos_first_project)" | tr '_-' '//')" ;;
DOCKER_NETWORK_DEFAULT) myos_env_print "$_v" "$(myos_network_default "$(myos_first_project)")" ;;
DOCKER_NETWORK_PRIVATE) myos_env_print "$_v" "$(myos_network_private "$USER" "$ENV")" ;;
DOCKER_NETWORK_PUBLIC) myos_env_print "$_v" "$(myos_network_public "${HOSTNAME:-}")" ;;
DOCKER_NETWORK)
if [ "$(myos_first_scope)" = user ]; then myos_env_print "$_v" "$USER"
else myos_env_print "$_v" "$(myos_network_private "$USER" "$ENV")"; fi ;;
# HOST_STACK and USER_STACK are what the make engine called the scope
HOST_STACK)
if [ "$(myos_first_scope)" = host ]; then myos_env_print "$_v" host
else myos_env_print "$_v" ""; fi ;;
USER_STACK)
if [ "$(myos_first_scope)" = user ]; then myos_env_print "$_v" User
else myos_env_print "$_v" ""; fi ;;
*) myos_env_print "$_v" "$(myos_var "$_v")" ;;
esac
done
}
+28
View File
@@ -0,0 +1,28 @@
#shellcheck shell=sh
# myos exec <stack> [-- command...] run a command in a running service
# myos run <stack> [-- command...] run it in a new container
#
# The service defaults to the stack name, which is what it is called in most
# stacks; SERVICE=<name> picks another one.
myos_cmd_exec() { myos_service_command exec; }
myos_cmd_run() { myos_service_command run; }
myos_service_command() {
_what=$1
_ref=$(printf '%s' "$MYOS_STACKS" | head -1)
[ -n "$_ref" ] || myos_die "$MYOS_E_USAGE" "usage: myos $_what <stack> [SERVICE=name] -- command..."
_service=${SERVICE:-$(myos_stack_name "$_ref")}
_files=$(myos_stack_compose_files "$_ref") || return $?
_fw=$(myos_framework_compose_files)
[ -n "$_fw" ] && _files="$_files
$_fw"
_app=$(myos_stack_name "$_ref")
case $_ref in .|./*|/*|../*) _app=$(basename "$(myos_stack_resolve "$_ref")") ;; esac
_project=$(myos_project_name "$(myos_scope "$_ref")" "$USER" "$ENV" "$_app")
case $_what in
exec) _opts="" ;;
run) _opts=${DOCKER_COMPOSE_RUN_OPTIONS:---rm} ;;
esac
# shellcheck disable=SC2086 # options and arguments are deliberate word lists
myos_compose "$_project" "$_files" -- "$_what" $_opts "$_service" ${MYOS_ARGS:-}
}
+36
View File
@@ -0,0 +1,36 @@
#shellcheck shell=sh
# myos export print every setting the requested stacks declare, as KEY=value.
#
# One call, so a Makefile can read the whole set at once:
# $(eval $(shell myos export STACK=host))
# Asking for each variable separately costs a process per variable.
myos_cmd_export() {
# shellcheck disable=SC2209 # these are literal format names, not commands
_fmt=sh
# shellcheck disable=SC2209
case ${MYOS_ARGS:-}${MYOS_VARS:-} in *--make*) _fmt=make ;; esac
for _v in $(myos_declared_defaults); do
_val=$(myos_var "$_v")
case $_fmt in
make)
# make would expand a $ and start a comment at a #, and := stops it
# from expanding the value again later
printf '%s := %s\n' "$_v" "$(printf '%s' "$_val" | sed -e 's/\$/$$/g' -e 's/#/\\#/g')" ;;
*) printf '%s=%s\n' "$_v" "$_val" ;;
esac
done
}
# myos_declared_defaults the variables the loaded hooks declare
myos_declared_defaults() {
for _ref in $MYOS_STACKS; do
for _d in $(myos_stack_dirs "$_ref"); do
for _f in "$_d"/_stack.sh "$_d/$(myos_stack_name "$_ref").sh"; do
[ -f "$_f" ] && sed -n 's/^myos_default_\([A-Za-z_][A-Za-z0-9_]*\)().*/\1/p' "$_f"
done
for _f in "$_d"/_stack.env "$_d/$(myos_stack_name "$_ref").env"; do
[ -f "$_f" ] && myos_dotenv_parse "$_f" | sed 's/=.*//'
done
done
done | sort -u
}
+96
View File
@@ -0,0 +1,96 @@
#shellcheck shell=sh
# myos expose [--strict] what the stacks publish, and to whom
#
# Two readings are joined: the compose files as written, which say which
# binding each port asks for, and the resolved configuration, which says the
# address it ends up on. The first is the intent, the second is the fact, and
# reporting both is the point: a port nobody bound answers the internet, and on
# linux the host firewall does not see it, because docker writes its own rules.
#
# --strict exits 1 when a port is published without a binding.
myos_cmd_expose() {
_strict=false
case ${MYOS_ARGS:-}${MYOS_VARS:-} in *--strict*) _strict=true ;; esac
_rows=$(myos_expose_rows)
[ -n "$_rows" ] || {
printf 'no published port: nothing is reachable from outside the docker network\n'
return 0
}
printf '%s%-20s %-14s %-22s %-6s %s%s\n' \
"$MYOS_C_HIGHLIGHT" STACK SERVICE "PUBLISHED ON" PORT BINDING "$MYOS_C_RESET"
_bad=0
_oIFS=$IFS; IFS='
'
for _row in $_rows; do
IFS=$_oIFS
_st=${_row%%|*}; _r=${_row#*|}
_sv=${_r%%|*}; _r=${_r#*|}
_on=${_r%%|*}; _r=${_r#*|}
_pt=${_r%%|*}; _sc=${_r#*|}
if [ "$_sc" = unbound ]; then
_bad=$((_bad + 1))
printf '%-20s %-14s %s%-22s%s %-6s %s%s%s\n' "$_st" "$_sv" \
"$MYOS_C_WARN" "$_on" "$MYOS_C_RESET" "$_pt" "$MYOS_C_WARN" "$_sc" "$MYOS_C_RESET"
else
printf '%-20s %-14s %-22s %-6s %s\n' "$_st" "$_sv" "$_on" "$_pt" "$_sc"
fi
IFS='
'
done
IFS=$_oIFS
if [ "$_bad" -gt 0 ]; then
myos_warning "$_bad port(s) published without a binding: docker opens them on every address"
# shellcheck disable=SC2016 # the variable name is the message, not a value
myos_warning 'bind them: ports: ["${MYOS_BIND_PRIVATE}::<port>"] for a service behind the load balancer'
[ "$_strict" = true ] && return "$MYOS_E_FAIL"
fi
return 0
}
# myos_expose_rows STACK|SERVICE|ADDR:PORT|CONTAINER_PORT|BINDING
myos_expose_rows() {
for _ref in $MYOS_STACKS; do
_files=$(myos_stack_compose_files "$_ref" 2>/dev/null) || continue
[ -n "$_files" ] || continue
_fw=$(myos_framework_compose_files)
[ -n "$_fw" ] && _files="$_files
$_fw"
_app=$(myos_stack_name "$_ref")
_project=$(myos_project_name "$(myos_scope "$_ref")" "$USER" "$ENV" "$_app")
# what the files ask for, later overlays overriding earlier ones
_decl=$(mktemp "${TMPDIR:-/tmp}/myos-expose.XXXXXX")
# shellcheck disable=SC2086 # a newline separated list of paths
myos_expose_declared $_files > "$_decl" 2>/dev/null
DRYRUN=false myos_compose "$_project" "$_files" -- config 2>/dev/null |
myos_expose_resolved |
while IFS='|' read -r _v _t _o; do
_b=$(awk -F'|' -v s="$_v" -v p="$_t" '$1==s && $2==p {last=$3} END {print last}' "$_decl")
printf '%s|%s|%s|%s|%s\n' "$_ref" "$_v" "$_o" "$_t" "${_b:-unbound}"
done
rm -f "$_decl"
done
}
# myos_expose_resolved (compose config on stdin) -> SERVICE|CONTAINER_PORT|ADDR:PORT
# compose normalises every port to the long form, so one shape is enough
myos_expose_resolved() {
awk '
/^services:/ { insvc = 1; next }
insvc && /^ [a-zA-Z0-9_.-]+:/ { svc = $1; sub(/:$/, "", svc); inports = 0 }
insvc && /^ ports:/ { inports = 1; next }
inports && /^ [a-z]/ { inports = 0 }
inports && /host_ip:/ { ip = $2 }
inports && /published:/ { pub = $2; gsub(/"/, "", pub) }
inports && /target:/ { tgt = $2 }
inports && /protocol:/ {
# compose leaves published empty when docker picks the port at run time
printf "%s|%s|%s:%s\n", svc, tgt, (ip == "" ? "0.0.0.0" : ip), (pub == "" ? "auto" : pub)
ip = ""; pub = ""; tgt = ""
}
'
}
+36
View File
@@ -0,0 +1,36 @@
#shellcheck shell=sh
# myos ls [--groups] list the stacks myos can see, and where they come from
myos_cmd_ls() {
case ${MYOS_REFS_RAW:-}${MYOS_ARGS:-} in
*--groups*) myos_ls_groups; return 0 ;;
esac
_IFS=$IFS; IFS=:
for _d in $(myos_path); do
IFS=$_IFS
printf '%s%s%s\n' "$MYOS_C_INFO" "$_d" "$MYOS_C_RESET"
for _s in "$_d"/*; do
[ -d "$_s" ] || continue
_n=$(basename "$_s")
_f=$(find "$_s" -maxdepth 1 \( -name '*.yml' -o -name '*.yaml' \) | wc -l | tr -d ' ')
[ "$_f" = 0 ] && continue
printf ' %-24s %s compose file(s)\n' "$_n" "$_f"
done
IFS=:
done
IFS=$_IFS
}
myos_ls_groups() {
_IFS=$IFS; IFS=:
for _d in $(myos_path); do
IFS=$_IFS
for _f in "$_d"/*.mk "$_d"/*.env "$_d"/*/*.mk; do
[ -f "$_f" ] || continue
_n=$(basename "$_f"); _n=${_n%.mk}; _n=${_n%.env}
_v=$(myos_group_value "$_n")
[ -n "$_v" ] && printf '%-16s %s\n' "$_n" "$_v"
done
IFS=:
done
IFS=$_IFS
}
+10
View File
@@ -0,0 +1,10 @@
#shellcheck shell=sh
# myos recreate remove the containers and create them again
# myos reload the same, under the name the make engine used
myos_cmd_recreate() {
# shellcheck source=lib/cmd/_compose.sh
. "$MYOS_ROOT/lib/cmd/_compose.sh"
MYOS_ARGS="--force-recreate ${MYOS_ARGS:-}"
myos_cmd_compose up
}
myos_cmd_reload() { myos_cmd_recreate; }
+4
View File
@@ -0,0 +1,4 @@
#shellcheck shell=sh
# myos reload: see lib/cmd/recreate.sh
# shellcheck source=lib/cmd/recreate.sh
. "$MYOS_ROOT/lib/cmd/recreate.sh"
+4
View File
@@ -0,0 +1,4 @@
#shellcheck shell=sh
# myos run: see lib/cmd/exec.sh
# shellcheck source=lib/cmd/exec.sh
. "$MYOS_ROOT/lib/cmd/exec.sh"
+13
View File
@@ -0,0 +1,13 @@
#shellcheck shell=sh
# shellcheck source=lib/cmd/_compose.sh
# myos scale <stack> SERVICE=<name> NUM=<n> run n containers of a service
. "$MYOS_ROOT/lib/cmd/_compose.sh"
myos_cmd_scale() {
_ref=$(printf '%s' "$MYOS_STACKS" | head -1)
[ -n "$_ref" ] || myos_die "$MYOS_E_USAGE" "usage: myos scale <stack> SERVICE=name NUM=n"
_service=${SERVICE:-$(myos_stack_name "$_ref")}
[ -n "${NUM:-}" ] || myos_die "$MYOS_E_USAGE" "myos scale needs NUM=<n>"
MYOS_ARGS="--scale $_service=$NUM ${MYOS_ARGS:-}"
myos_cmd_compose up
}
+7
View File
@@ -0,0 +1,7 @@
#shellcheck shell=sh
# myos status what is running, under the name the make engine used for ps
myos_cmd_status() {
# shellcheck source=lib/cmd/_compose.sh
. "$MYOS_ROOT/lib/cmd/_compose.sh"
myos_cmd_compose ps
}
+69
View File
@@ -0,0 +1,69 @@
#shellcheck shell=sh
# compose: find a usable docker compose, and call it once per project.
#
# The legacy engine ran one `docker compose up` per sub-stack, each time with
# the full file list, so N sub-stacks meant N identical calls. The CLI calls
# compose once per project (documented in spec/golden/DELTAS.md).
MYOS_COMPOSE_MIN_VERSION=${COMPOSE_VERSION:-2.24.4}
# myos_compose_bin print the compose command to use, fail with MYOS_E_NOREQ
myos_compose_bin() {
[ -n "${MYOS_COMPOSE_BIN:-}" ] && { printf '%s' "$MYOS_COMPOSE_BIN"; return 0; }
if myos_have docker; then
_v=$(docker compose version --short 2>/dev/null)
if myos_verle "$MYOS_COMPOSE_MIN_VERSION" "$_v"; then printf 'docker compose'; return 0; fi
fi
if myos_have docker-compose; then
_v=$(docker-compose version --short 2>/dev/null)
if myos_verle "$MYOS_COMPOSE_MIN_VERSION" "$_v"; then printf 'docker-compose'; return 0; fi
fi
myos_error "docker compose >= $MYOS_COMPOSE_MIN_VERSION not found (install the docker compose plugin or docker-compose)"
return "$MYOS_E_NOREQ"
}
# myos_compose PROJECT FILES -- ARGS...
# FILES is a newline separated list; the project directory is that of the first
# file, so relative build contexts and env_file entries keep working.
myos_compose() {
_project=$1; _files=$2; shift 2
[ "${1:-}" = "--" ] && shift
[ -n "$_files" ] || { myos_error "no compose file for project $_project"; return "$MYOS_E_NOSTACK"; }
_bin=$(myos_compose_bin) || return $?
_fargs=""
_first=""
for _f in $_files; do
[ -n "$_first" ] || _first=$_f
_fargs="$_fargs -f $_f"
done
_dir=$(dirname "$_first")
# the variables the compose files reference, plus the network names: those
# live in networks.yml, which is appended after the scan
# shellcheck disable=SC2086 # both are deliberate word lists
_vars=$(myos_env_vars $_files)
# shellcheck disable=SC2086
_envargs=$(myos_env_export $_vars DOCKER_NETWORK_DEFAULT DOCKER_NETWORK_PRIVATE DOCKER_NETWORK_PUBLIC COMPOSE_SERVICE_NAME)
if [ "${DRYRUN:-false}" = true ]; then
# shellcheck disable=SC2086 # printed, not executed
printf '%s%s -p %s --project-directory %s %s\n' "$_bin" "$_fargs" "$_project" "$_dir" "$*"
return 0
fi
# Export the variables in a subshell rather than through env(1): a value may
# contain spaces, and the command line must still be split on spaces (the
# compose binary can be the two words "docker compose").
(
while IFS= read -r _kv; do
[ -n "$_kv" ] || continue
# shellcheck disable=SC2163 # _kv is a NAME=value pair, not a name
export "$_kv"
done <<EOF
$_envargs
EOF
# shellcheck disable=SC2086 # _bin and _fargs are deliberate word lists
exec $_bin --ansi=auto $_fargs -p "$_project" --project-directory "$_dir" "$@"
)
}
+173
View File
@@ -0,0 +1,173 @@
#shellcheck shell=sh
# config: where settings come from, and in which order.
#
# Layers, last one wins:
# defaults < /etc/conf.d/myos, /etc/default/myos < ~/.config/myos/config
# < <workdir>/.env < <workdir>/.env.<env> < environment < CLI VAR=val
# MYOS_CONF_PRIORITY=system restores the old make behaviour where the system
# file won over the project .env.
#
# Files are dotenv: KEY=value, one per line, # comments, optional quotes.
# They are parsed, never sourced: a value never runs as code.
# myos_dotenv_parse FILE print normalized KEY=value lines
myos_dotenv_parse() {
[ -f "$1" ] || return 0
sed -e 's/\r$//' -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*$/d' "$1" |
while IFS= read -r _line; do
case $_line in *=*) ;; *) continue ;; esac
_k=${_line%%=*}
_v=${_line#*=}
_k=$(printf '%s' "$_k" | tr -d '[:space:]')
# whitespace around the = is not part of the value (make: s/[[:space:]]*=[[:space:]]*/=/)
_v=${_v#"${_v%%[![:space:]]*}"}
case $_k in ''|*[!A-Za-z0-9_]*) continue ;; esac
# strip one layer of matching quotes
case $_v in
\"*\") _v=${_v#\"}; _v=${_v%\"} ;;
\'*\') _v=${_v#\'}; _v=${_v%\'} ;;
esac
printf '%s=%s\n' "$_k" "$_v"
done
}
# myos_dotenv_load FILE set the variables of FILE that are not already set
# (an already exported variable wins, as `?=` does in make)
myos_dotenv_load() {
[ -f "$1" ] || return 0
while IFS= read -r _kv; do
# an empty file still yields one empty line through the here-document
[ -n "$_kv" ] || continue
_k=${_kv%%=*}
[ -n "$_k" ] || continue
[ -n "$(myos_var "$_k")" ] && continue
eval "$_k=\${_kv#*=}"
done <<EOF
$(myos_dotenv_parse "$1")
EOF
}
# myos_dotenv_has FILE KEY
myos_dotenv_has() { myos_dotenv_parse "$1" | grep -q "^$2="; }
# myos_conf_files the system config files, most significant first
myos_conf_files() {
[ -n "${MYOS_CONF:-}" ] && { printf '%s\n' "$MYOS_CONF"; return 0; }
for _f in /etc/conf.d/myos /etc/default/myos; do
[ -r "$_f" ] && printf '%s\n' "$_f"
done
return 0
}
# myos_env_vars FILE... the ${VAR} names referenced by the compose files
# (make: env-vars). $$VAR is a compose escape, not a shell variable.
myos_env_vars() {
[ $# -gt 0 ] || return 0
sed 's/\$\$//g' "$@" 2>/dev/null |
grep -oE '\$\{?[A-Z0-9_]+' |
tr -d '{}$' |
sort -u |
tr '\n' ' ' |
sed 's/ $//'
}
# myos_env_export VAR... print VAR='value' for each variable that has a value,
# ready to be passed to env(1)
myos_env_export() {
for _v in "$@"; do
_val=$(myos_var "$_v")
[ -n "$_val" ] && printf "%s=%s\n" "$_v" "$_val"
done
return 0
}
# myos_expand STRING substitute ${VAR} and $(command) in STRING.
# shellcheck disable=SC2016 # the single quotes are deliberate: these patterns
# match the literal characters ${ and $( in the input, they are not expansions
# This is what the make engine did when it generated a .env out of a .env.dist:
# ${VAR} takes the current value, $(cmd) runs the command. Nothing else is
# interpreted, so the rest of the line can hold anything.
myos_expand() {
_in=$1
_guard=0
while [ "$_guard" -lt 16 ]; do
_guard=$((_guard + 1))
case $_in in
*'${'*'}'*)
_pre=${_in%%'${'*}
_rest=${_in#*'${'}
_name=${_rest%%\}*}
_post=${_rest#*\}}
case $_name in
''|*[!A-Za-z0-9_]*) _in="$_pre\${$_name}$_post"; break ;;
esac
_in="$_pre$(myos_var "$_name")$_post" ;;
*) break ;;
esac
done
_guard=0
while [ "$_guard" -lt 16 ]; do
_guard=$((_guard + 1))
case $_in in
*'$('*')'*)
_pre=${_in%%'$('*}
_rest=${_in#*'$('}
_cmd=${_rest%%)*}
_post=${_rest#*)}
_in="$_pre$(eval "$_cmd" 2>/dev/null)$_post" ;;
*) break ;;
esac
done
printf '%s' "$_in"
}
# myos_env_update FILE DIST [OVER...]
# Add to FILE every variable of DIST that is missing from it, expanded.
# A variable that already has a value keeps it, whether it comes from the
# environment, from FILE, or from one of the OVER files: the .env is a record
# of the choices already made, never something that overwrites them.
myos_env_update() {
_file=$1; _dist=$2; shift 2
[ -f "$_dist" ] || return 0
[ -e "$_file" ] || : > "$_file"
# what the overrides pin, read before anything else
for _over in "$@"; do
[ -f "$_over" ] || continue
myos_dotenv_load "$_over"
done
# Which keys already hold a choice, made in the environment, in the .env or
# in an override. Those are kept verbatim; everything else is a template.
_preset=" "
while IFS= read -r _kv; do
[ -n "$_kv" ] || continue
_k=${_kv%%=*}
[ -n "$_k" ] || continue
[ -n "$(myos_var "$_k")" ] && _preset="$_preset$_k "
done <<EOF
$(myos_dotenv_parse "$_dist")
EOF
# Make the templates themselves visible, so a line may refer to a variable
# defined further down the file, as the make engine allowed.
myos_dotenv_load "$_dist"
_added=0
while IFS= read -r _kv; do
[ -n "$_kv" ] || continue
_k=${_kv%%=*}
[ -n "$_k" ] || continue
myos_dotenv_has "$_file" "$_k" && continue
case $_preset in
*" $_k "*) _v=$(myos_var "$_k") ;;
*) _v=$(myos_expand "${_kv#*=}") ;;
esac
printf '%s=%s\n' "$_k" "$_v" >> "$_file"
_added=$((_added + 1))
done <<EOF
$(myos_dotenv_parse "$_dist")
EOF
[ "$_added" -gt 0 ] && myos_info "added $_added variable(s) to $_file"
return 0
}
+72
View File
@@ -0,0 +1,72 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# context: what the requested stacks resolve to, and the framework variables a
# stack hook may read.
#
# A hook is written the way a .mk was: it may mention COMPOSE_PROJECT_NAME,
# USER or DOMAIN and expect the framework value. Those are registered as lazy
# defaults, so each is computed when read and an explicit value still wins.
# myos_all_compose_files every compose file of every requested stack, in order
myos_all_compose_files() {
for _ref in $MYOS_STACKS; do
myos_stack_compose_files "$_ref" 2>/dev/null
done
myos_framework_compose_files
return 0
}
# myos_first_app / myos_first_scope / myos_first_project
# describe the first requested stack, which is what the introspection commands
# report when several stacks are asked for at once.
myos_first_app() {
for _ref in $MYOS_STACKS; do
case $_ref in
.|./*|/*|../*) basename "$(myos_stack_resolve "$_ref" 2>/dev/null)" ;;
*) myos_stack_name "$_ref" ;;
esac
return 0
done
}
myos_first_scope() {
for _ref in $MYOS_STACKS; do myos_scope "$_ref"; return 0; done
}
myos_first_project() {
for _ref in $MYOS_STACKS; do
myos_project_name "$(myos_scope "$_ref")" "$USER" "$ENV" "$(myos_first_app)"
return 0
done
}
# myos_context_defaults register the framework variables as lazy defaults
# shellcheck disable=SC2329 # these are reached through myos_var
myos_context_defaults() {
myos_default_APP() { myos_first_app; }
myos_default_APP_NAME() { myos_name "$(myos_first_app)"; }
myos_default_SCOPE() { myos_first_scope; }
myos_default_COMPOSE_PROJECT_NAME() { myos_first_project; }
myos_default_COMPOSE_SERVICE_NAME() { myos_service_name "$(myos_first_project)"; }
myos_default_DOCKER_REPOSITORY() { printf '%s' "$(myos_first_project)" | tr '_-' '//'; }
myos_default_DOCKER_NETWORK_DEFAULT() { myos_network_default "$(myos_first_project)"; }
myos_default_DOCKER_NETWORK_PRIVATE() { myos_network_private "$USER" "$ENV"; }
myos_default_DOCKER_NETWORK_PUBLIC() { myos_network_public "${HOSTNAME:-}"; }
myos_default_DOCKER_NETWORK() { myos_network_private "$USER" "$ENV"; }
myos_default_DOCKER_IMAGE_TAG() { printf 'latest'; }
myos_default_GIT_USER() { printf '%s' "$USER"; }
myos_default_HOST() { myos_addprefix "${HOSTNAME:-}." "$(myos_var DOMAIN)"; }
myos_default_HOSTNAME() { printf '%s' "${HOSTNAME:-}"; }
myos_default_DOMAINNAME() { myos_firstword "$(myos_var DOMAIN)"; }
# the addresses a stack binds its published ports to
myos_default_MYOS_BIND_PUBLIC() { myos_bind public; }
myos_default_MYOS_BIND_PRIVATE() { myos_bind private; }
myos_default_MYOS_BIND_MESH() { myos_bind mesh; }
myos_default_MACHINE() { uname -m 2>/dev/null; }
myos_default_SYSTEM() { uname -s 2>/dev/null; }
myos_default_HOST_COMPOSE_PROJECT_NAME() { printf '%s' "${HOSTNAME:-}"; }
myos_default_HOST_DOCKER_VOLUME() { printf '%s' "${HOSTNAME:-}"; }
myos_default_HOST_DOCKER_REPOSITORY() { printf '%s' "${HOSTNAME:-}" | tr '_-' '//'; }
myos_default_USER_COMPOSE_PROJECT_NAME() { myos_resu "${MAIL:-}" | tr '.' '-'; }
myos_default_RESU() { myos_resu "${MAIL:-}"; }
}
+59
View File
@@ -0,0 +1,59 @@
#shellcheck shell=sh
# core: logging, error handling and command execution.
#
# Every myos command goes through myos_run, which honours DRYRUN by printing
# the command instead of running it. Messages go to stderr so that stdout stays
# usable for data (myos env, myos config, myos ls).
# Exit codes and colors are consumed by the other lib/ files and by bin/myos.
# shellcheck disable=SC2034
MYOS_E_OK=0 # success
MYOS_E_FAIL=1 # command failed
MYOS_E_USAGE=2 # bad invocation
MYOS_E_NOSTACK=3 # stack not found
MYOS_E_NOREQ=4 # missing requirement
# myos_colors decide whether to emit colour.
# MYOS_COLOR=always|never|auto (default auto: only when stdout is a terminal).
# The make engine always emitted the escape codes, even into a pipe.
myos_colors() {
_want=${MYOS_COLOR:-auto}
[ -n "${NO_COLOR:-}" ] && _want=never
case $_want in
never) _want=no ;;
always) _want=yes ;;
*) if [ -t 1 ] && [ "${TERM:-dumb}" != dumb ]; then _want=yes; else _want=no; fi ;;
esac
if [ "$_want" = yes ]; then
MYOS_C_ERROR=$(printf '\033[31m'); MYOS_C_WARN=$(printf '\033[01;33m')
MYOS_C_INFO=$(printf '\033[33m'); MYOS_C_DEBUG=$(printf '\033[01;34m')
MYOS_C_VALUE=$(printf '\033[36m'); MYOS_C_RESET=$(printf '\033[0m')
MYOS_C_HIGHLIGHT=$(printf '\033[32m')
else
MYOS_C_ERROR=; MYOS_C_WARN=; MYOS_C_INFO=; MYOS_C_DEBUG=; MYOS_C_VALUE=; MYOS_C_RESET=
MYOS_C_HIGHLIGHT=
fi
}
myos_error() { printf '%sERROR:%s %s\n' "$MYOS_C_ERROR" "$MYOS_C_RESET" "$*" >&2; }
myos_warning() { printf '%sWARNING:%s %s\n' "$MYOS_C_WARN" "$MYOS_C_RESET" "$*" >&2; }
myos_info() { [ -n "${VERBOSE:-}" ] && printf '%s%s%s\n' "$MYOS_C_INFO" "$*" "$MYOS_C_RESET" >&2; return 0; }
myos_debug() { [ -n "${DEBUG:-}" ] && printf '%s%s%s\n' "$MYOS_C_DEBUG" "$*" "$MYOS_C_RESET" >&2; return 0; }
# myos_die CODE MESSAGE...
myos_die() { _code=$1; shift; myos_error "$*"; exit "$_code"; }
# myos_run CMD... run a command, or print it when DRYRUN is true
myos_run() {
if [ "${DRYRUN:-false}" = true ]; then
printf '%s\n' "$*"
return 0
fi
myos_debug "+ $*"
"$@"
}
# myos_have CMD is a command available?
myos_have() { command -v "$1" >/dev/null 2>&1; }
myos_colors
+119
View File
@@ -0,0 +1,119 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# expose: which addresses a published port binds to.
#
# Docker writes its own firewall rules, so on linux a port published with
# `-p 8080:80` answers the internet whatever the host firewall says. ufw-docker
# patches that afterwards, on linux only, as root.
#
# The portable answer is to publish where you mean to in the first place:
# `-p 127.0.0.1:8080:80` only ever listens on the loopback, identically on
# linux and on macOS, with no firewall and no privilege. A stack says which
# scope a port belongs to, and myos resolves the address.
#
# public the internet: a load balancer, a public DNS or mail service
# mesh the private network between the hosts of the fleet
# private this host only: everything the load balancer reaches for you
#
# A stack does not declare its scope on the side: it is which of these it binds
# to, read from the compose file. One source of truth, which cannot drift from
# what is actually published. MYOS_BIND_<SCOPE> sets the address of a scope on
# a given host, which is the part that belongs to the host rather than to the
# stack.
# myos_bind SCOPE the address a port of that scope binds to
myos_bind() {
case $1 in
public) printf '%s' "${MYOS_BIND_PUBLIC:-0.0.0.0}" ;;
mesh) printf '%s' "${MYOS_BIND_MESH:-$(myos_bind_mesh)}" ;;
private|*) printf '%s' "${MYOS_BIND_PRIVATE:-127.0.0.1}" ;;
esac
}
# myos_bind_mesh the address of the mesh interface, empty when there is none.
# Falls back to the private address so that a stack scoped to the mesh on a
# host that has none stays local rather than becoming public.
myos_bind_mesh() {
_if=${MYOS_MESH_IFACE:-}
if [ -z "$_if" ]; then
for _c in easytier tun0 tailscale0 mycelium wg0; do
if myos_iface_addr "$_c" >/dev/null 2>&1 && [ -n "$(myos_iface_addr "$_c")" ]; then
_if=$_c; break
fi
done
fi
[ -n "$_if" ] || { printf '%s' "${MYOS_BIND_PRIVATE:-127.0.0.1}"; return 0; }
_a=$(myos_iface_addr "$_if")
[ -n "$_a" ] || _a=${MYOS_BIND_PRIVATE:-127.0.0.1}
printf '%s' "$_a"
}
# myos_iface_addr NAME the first address of an interface, on linux or macOS
myos_iface_addr() {
if myos_have ip; then
ip -o addr show "$1" 2>/dev/null | awk '$3 ~ /^inet6?$/ {sub(/\/.*/,"",$4); print $4; exit}'
elif myos_have ifconfig; then
ifconfig "$1" 2>/dev/null | awk '$1 == "inet" || $1 == "inet6" {print $2; exit}'
fi
}
# myos_stack_prefix REF the prefix the settings of a stack use.
# A host stack is prefixed by HOST_, which is how the catalogue names them:
# HOST_FABIO_SERVICE_9998_TAGS, HOST_FTPS_UFW_DOCKER. Everything else uses the
# stack name alone: SUPABASE_KONG_SERVICE_8000_TAGS.
myos_stack_prefix() {
_n=$(myos_upper "$(myos_stack_name "$1")")
case $(myos_scope "$1") in
host) printf 'HOST_%s' "$_n" ;;
user) printf 'USER_%s' "$_n" ;;
*) printf '%s' "$_n" ;;
esac
}
# myos_expose_declared FILE... SERVICE|CONTAINER_PORT|SCOPE for every port a
# compose file publishes, read from the file as written rather than from the
# resolved configuration.
#
# The scope is not declared twice: it is which binding the file asks for.
# ${MYOS_BIND_PUBLIC}:443:443 public
# ${MYOS_BIND_PRIVATE}::8080 private
# ${MYOS_BIND_MESH}::7946 mesh
# 127.0.0.1:5432:5432 pinned to an address, deliberate but fixed
# 80 or 8080:80 unbound: docker binds every address, and
# nobody chose that
#
# Resolving first would lose the difference: ${MYOS_BIND_PRIVATE} and a
# hand-written 127.0.0.1 both become 127.0.0.1, and an unbound port becomes
# 0.0.0.0 exactly like a deliberate public one.
myos_expose_declared() {
awk '
function indent(line, n) { match(line, /^ */); return RLENGTH }
function emit(entry, e, scope, target, n, parts) {
e = entry
sub(/^ *- */, "", e)
gsub(/^["'"'"']|["'"'"']$/, "", e)
if (e ~ /\$\{MYOS_BIND_PUBLIC[^}]*\}/) scope = "public"
else if (e ~ /\$\{MYOS_BIND_MESH[^}]*\}/) scope = "mesh"
else if (e ~ /\$\{MYOS_BIND_PRIVATE[^}]*\}/) scope = "private"
else if (e ~ /^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+:/) scope = "pinned"
else if (e ~ /^\[/) scope = "pinned"
else scope = "unbound"
target = e
sub(/\/[a-z]+$/, "", target)
n = split(target, parts, ":")
target = parts[n]
if (target ~ /^[0-9]+(-[0-9]+)?$/) printf "%s|%s|%s\n", svc, target, scope
}
/^services:[ \t]*$/ { insvc = 1; svcind = -1; next }
!insvc { next }
# a service is the first level of keys under services:
/^ *[a-zA-Z0-9_.-]+:[ \t]*$/ && (svcind == -1 || indent($0) == svcind) {
if (svcind == -1) svcind = indent($0)
svc = $1; sub(/:$/, "", svc); inports = 0; next
}
/^ *ports:/ { inports = 1; portind = indent($0); next }
# the list items of a ports: block, whatever indent they use
inports && /^ *- / && indent($0) >= portind { emit($0); next }
inports && /^ *[a-zA-Z0-9_.-]+:/ { inports = 0 }
' "$@"
}
+69
View File
@@ -0,0 +1,69 @@
#shellcheck shell=sh
# shellcheck disable=SC1090 # hooks are sourced by a path built at run time
# hooks: the per-stack settings that used to live in a .mk file.
#
# A stack directory may ship, next to its compose files:
# _stack.env settings shared by every stack of the directory
# _stack.sh the same, computed
# <name>.env settings of one stack
# <name>.env.<env> the same, for one environment
# <name>.sh computed settings of one stack
# <name>.mk the legacy make snippet, still read for its groups
#
# A .sh hook declares lazy defaults (see lib/var.sh): functions named
# myos_default_<VARIABLE>, called only when the variable has no value and
# called again at each reference. That is a make `?=` on a recursive variable,
# and it is what lets a stack of the catalogue work without make installed.
#
# The _stack hooks of every directory between the stack path root and the stack
# itself are loaded, outermost first: make included both $(dir)/*.mk and
# $(dir)/*/*.mk, so a stack in a subdirectory saw its parent's settings.
# myos_stack_hooks DIR NAME load the hooks that apply to one stack
myos_stack_hooks() {
_hdir=$1; _hname=$2
# the stack path entry this directory belongs to
_root=
_IFS=$IFS; IFS=:
for _r in $(myos_path); do
IFS=$_IFS
case $_hdir in "$_r"|"$_r"/*) _root=$_r; break ;; esac
IFS=:
done
IFS=$_IFS
# every directory from the root down to the stack, outermost first
_chain=$_hdir
if [ -n "$_root" ]; then
_d=$_hdir
while [ "$_d" != "$_root" ] && [ "$_d" != "/" ] && [ -n "$_d" ]; do
_d=$(dirname "$_d")
_chain="$_d
$_chain"
done
fi
for _d in $_chain; do
myos_hook_load "$_d/_stack.env" dotenv
myos_hook_load "$_d/_stack.sh" shell
done
myos_hook_load "$_hdir/$_hname.env" dotenv
myos_hook_load "$_hdir/$_hname.env.$ENV" dotenv
myos_hook_load "$_hdir/$_hname.sh" shell
myos_hook_load "$_hdir/$_hname.$ENV.sh" shell
return 0
}
# myos_hook_load FILE KIND
myos_hook_load() {
[ -f "$1" ] || return 0
myos_debug "hook $1"
# a hook may need to read a file it ships next to itself
# shellcheck disable=SC2034 # read by the hooks sourced below
MYOS_STACK_DIR=$(dirname "$1")
case $2 in
dotenv) myos_dotenv_load "$1" ;;
shell) . "$1" ;;
esac
}
+122
View File
@@ -0,0 +1,122 @@
#shellcheck shell=sh
# shellcheck disable=SC3028 # HOSTNAME is a myos variable, set by bin/myos
# naming: compose project name, service name, networks, user identity.
#
# Ported from make/apps/def.docker.mk (COMPOSE_PROJECT_NAME, COMPOSE_SERVICE_NAME),
# make/def.docker.mk (HOST_*/USER_*, DOCKER_NETWORK_*) and make/def.mk (RESU).
# myos_scope REF -> host | user | cluster | app
# The first segment of a stack reference decides how the stack is named:
# host stacks are singletons of the machine (they bind privileged ports),
# user stacks are singletons of the user, cluster stacks are swarm namespaces.
myos_scope() {
[ -n "${MYOS_SCOPE:-}" ] && { printf '%s' "$MYOS_SCOPE"; return 0; }
case ${1%%/*} in
host) printf 'host' ;;
User|user) printf 'user' ;;
cluster) printf 'cluster' ;;
*) printf 'app' ;;
esac
}
# myos_resu MAIL -> user.domain identity of a mail address (make: RESU)
# Also sets MYOS_RESU_NIAMOD (reversed domain + reversed user) and
# MYOS_RESU_PATH (that identity as a path), used by the User stacks.
myos_resu() {
_mail=$(myos_lower "${1:-}" | tr '+_' '..')
case $_mail in
*@*) ;;
*) MYOS_RESU_NIAMOD=; MYOS_RESU_PATH=; printf '%s' "${USER:-}"; return 0 ;;
esac
_user=${_mail%@*}
_domain=${_mail##*@}
[ -n "$_domain" ] || { MYOS_RESU_NIAMOD=; MYOS_RESU_PATH=; printf '%s' "${USER:-}"; return 0; }
_niamod=$(myos_reverse "$(printf '%s' "$_domain" | tr '.' ' ')" | tr ' ' '.')
_resu=$(myos_reverse "$(printf '%s' "$_user" | tr '.' ' ')" | tr ' ' '.')
MYOS_RESU_NIAMOD="$_niamod.$_resu"
# consumed by the User stacks, not by this file
# shellcheck disable=SC2034
MYOS_RESU_PATH=$(printf '%s' "$MYOS_RESU_NIAMOD" | tr '.' '/')
printf '%s.%s' "$_user" "$_domain"
}
# myos_project_name SCOPE USER ENV APP [PATH]
# host -> HOST_COMPOSE_PROJECT_NAME, defaults to the hostname
# user -> USER_COMPOSE_PROJECT_NAME, defaults to the RESU identity
# cluster -> the stack name: one namespace per swarm, not per user
# app -> MYOS_PROJECT_FORMAT: user-env-app (default) or user-app-env (legacy)
myos_project_name() {
_scope=$1; _user=$2; _env=$3; _app=$4; _path=${5:-}
[ -n "${DOCKER_COMPOSE_PROJECT_NAME:-}" ] && { printf '%s' "$DOCKER_COMPOSE_PROJECT_NAME"; return 0; }
case $_scope in
host)
printf '%s' "${HOST_COMPOSE_PROJECT_NAME:-${HOSTNAME:-localhost}}"; return 0 ;;
user)
if [ -n "${USER_COMPOSE_PROJECT_NAME:-}" ]; then printf '%s' "$USER_COMPOSE_PROJECT_NAME"
else printf '%s' "$(myos_resu "${MAIL:-}" | tr '.' '-')"; fi
return 0 ;;
cluster)
printf '%s' "${MYOS_CLUSTER_PROJECT:-$(myos_name "$_app")}"; return 0 ;;
esac
_n=$(myos_name "$_app")
case ${MYOS_PROJECT_FORMAT:-user-env-app} in
user-app-env) _out="$_user-$_n-$_env" ;;
user-env-app) _out="$_user-$_env-$_n" ;;
*) myos_die "$MYOS_E_USAGE" "unknown MYOS_PROJECT_FORMAT: ${MYOS_PROJECT_FORMAT}" ;;
esac
# the path fragment loses its slashes too (make: $(subst /,,$(subst -,,$(APP_PATH))))
[ -n "$_path" ] && _out="$_out-$(myos_name "$_path" | tr -d /)"
myos_lower "$_out" | tr -d '.'
}
# myos_service_name PROJECT prefix of the SERVICE_<port>_NAME labels
myos_service_name() { printf '%s' "$1" | tr '_' '-'; }
# myos_network_default PROJECT
# The leading underscore keeps this network first in alphabetical order, so it
# is the first interface attached and service names never resolve across stacks.
# https://github.com/moby/libnetwork/issues/2093
myos_network_default() { printf '_%s' "$1"; }
myos_network_private() { printf '%s' "${DOCKER_NETWORK_PRIVATE:-${1}-${2}}"; }
myos_network_public() { printf '%s' "${DOCKER_NETWORK_PUBLIC:-${1}}"; }
# myos_app_domain SCOPE USER DOMAIN
# The domain a stack is served on. A host stack is never prefixed by the user:
# it belongs to the machine.
myos_app_domain() {
_scope=$1; _u=$2; _dom=$3
if [ "$_scope" != host ] && [ "${APP_HOST_MULTI_USER:-false}" = true ]; then
printf '%s.%s' "$_u" "$_dom"
else
printf '%s' "$_dom"
fi
}
# myos_app_host SCOPE USER ENV APP DOMAIN HOSTNAME
# A host stack is served on <hostname>.<domain>; anything else is prefixed by
# the environment unless the environment is the main one.
myos_app_host() {
_scope=$1; _u=$2; _env=$3; _app=$4; _dom=$5; _host=$6
_multi_env=${APP_HOST_MULTI_ENV:-}
if [ -z "$_multi_env" ]; then
case $_env in local|master|main) _multi_env=false ;; *) _multi_env=true ;; esac
fi
_prefix=
if [ "$_scope" = host ]; then _prefix="$_host."
elif [ "$_multi_env" = true ]; then _prefix="$_env."
fi
_name=
[ "${APP_HOST_MULTI_APP:-false}" = true ] && _name="$(myos_name "$_app")."
_out="$_prefix$_name$(myos_app_domain "$_scope" "$_u" "$_dom")"
# a host stack behind the load balancer also answers on the bare domain
[ "$_scope" = host ] && [ -n "${HOST_LB:-}" ] && _out="$_out $_dom"
printf '%s' "$_out"
}
# myos_app_uri HOST [PATH] the base uri the fabio tags are built on;
# it always ends with a slash
myos_app_uri() {
_out=
for _h in $1; do _out="${_out:+$_out }$_h/${2:-}"; done
printf '%s' "$_out"
}
+190
View File
@@ -0,0 +1,190 @@
#shellcheck shell=sh
# stack: where stacks live, how a reference resolves to compose files.
#
# A stack is a directory holding compose files. A reference is
# [<group>/]<name>[:<version>] resolved along MYOS_PATH
# ./ or /abs/path or rel/path the directory itself
# Ported from make/def.docker.mk (STACK_DIR/SHARE_DIR) and
# make/apps/def.docker.mk (compose-file, docker-stack, docker-stack-update).
# myos_path the stack search path, colon separated, existing directories only.
# Project first, then the shared catalogues: a project always wins over the
# catalogue installed system wide.
myos_path() {
[ -n "${MYOS_PATH:-}" ] && { printf '%s' "$MYOS_PATH"; return 0; }
_wd=${WORKDIR:-$PWD}
_name=${STACK_DIR_NAME:-stack}
_out=
# <prefix>/share comes from MYOS_ROOT, so an installation under any prefix
# finds the catalogue installed beside it
_prefix=
[ -n "${MYOS_ROOT:-}" ] && _prefix=$(dirname "$(dirname "$MYOS_ROOT")")/share
for _d in "$_wd" "$_wd/.." "${HOME:-/nonexistent}/.local/share" \
${_prefix:+"$_prefix"} /usr/local/share /usr/share; do
for _c in "$_d/$_name" "$_d/myos/$_name"; do
[ -d "$_c" ] || continue
_c=$(cd "$_c" && pwd -P)
case ":$_out:" in *":$_c:"*) continue ;; esac
_out="${_out:+$_out:}$_c"
done
done
printf '%s' "$_out"
}
# myos_stack_name REF the stack name: "host/fabio:1.6" -> "fabio"
# myos_stack_version REF the version, "latest" when the reference has none
# Both are pure, so a caller can use them inside a command substitution.
myos_stack_name() {
_r=${1%/}
case $_r in *:*) _r=${_r%:*} ;; esac
basename "$_r" .yml
}
myos_stack_version() {
_r=${1%/}
case $_r in *:*) printf '%s' "${_r##*:}" ;; *) printf 'latest' ;; esac
}
# myos_stack_dirs REF every directory of MYOS_PATH holding this stack, least
# specific first. A compose overlay wins over the ones before it, so a project
# that ships stack/postgres/postgres.local.yml refines the postgres stack of
# the catalogue instead of replacing it.
myos_stack_dirs() {
_ref=${1%/}
case $_ref in *:*) _ref=${_ref%:*} ;; esac
_name=$(myos_stack_name "$1")
case $_ref in
.|./*|/*|../*)
[ -d "$_ref" ] && (cd "$_ref" && pwd -P)
return 0 ;;
esac
_found=
_IFS=$IFS; IFS=:
for _d in $(myos_path); do
IFS=$_IFS
_hit=
if [ -d "$_d/$_ref" ]; then _hit=$_d/$_ref
elif [ -f "$_d/$_ref.yml" ] || [ -f "$_d/$_ref.yaml" ]; then _hit=$(dirname "$_d/$_ref")
elif [ -d "$_d/$_name" ]; then _hit=$_d/$_name
fi
# least specific last here, then reversed below
[ -n "$_hit" ] && _found="$_hit
$_found"
IFS=:
done
IFS=$_IFS
printf '%s' "$_found" | sed '/^$/d'
}
# myos_stack_resolve REF print the directory holding the stack,
# or fail with MYOS_E_NOSTACK
myos_stack_resolve() {
_ref=${1%/}
case $_ref in *:*) _ref=${_ref%:*} ;; esac
_name=$(myos_stack_name "$1")
# a path reference resolves to itself
case $_ref in
.|./*|/*|../*)
if [ -d "$_ref" ]; then printf '%s' "$(cd "$_ref" && pwd -P)"; return 0; fi
myos_error "no such directory: $_ref"; return "$MYOS_E_NOSTACK" ;;
esac
_IFS=$IFS; IFS=:
for _d in $(myos_path); do
IFS=$_IFS
if [ -d "$_d/$_ref" ]; then printf '%s' "$_d/$_ref"; return 0; fi
if [ -f "$_d/$_ref.yml" ] || [ -f "$_d/$_ref.yaml" ]; then
printf '%s' "$(dirname "$_d/$_ref")"; return 0
fi
if [ -d "$_d/$_name" ]; then printf '%s' "$_d/$_name"; return 0; fi
IFS=:
done
IFS=$_IFS
myos_error "stack not found: $1 (searched $(myos_path))"
return "$MYOS_E_NOSTACK"
}
# myos_compose_suffixes the overlay suffixes, from the COMPOSE_FILE_* variables
# that are not false (make: COMPOSE_FILE_SUFFIX). A value other than true also
# yields "<suffix>.<value>", which is how COMPOSE_FILE_WWW=nginx works.
myos_compose_suffixes() {
_out=
for _v in $(set | sed -n 's/^\(COMPOSE_FILE_[A-Z0-9_]*\)=.*/\1/p' | sort -u); do
case $_v in COMPOSE_FILE_SUFFIX) continue ;; esac
_val=$(myos_var "$_v")
case $_val in false|False|FALSE|'') continue ;; esac
_s=$(myos_lower "${_v#COMPOSE_FILE_}")
_out="${_out:+$_out }$_s"
case $_val in true|True|TRUE) ;; *) for _x in $_val; do _out="$_out $_s.$_x"; done ;; esac
done
printf '%s' "$_out"
}
# myos_compose_files DIR NAMES SUFFIXES [ENV]
# Print the compose files that exist, in the order the framework loads them.
myos_compose_files() {
_dir=$1; _names=$2; _suffixes=${3:-}; _env=${4:-${ENV:-local}}
for _e in yml yaml; do
for _n in $_names; do
for _f in \
"$_dir/$_n.$_e" "$_dir/$_n.$_env.$_e" \
"$_dir/$_env/$_n.$_e" "$_dir/$_env/$_n.$_env.$_e"; do
[ -f "$_f" ] && printf '%s\n' "$_f"
done
for _s in $_suffixes; do
for _f in "$_dir/$_n.$_s.$_e" "$_dir/$_n.$_s.$_env.$_e"; do
[ -f "$_f" ] && printf '%s\n' "$_f"
done
done
done
done
return 0
}
# myos_group_expand REF... expand group references recursively.
# A group is a variable whose name is the reference: `host=host/consul host/fabio`
# in the environment, in a .env, in <path>/<group>.env or in a legacy <group>.mk.
myos_group_expand() {
_depth=${MYOS_GROUP_DEPTH:-0}
[ "$_depth" -gt 16 ] && myos_die "$MYOS_E_USAGE" "stack group nested too deep: $*"
for _ref in "$@"; do
_val=$(myos_group_value "$_ref")
if [ -n "$_val" ]; then
# shellcheck disable=SC2086 # the group value is a list of references
MYOS_GROUP_DEPTH=$((_depth + 1)) myos_group_expand $_val
else
printf '%s\n' "$_ref"
fi
done
}
# myos_group_value REF the list a group expands to, empty when not a group.
# Groups are lowercase by convention (host, testing, coroot, default): without
# that rule any environment variable sharing a stack name would be expanded,
# which is how the make engine behaved. The character class is spelled out
# because a-z also matches uppercase under a dictionary collation (fr_FR).
myos_group_value() {
case $1 in .|/*|*/*|*:*) return 0 ;; esac
case $1 in *[![:lower:][:digit:]_-]*) return 0 ;; esac
_v=$(myos_var "$1")
[ -n "$_v" ] && { printf '%s' "$_v"; return 0; }
_IFS=$IFS; IFS=:
for _d in $(myos_path); do
IFS=$_IFS
for _f in "$_d/$1.env" "$_d/$1/$1.env" "$_d/$1/_stack.env"; do
[ -z "$_v" ] && [ -f "$_f" ] && _v=$(sed -n "s/^$1=//p" "$_f" | tail -1 | tr -d '"')
done
for _f in "$_d/$1.mk" "$_d/$1/$1.mk"; do
[ -z "$_v" ] && [ -f "$_f" ] && _v=$(myos_mk_group "$_f" "$1")
done
[ -n "$_v" ] && { printf '%s' "$_v"; return 0; }
IFS=:
done
IFS=$_IFS
return 0
}
# myos_mk_group FILE NAME read `name ?= a b c` out of a legacy .mk snippet
myos_mk_group() {
sed -n "s/^$2[[:space:]]*[?:]\{0,1\}=[[:space:]]*//p" "$1" 2>/dev/null | tail -1
}
+135
View File
@@ -0,0 +1,135 @@
#shellcheck shell=sh
# str: string helpers ported from make/utils.mk and make/def.mk.
# myos_lower STRING / myos_upper STRING
myos_lower() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]'; }
myos_upper() { printf '%s' "$1" | tr '[:lower:]-.' '[:upper:]__'; }
# myos_name STRING compose-project-safe name: lowercase, no . - _
# (make: $(subst _,,$(subst -,,$(subst .,,$(call LOWERCASE,$(1))))))
myos_name() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | tr -d '._-'; }
# myos_slugify STRING keep [a-z0-9_], everything else becomes _
myos_slugify() { printf '%s' "$1" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9_]/_/g'; }
# myos_reverse WORDS... reverse the order of space separated words
myos_reverse() {
set -f # a word may be a pattern such as *.example.org, do not glob it
_out=
for _w in $1; do _out="$_w${_out:+ }$_out"; done
set +f
printf '%s' "$_out"
}
# myos_verle A B true when version A <= B (make: verle)
myos_verle() {
[ -n "$1" ] || return 1
[ -n "$2" ] || return 1
[ "$1" = "$(printf '%s\n%s\n' "$1" "$2" | sort -V | head -n1)" ]
}
# myos_verlt A B true when version A < B
myos_verlt() {
[ "$1" = "$2" ] && return 1
myos_verle "$1" "$2"
}
# The make list functions the catalogue uses, on space separated words.
# myos_firstword LIST / myos_lastword LIST
myos_firstword() { set -f; for _w in $1; do set +f; printf '%s' "$_w"; return 0; done; set +f; }
myos_lastword() { _l=; for _w in $1; do _l=$_w; done; printf '%s' "$_l"; }
# myos_or A B... the first argument that is not empty
myos_or() { for _a in "$@"; do [ -n "$_a" ] && { printf '%s' "$_a"; return 0; }; done; }
# myos_patsubst PATTERN REPLACEMENT LIST
# The pattern holds one %, standing for any text; the replacement puts it back.
myos_patsubst() {
set -f # a word may be a pattern such as *.example.org, do not glob it
_pre=${1%%%*}; _suf=${1#*%}
_rpre=${2%%%*}; _rsuf=${2#*%}
_out=
for _w in $3; do
case $_w in
"$_pre"*"$_suf")
_stem=${_w#"$_pre"}; _stem=${_stem%"$_suf"}
_out="${_out:+$_out }$_rpre$_stem$_rsuf" ;;
*) _out="${_out:+$_out }$_w" ;;
esac
done
set +f
printf '%s' "$_out"
}
# myos_pattern MAKE_PATTERN the shell pattern matching a make pattern.
# In make only % is a wildcard, so a literal *, ? or [ has to be protected
# before % becomes *: "*.%" means "starts with a star and a dot", not
# "anything".
myos_pattern() {
printf '%s' "$1" | sed -e 's/[][*?]/\\&/g' -e 's/%/*/g'
}
# myos_filter PATTERNS LIST / myos_filter_out PATTERNS LIST
myos_filter() {
set -f # a word may itself be a pattern, do not glob it
_out=
for _w in $2; do
for _raw in $1; do
_p=$(myos_pattern "$_raw")
# shellcheck disable=SC2254 # the pattern is meant to match, not to glob
case $_w in $_p) _out="${_out:+$_out }$_w"; break ;; esac
done
done
set +f
printf '%s' "$_out"
}
myos_filter_out() {
set -f
_out=
for _w in $2; do
_keep=yes
for _raw in $1; do
_p=$(myos_pattern "$_raw")
# shellcheck disable=SC2254 # the pattern is meant to match, not to glob
case $_w in $_p) _keep=no; break ;; esac
done
[ "$_keep" = yes ] && _out="${_out:+$_out }$_w"
done
set +f
printf '%s' "$_out"
}
# myos_addprefix PREFIX LIST / myos_addsuffix SUFFIX LIST
myos_addprefix() { set -f; _out=; for _w in $2; do _out="${_out:+$_out }$1$_w"; done; set +f; printf '%s' "$_out"; }
myos_addsuffix() { set -f; _out=; for _w in $2; do _out="${_out:+$_out }$_w$1"; done; set +f; printf '%s' "$_out"; }
# myos_b64url read stdin, write url-safe base64 without padding
myos_b64url() { openssl enc -A -base64 | tr '+/' '-_' | tr -d '='; }
# myos_jwt HEADER PAYLOAD SECRET a HS256 JSON web token
# Ported from the JWT macro of make/def.mk, which supabase uses to derive its
# anon and service keys from one secret. The make macro split on the commas of
# the payload; this one does not.
myos_jwt() {
_h=${1:-'{"alg":"HS256","typ":"JWT"}'}
_p=$2
_s=$3
_hb=$(printf '%s' "$_h" | myos_b64url)
_pb=$(printf '%s' "$_p" | myos_b64url)
_sig=$(printf '%s' "$_hb.$_pb" | openssl dgst -sha256 -binary -hmac "$_s" | myos_b64url)
printf '%s.%s.%s' "$_hb" "$_pb" "$_sig"
}
# myos_patsublist PATTERN REPLACEMENT LIST
# patsubst over a list, joined by commas. The fabio tags are built this way:
# one route per uri, in a single label.
myos_patsublist() {
set -f # a word may be a pattern such as *.example.org, do not glob it
_out=
for _w in $3; do
_out="${_out:+$_out,}$(myos_patsubst "$1" "$2" "$_w")"
done
set +f
printf '%s' "$_out"
}
+95
View File
@@ -0,0 +1,95 @@
#shellcheck shell=sh
# tags: fabio route tags derived from stack variables.
#
# Ported from make/apps/def.mk (uri, url, urlprefix, urlprefixs, tagprefix,
# envprefix, servicenvs). Registrator publishes the SERVICE_<port>_TAGS label
# to consul, fabio routes on the urlprefix- tags it finds there.
#
# Every lookup goes through myos_var (lib/var.sh), so a stack setting may be a
# plain value or a lazy default, and the two behave the same here.
# myos_uri SERVICES PORT [BASE_URI]
# <service>.<base uri> for each service and each base uri, unless
# <SERVICE>_SERVICE[_<port>]_NAME overrides the prefix. The first argument is a
# list: one stack may publish several services on one port.
myos_uri() {
set -f # a uri may be a pattern such as *.ipns.example.org
_svcs=$1; _port=${2:-}; _base=${3:-${APP_URI:-}}
_out=
for _svc in $_svcs; do
_u=$(myos_upper "$_svc")
_name=$(myos_var "${_u}_SERVICE_${_port}_NAME")
[ -n "$_name" ] || _name=$(myos_var "${_u}_SERVICE_NAME")
[ -n "$_name" ] || _name=$_svc
for _b in $_base; do _out="${_out:+$_out }${_name}.${_b}"; done
done
set +f
printf '%s' "$_out"
}
# myos_url SERVICE PORT [BASE_URI]
myos_url() {
set -f
_out=
for _u in $(myos_uri "$@"); do _out="${_out:+$_out }${APP_SCHEME:-http}://$_u"; done
set +f
printf '%s' "$_out"
}
# myos_urlprefix [PATH] [OPTS] [URIS]
# one comma separated "urlprefix-<uri><path>* [opts]" per uri
myos_urlprefix() {
set -f
_path=${1:-}; _opts=${2:-}; _uris=${3:-${APP_URI:-}}
_out=
for _u in $_uris; do
_tag="urlprefix-${_u}${_path}${MYOS_URL_SUFFIX:-*}${_opts:+ $_opts}"
_out="${_out:+$_out,}$_tag"
done
set +f
printf '%s' "$_out"
}
# myos_envprefix STACK PORT KEYS...
# "key=value" for each <STACK>_SERVICE_<port>_<KEY> that is set
myos_envprefix() {
_stack=$1; _port=$2; shift 2
_out=
for _k in "$@"; do
_v=$(myos_var "$(myos_upper "${_stack}_SERVICE_${_port}_${_k}")")
[ -n "$_v" ] && _out="${_out:+$_out }${_k}=${_v}"
done
printf '%s' "$_out"
}
# myos_tagprefix STACK PORT [URI_KEYS...]
# the fabio tag of a service, assembled from its PATH, OPTS and URIS variables
myos_tagprefix() {
_stack=$1; _port=$2; shift 2
_u=$(myos_upper "$_stack")
_path=$(myos_var "${_u}_SERVICE_${_port}_PATH"); [ -n "$_path" ] || _path=$(myos_var "${_u}_SERVICE_PATH")
_opts=$(myos_var "${_u}_SERVICE_${_port}_OPTS"); [ -n "$_opts" ] || _opts=$(myos_var "${_u}_SERVICE_OPTS")
[ -n "$_opts" ] || _opts=$(myos_envprefix "$_stack" "$_port" allow auth deny prepend proto register strip)
_uris=
for _k in "$@"; do
_v=$(myos_var "${_u}_SERVICE_${_port}_$(myos_upper "$_k")")
[ -n "$_v" ] && _uris="${_uris:+$_uris }$_v"
done
[ -n "$_uris" ] || _uris=$(myos_var "${_u}_SERVICE_${_port}_URIS")
[ -n "$_uris" ] || _uris=$(myos_uri "$_stack" "$_port")
myos_urlprefix "$_path" "$_opts" "$_uris"
}
# myos_servicenvs STACK GROUP KEY
# Collect <STACK>_SERVICE_<env>_<KEY> for every env listed in
# <STACK>_SERVICE_<GROUP>_ENVS. Used to build a list of listeners out of one
# variable per protocol.
myos_servicenvs() {
_s=$(myos_upper "$1"); _g=$(myos_upper "$2"); _k=$(myos_upper "$3")
_out=
for _e in $(myos_var "${_s}_SERVICE_${_g}_ENVS"); do
_v=$(myos_var "${_s}_SERVICE_$(myos_upper "$_e")_${_k}")
[ -n "$_v" ] && _out="${_out:+$_out }$_v"
done
printf '%s' "$_out"
}
+56
View File
@@ -0,0 +1,56 @@
#shellcheck shell=sh
# var: variable resolution, with the lazy defaults the make engine had.
#
# make gives every `VAR ?= $(call ...)` two properties at once: an explicit
# value wins, and the default is re-evaluated at each reference, so it sees
# whatever a .env loaded later has changed.
#
# The shell gets both by keeping defaults in functions: `myos_var NAME` reads
# the variable when it has a value, and otherwise calls the function
# `myos_default_NAME`. The prefix matters: a bare function named after the
# variable would collide with commands on PATH, and a stack setting called
# `host` or `test` would then run a program instead of returning a value.
#
# myos_default_HOST_FABIO_SERVICE_9998_TAGS() { myos_tagprefix HOST_FABIO 9998; }
#
# is the exact equivalent of
#
# HOST_FABIO_SERVICE_9998_TAGS ?= $(call tagprefix,HOST_FABIO,9998)
MYOS_VAR_MAX_DEPTH=${MYOS_VAR_MAX_DEPTH:-32}
# myos_var NAME the value of NAME: the variable if it has one, else the lazy
# default, else empty.
myos_var() {
[ -n "${1:-}" ] || return 0
eval "_myos_set=\${$1+yes}"
if [ "${_myos_set:-}" = yes ]; then
eval "printf '%s' \"\$$1\""
return 0
fi
myos_var_is_lazy "$1" || return 0
# a default written in terms of itself would loop for ever
_myos_depth=$((${MYOS_VAR_DEPTH:-0} + 1))
if [ "$_myos_depth" -gt "$MYOS_VAR_MAX_DEPTH" ]; then
myos_error "variable $1 is defined in terms of itself"
return 1
fi
MYOS_VAR_DEPTH=$_myos_depth "myos_default_$1"
}
# myos_default NAME BODY declare a lazy default from a string, for callers
# that build the variable name at run time
myos_default() {
eval "myos_default_$1() { $2; }"
}
# myos_var_is_lazy NAME true when NAME has no value but has a lazy default
myos_var_is_lazy() {
eval "_myos_set=\${$1+yes}"
[ "${_myos_set:-}" = yes ] && return 1
# a shell function, never a command on PATH: command -v prints the name back
# for a function and a path for a program
_myos_fn=$(command -v "myos_default_$1" 2>/dev/null) || return 1
[ "$_myos_fn" = "myos_default_$1" ]
}
+3 -7
View File
@@ -43,7 +43,7 @@ DOCKER_COMPOSE_RUN_WORKDIR ?= $(if $(DOCKER_COMPOSE_WORKDIR),-w $(DOCKER_CO
DOCKER_COMPOSE_SERVICE_NAME ?= $(subst _,-,$(DOCKER_COMPOSE_PROJECT_NAME)) DOCKER_COMPOSE_SERVICE_NAME ?= $(subst _,-,$(DOCKER_COMPOSE_PROJECT_NAME))
DOCKER_COMPOSE_UP_OPTIONS ?= -d DOCKER_COMPOSE_UP_OPTIONS ?= -d
DOCKER_IMAGE_TAG ?= $(if $(filter true,$(DEPLOY)),$(if $(filter $(ENV),$(ENV_DEPLOY)),$(VERSION)),$(if $(DRONE_BUILD_NUMBER),$(DRONE_BUILD_NUMBER),latest)) DOCKER_IMAGE_TAG ?= $(if $(filter true,$(DEPLOY)),$(if $(filter $(ENV),$(ENV_DEPLOY)),$(VERSION)),$(if $(DRONE_BUILD_NUMBER),$(DRONE_BUILD_NUMBER),latest))
DOCKER_IMAGES ?= $(patsubst %/,%,$(patsubst docker/%,%,$(dir $(wildcard docker/*/Dockerfile)))) DOCKER_IMAGES ?= $(foreach dir,$(DOCKER_DIR),$(patsubst $(dir)/%/Dockerfile,%,$(wildcard $(dir)/*/Dockerfile)))
DOCKER_PLUGIN ?= rexray/s3fs:latest DOCKER_PLUGIN ?= rexray/s3fs:latest
DOCKER_PLUGIN_ARGS ?= $(foreach var,$(DOCKER_PLUGIN_VARS),$(if $(DOCKER_PLUGIN_$(var)),$(var)='$(DOCKER_PLUGIN_$(var))')) DOCKER_PLUGIN_ARGS ?= $(foreach var,$(DOCKER_PLUGIN_VARS),$(if $(DOCKER_PLUGIN_$(var)),$(var)='$(DOCKER_PLUGIN_$(var))'))
DOCKER_PLUGIN_OPTIONS ?= --grant-all-permissions DOCKER_PLUGIN_OPTIONS ?= --grant-all-permissions
@@ -109,11 +109,7 @@ define docker-compose
$(if $(COMPOSE_FILE), $(if $(COMPOSE_FILE),
$(if $(DOCKER_COMPOSE), $(if $(DOCKER_COMPOSE),
$(call env-exec,$(RUN) $(DOCKER_COMPOSE) $(DOCKER_COMPOSE_ARGS) $(patsubst %,-f %,$(COMPOSE_FILE)) -p $(COMPOSE_PROJECT_NAME) $(1)) $(call env-exec,$(RUN) $(DOCKER_COMPOSE) $(DOCKER_COMPOSE_ARGS) $(patsubst %,-f %,$(COMPOSE_FILE)) -p $(COMPOSE_PROJECT_NAME) $(1))
, $(if $(DOCKER_RUN), , $(call ERROR,docker compose >= $(COMPOSE_VERSION) not found: install the docker compose plugin or docker-compose)
$(call docker-build,$(call docker-path,compose),docker/compose,$(COMPOSE_VERSION))
$(call docker-run,docker/compose:$(COMPOSE_VERSION) $(DOCKER_COMPOSE_ARGS),$(patsubst %,-f %,$(COMPOSE_FILE)) -p $(COMPOSE_PROJECT_NAME) $(1))
, $(call env-exec,$(RUN) docker-compose $(DOCKER_COMPOSE_ARGS) $(patsubst %,-f %,$(COMPOSE_FILE)) -p $(COMPOSE_PROJECT_NAME) $(1))
)
) )
) )
endef endef
@@ -156,7 +152,7 @@ define docker-stack-update
$(eval stack_update := $(patsubst %.yml,%,$(notdir $(1)))) $(eval stack_update := $(patsubst %.yml,%,$(notdir $(1))))
$(eval stack_name := $(firstword $(subst :, ,$(stack_update)))) $(eval stack_name := $(firstword $(subst :, ,$(stack_update))))
$(eval stack_version := $(or $(2),$(if $(findstring :,$(stack_update)),$(lastword $(subst :, ,$(stack_update))),latest))) $(eval stack_version := $(or $(2),$(if $(findstring :,$(stack_update)),$(lastword $(subst :, ,$(stack_update))),latest)))
$(eval stack_path := $(patsubst %/,%,$(or $(3),$(realpath $(foreach stack_dir,$(STACK_DIR),$(if $(findstring /,$(1)),$(if $(wildcard $(stack_dir)/$(1) $(stack_dir)/$(1).yml),$(stack_dir)/$(if $(findstring .yml,$(1)),$(dir $(1)),$(if $(wildcard $(stack_dir)/$(1).yml),$(dir $(1)),$(1))),$(if $(wildcard $(stack_dir)/$(stackz)/$(1) $(stack_dir)/$(stackz)/$(1).yml),$(stack_dir)/$(stackz)/$(if $(findstring .yml,$(1)),$(dir $(1)),$(if $(wildcard $(stack_dir)/$(stackz)/$(1).yml),$(dir $(1)),$(1))),$(dir $(1)))))),$(foreach stack_dir,$(STACK_DIR),$(firstword $(wildcard $(stack_dir)/$(stackz)/$(stack_name) $(stack_dir)/$(stackz) $(stack_dir)/$(stack_name)))))))) $(eval stack_path := $(patsubst %/,%,$(or $(3),$(realpath $(foreach stack_dir,$(STACK_DIR),$(if $(findstring /,$(1)),$(if $(wildcard $(stack_dir)/$(1) $(stack_dir)/$(1).yml),$(stack_dir)/$(if $(findstring .yml,$(1)),$(dir $(1)),$(if $(wildcard $(stack_dir)/$(1).yml),$(dir $(1)),$(1))),$(if $(wildcard $(stack_dir)/$(stackz)/$(1) $(stack_dir)/$(stackz)/$(1).yml),$(stack_dir)/$(stackz)/$(if $(findstring .yml,$(1)),$(dir $(1)),$(if $(wildcard $(stack_dir)/$(stackz)/$(1).yml),$(dir $(1)),$(1))),$(dir $(1))))))),$(realpath $(foreach stack_dir,$(STACK_DIR),$(firstword $(wildcard $(stack_dir)/$(stackz)/$(stack_name) $(stack_dir)/$(stackz) $(stack_dir)/$(stack_name))))))))
$(call debug,stack_path) $(call debug,stack_path)
$(call compose-file,$(stack_path),docker-compose $(stack_name),$(COMPOSE_FILE_SUFFIX) $(stack_version)) $(call compose-file,$(stack_path),docker-compose $(stack_name),$(COMPOSE_FILE_SUFFIX) $(stack_version))
$(if $(wildcard $(stack_path)/.env.dist),$(call .env,,$(stack_path)/.env.dist,$(wildcard $(CONFIG)/$(ENV)/$(APP)/.env $(stack_path)/.env.$(ENV) .env))) $(if $(wildcard $(stack_path)/.env.dist),$(call .env,,$(stack_path)/.env.dist,$(wildcard $(CONFIG)/$(ENV)/$(APP)/.env $(stack_path)/.env.$(ENV) .env)))
+3 -1
View File
@@ -30,7 +30,9 @@ NFS_HOST ?= host.docker.internal
SERVICES ?= $(DOCKER_SERVICES) SERVICES ?= $(DOCKER_SERVICES)
envprefix = $(foreach env,$(3),$(if $($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env))),$(env)=$($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env))))) envprefix = $(foreach env,$(3),$(if $($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env))),$(env)=$($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env)))))
patsublist = $(patsubst $(1),$(2),$(firstword $(3)))$(foreach pattern,$(wordlist 2,255,$(3)),$(comma)$(patsubst $(1),$(2),$(pattern))) ## the replacement ends with the options, so an empty option list left a space
## before the comma joining two routes: "urlprefix-a/* ,urlprefix-b/*"
patsublist = $(subst $(space)$(comma),$(comma),$(patsubst $(1),$(2),$(firstword $(3)))$(foreach pattern,$(wordlist 2,255,$(3)),$(comma)$(patsubst $(1),$(2),$(pattern))))
servicenvs = $(foreach env,$(call UPPERCASE,$($(1)_SERVICE_$(2)_ENVS)),$(if $(3),$($(1)_SERVICE_$(env)_$(3)),$($(1)_SERVICE_$(2)_$(env)))) servicenvs = $(foreach env,$(call UPPERCASE,$($(1)_SERVICE_$(2)_ENVS)),$(if $(3),$($(1)_SERVICE_$(env)_$(3)),$($(1)_SERVICE_$(2)_$(env))))
tagprefix = $(call urlprefix,$(or $($(call UPPERCASE,$(1)_SERVICE_$(2)_PATH)),$($(call UPPERCASE,$(1)_SERVICE_PATH))),$(or $($(call UPPERCASE,$(1)_SERVICE_$(2)_OPTS)),$($(call UPPERCASE,$(1)_SERVICE_OPTS)),$(call envprefix,$(1),$(2),allow auth deny preprend proto register strip)),$(or $(foreach env,$(3),$($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env)))),$($(call UPPERCASE,$(1)_SERVICE_$(2)_URIS)),$(call uri,$(1),$(2)))) tagprefix = $(call urlprefix,$(or $($(call UPPERCASE,$(1)_SERVICE_$(2)_PATH)),$($(call UPPERCASE,$(1)_SERVICE_PATH))),$(or $($(call UPPERCASE,$(1)_SERVICE_$(2)_OPTS)),$($(call UPPERCASE,$(1)_SERVICE_OPTS)),$(call envprefix,$(1),$(2),allow auth deny preprend proto register strip)),$(or $(foreach env,$(3),$($(call UPPERCASE,$(1)_SERVICE_$(2)_$(env)))),$($(call UPPERCASE,$(1)_SERVICE_$(2)_URIS)),$(call uri,$(1),$(2))))
uri = $(foreach svc,$(1),$(patsubst %,$(addsuffix .,$(or $($(call UPPERCASE,$(svc)_SERVICE_$(2)_NAME)),$($(call UPPERCASE,$(svc)_SERVICE_NAME)),$(svc)))%,$(or $(3),$(APP_URI)))) uri = $(foreach svc,$(1),$(patsubst %,$(addsuffix .,$(or $($(call UPPERCASE,$(svc)_SERVICE_$(2)_NAME)),$($(call UPPERCASE,$(svc)_SERVICE_NAME)),$(svc)))%,$(or $(3),$(APP_URI))))
+6 -12
View File
@@ -1,9 +1,9 @@
## ##
# DOCKER # DOCKER
# target docker-build: Fire docker-image-myos, Call docker-build-% target for each DOCKER_IMAGES # target docker-build: Call docker-build-% target for each DOCKER_IMAGES
.PHONY: docker-build .PHONY: docker-build
docker-build: docker-image-myos docker-build:
$(foreach image,$(or $(SERVICE),$(DOCKER_IMAGES)),$(call make,docker-build-$(image))) $(foreach image,$(or $(SERVICE),$(DOCKER_IMAGES)),$(call make,docker-build-$(image)))
# target docker-build-%: Call docker-build for each Dockerfile in docker/% folder # target docker-build-%: Call docker-build for each Dockerfile in docker/% folder
@@ -27,10 +27,10 @@ docker-commit: stack
docker-commit-%: stack docker-commit-%: stack
$(foreach service,$(or $(SERVICE),$(SERVICES)),$(call docker-commit,$(service),,,$*)) $(foreach service,$(or $(SERVICE),$(SERVICES)),$(call docker-commit,$(service),,,$*))
# target docker-compose-build: Fire docker-image-myos, Call docker-compose build SERVICE # target docker-compose-build: Call docker-compose build SERVICE
.PHONY: docker-compose-build .PHONY: docker-compose-build
docker-compose-build: DOCKER_RUN_OPTIONS += -it docker-compose-build: DOCKER_RUN_OPTIONS += -it
docker-compose-build: docker-image-myos stack docker-compose-build: stack
$(call docker-compose,build $(DOCKER_BUILD_ARGS) $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE))) $(call docker-compose,build $(DOCKER_BUILD_ARGS) $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE)))
# target docker-compose-config: Call docker-compose config # target docker-compose-config: Call docker-compose config
@@ -110,18 +110,12 @@ docker-compose-start: stack
docker-compose-stop: stack docker-compose-stop: stack
$(call docker-compose,stop $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE))) $(call docker-compose,stop $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE)))
# target docker-compose-up: Fire docker-image-myos, Call docker-compose up SERVICE # target docker-compose-up: Call docker-compose up SERVICE
.PHONY: docker-compose-up .PHONY: docker-compose-up
docker-compose-up: DOCKER_RUN_OPTIONS += -it docker-compose-up: DOCKER_RUN_OPTIONS += -it
docker-compose-up: docker-image-myos bootstrap-stack stack docker-compose-up: bootstrap-stack stack
$(call docker-compose,up $(DOCKER_COMPOSE_UP_OPTIONS) $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE))) $(call docker-compose,up $(DOCKER_COMPOSE_UP_OPTIONS) $(if $(filter $(SERVICE),$(SERVICES)),$(SERVICE)))
# target docker-image-myos: Call myos-docker-build-% target for each MYOS_DOCKER_IMAGES
.PHONY: docker-image-myos
docker-image-myos: MAKE_VARS += DOCKER_REPOSITORY STACK
docker-image-myos:
$(foreach image,$(subst $(quote),,$(MYOS_DOCKER_IMAGES)),$(call make,docker-build-$(image),$(MYOS)))
# target docker-image-rm: Remove docker images matching DOCKER_REPOSITORY # target docker-image-rm: Remove docker images matching DOCKER_REPOSITORY
.PHONY: docker-image-rm .PHONY: docker-image-rm
docker-image-rm: docker-image-rm:
+3 -1
View File
@@ -13,7 +13,9 @@ endif
setup-docker-group: setup-docker-group:
ifneq ($(DOCKER),) ifneq ($(DOCKER),)
ifeq ($(or $(filter $(USER),$(subst $(comma), ,$(shell awk -F':' '$$1 == "docker" {print $$4}' /etc/group))),$(filter 0,$(UID))),) ifeq ($(or $(filter $(USER),$(subst $(comma), ,$(shell awk -F':' '$$1 == "docker" {print $$4}' /etc/group))),$(filter 0,$(UID))),)
$(call ansible-user-add-groups,$(USER),docker) $(RUN) $(SUDO) usermod -aG docker $(USER) 2>/dev/null \
|| $(RUN) $(SUDO) addgroup $(USER) docker 2>/dev/null \
|| $(call ERROR,unable to add user,$(USER),to group,docker)
$(call WARNING,user,$(USER),added in group,docker) $(call WARNING,user,$(USER),added in group,docker)
endif endif
ifeq ($(filter 0 $(DOCKER_GID),$(GIDS)),) ifeq ($(filter 0 $(DOCKER_GID),$(GIDS)),)
+23 -12
View File
@@ -1,10 +1,25 @@
## ##
# SSH # SSH
#
# The remote hosts used to come from AWS, through
# ssh-get-PrivateIpAddress-% -> aws-ec2-get-instances-PrivateIpAddress-%.
# make/apps/aws was removed and AWS_INSTANCE_IP is defined nowhere, so these
# targets looped over an empty list and exited 0 without doing anything.
# They now take SSH_HOSTS, and say so when it is empty rather than pretending
# to have connected.
# variable SSH_HOSTS: hosts the ssh targets act on, space separated
SSH_HOSTS ?= $(AWS_INSTANCE_IP)
# target ssh-hosts-check: Fail when no remote host is known
.PHONY: ssh-hosts-check
ssh-hosts-check:
$(if $(SSH_HOSTS),,$(call ERROR,no remote host: set SSH_HOSTS=host1 host2))
# target ssh: Call ssh-connect ARGS or SHELL # target ssh: Call ssh-connect ARGS or SHELL
.PHONY: ssh .PHONY: ssh
ssh: # ssh-get-PrivateIpAddress-$(SERVER_NAME) ## Connect to first remote host ssh: ssh-hosts-check ## Connect to first remote host
$(call ssh-connect,$(AWS_INSTANCE_IP),$(if $(ARGS),$(ARGS),$(SHELL))) $(call ssh-connect,$(SSH_HOSTS),$(if $(ARGS),$(ARGS),$(SHELL)))
# target ssh-add: Fire ssh-key and ssh-add file SSH_PRIVATE_KEYS in folder SSH_DIR # target ssh-add: Fire ssh-key and ssh-add file SSH_PRIVATE_KEYS in folder SSH_DIR
.PHONY: ssh-add .PHONY: ssh-add
@@ -15,8 +30,8 @@ ssh-add: ssh-key
# target ssh-connect: Call ssh-connect make connect SERVICE # target ssh-connect: Call ssh-connect make connect SERVICE
.PHONY: ssh-connect .PHONY: ssh-connect
ssh-connect: # ssh-get-PrivateIpAddress-$(SERVER_NAME) ssh-connect: ssh-hosts-check
$(call ssh-connect,$(AWS_INSTANCE_IP),make connect COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME) ENV=$(ENV) $(if $(SERVICE),SERVICE=$(SERVICE))) $(call ssh-connect,$(SSH_HOSTS),make connect COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME) ENV=$(ENV) $(if $(SERVICE),SERVICE=$(SERVICE)))
# target ssh-del: ssh-add -d file SSH_PRIVATE_KEYS in folder SSH_DIR # target ssh-del: ssh-add -d file SSH_PRIVATE_KEYS in folder SSH_DIR
.PHONY: ssh-del .PHONY: ssh-del
@@ -26,12 +41,8 @@ ssh-del:
# target ssh-exec: Call ssh-exec make exec SERVICE ARGS # target ssh-exec: Call ssh-exec make exec SERVICE ARGS
.PHONY: ssh-exec .PHONY: ssh-exec
ssh-exec: # ssh-get-PrivateIpAddress-$(SERVER_NAME) ssh-exec: ssh-hosts-check
$(call ssh-exec,$(AWS_INSTANCE_IP),make exec COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME) ENV=$(ENV) $(if $(SERVICE),SERVICE=$(SERVICE)) $(if $(ARGS),ARGS='\''"$(ARGS)"'\'')) $(call ssh-exec,$(SSH_HOSTS),make exec COMPOSE_PROJECT_NAME=$(COMPOSE_PROJECT_NAME) ENV=$(ENV) $(if $(SERVICE),SERVICE=$(SERVICE)) $(if $(ARGS),ARGS='\''"$(ARGS)"'\''))
# target ssh-get-PrivateIpAddress-%: Fire aws-ec2-get-instances-PrivateIpAddress-%
.PHONY: ssh-get-PrivateIpAddress-%
ssh-get-PrivateIpAddress-%: aws-ec2-get-instances-PrivateIpAddress-%;
# target ssh-key: Add ssh private key SSH_KEY to SSH_DIR # target ssh-key: Add ssh private key SSH_KEY to SSH_DIR
.PHONY: ssh-key .PHONY: ssh-key
@@ -43,5 +54,5 @@ endif
# target ssh-run: Call ssh-run make run SERVICE ARGS # target ssh-run: Call ssh-run make run SERVICE ARGS
.PHONY: ssh-run .PHONY: ssh-run
ssh-run: # ssh-get-PrivateIpAddress-$(SERVER_NAME) ssh-run: ssh-hosts-check
$(call ssh-exec,$(AWS_INSTANCE_IP),make run $(if $(SERVICE),SERVICE=$(SERVICE)) $(if $(ARGS),ARGS='\''"$(ARGS)"'\'')) $(call ssh-exec,$(SSH_HOSTS),make run $(if $(SERVICE),SERVICE=$(SERVICE)) $(if $(ARGS),ARGS='\''"$(ARGS)"'\''))
+1 -1
View File
@@ -37,7 +37,7 @@ HOST_DOCKER_VOLUME ?= $(HOST_COMPOSE_PROJECT_NAME)
HOST_GID ?= $(HOST_UID) HOST_GID ?= $(HOST_UID)
HOST_UID ?= 123 HOST_UID ?= 123
HOST_STACK ?= $(filter host,$(firstword $(subst /, ,$(STACK)))) HOST_STACK ?= $(filter host,$(firstword $(subst /, ,$(STACK))))
MYOS_STACK ?= $(wildcard $(foreach stack_dir,$(STACK_DIR),$(stack_dir)/myos)) MYOS_STACK ?= $(if $(filter . myos,$(MYOS)),$(realpath $(MYOS)/share/compose))
MYOS_STACK_FILE ?= networks volumes MYOS_STACK_FILE ?= networks volumes
RESU_DOCKER_REPOSITORY ?= $(subst -,/,$(USER_COMPOSE_PROJECT_NAME)) RESU_DOCKER_REPOSITORY ?= $(subst -,/,$(USER_COMPOSE_PROJECT_NAME))
SHARE_DIR ?= . .. ~/.local/share /usr/local/share /usr/share SHARE_DIR ?= . .. ~/.local/share /usr/local/share /usr/share
+8 -2
View File
@@ -141,7 +141,9 @@ MACHINE ?= $(shell uname -m 2>/dev/null)
ifeq ($(SYSTEM),Darwin) ifeq ($(SYSTEM),Darwin)
SED_SUFFIX := '' SED_SUFFIX := ''
STAT_FORMAT_ARG := -f STAT_FORMAT_ARG := -f
STAT_FORMAT_FILE := '%a %N' # %m is the modification time; %a is the access time, which is what this used
# to ask for, so newer/older did not mean the same thing as on linux
STAT_FORMAT_FILE := '%m %N'
else else
STAT_FORMAT_ARG := -c STAT_FORMAT_ARG := -c
STAT_FORMAT_FILE := '%Y %n' STAT_FORMAT_FILE := '%Y %n'
@@ -199,6 +201,9 @@ rs256 = $(shell echo -n '$(1)' |openssl dgst -sha256 -binary -sign '$(2)')
JWT_HEADER = {"alg":"HS256","typ":"JWT"} JWT_HEADER = {"alg":"HS256","typ":"JWT"}
# macro JWT: Print Json Web Token for header $1 payload $2 and key $3 # macro JWT: Print Json Web Token for header $1 payload $2 and key $3
## a payload is JSON and holds commas, which make read as argument separators,
## so the token came out with an empty payload. Pass the payload in a variable
## and name it here rather than inlining it.
JWT := $(strip \ JWT := $(strip \
$(eval header := $(or $(1),$(JWT_HEADER))) \ $(eval header := $(or $(1),$(JWT_HEADER))) \
$(eval payload := $(or $(2),$(JWT_PAYLOAD))) \ $(eval payload := $(or $(2),$(JWT_PAYLOAD))) \
@@ -284,7 +289,8 @@ sed = $(RUN) sed -i $(SED_SUFFIX) '$(1)' $(2)
verle = [ -n "$(1)" ] && [ "$(1)" = "$(shell echo -e "$(1)\n$(2)" |sort -V |head -n1)" ] verle = [ -n "$(1)" ] && [ "$(1)" = "$(shell echo -e "$(1)\n$(2)" |sort -V |head -n1)" ]
# macro verlt: Return true when version 1 < 2 # macro verlt: Return true when version 1 < 2
verlt = [ "$(1)" = "$(2)" ] && return 1 || $(call verlte,$(1),$(2)) ## it was calling verlte, which does not exist, and returning from no function
verlt = [ "$(1)" != "$(2)" ] && $(call verle,$(1),$(2))
# function conf: Extract variable=value line from configuration files # function conf: Extract variable=value line from configuration files
## it prints the line with variable 3 definition from block 2 in file 1 ## it prints the line with variable 3 definition from block 2 in file 1
+1 -1
View File
@@ -19,7 +19,7 @@ include $(filter-out $(wildcard $(MAKE_FILE) $(MAKE_FIRST) $(MAKE_LATEST)),$(wil
## it includes $(MAKE_DIR)/$(MAKE_SUBDIRS)/*.mk ## it includes $(MAKE_DIR)/$(MAKE_SUBDIRS)/*.mk
include $(foreach subdir,$(MAKE_SUBDIRS),$(filter-out $(wildcard $(MAKE_DIR)/$(subdir)/def.mk $(MAKE_DIR)/$(subdir)/def.*.mk),$(wildcard $(MAKE_DIR)/$(subdir)/*.mk))) include $(foreach subdir,$(MAKE_SUBDIRS),$(filter-out $(wildcard $(MAKE_DIR)/$(subdir)/def.mk $(MAKE_DIR)/$(subdir)/def.*.mk),$(wildcard $(MAKE_DIR)/$(subdir)/*.mk)))
## if not in $(MYOS) nor $(MONOREPO), it includes def.mk def.*.mk */def.mk */def.*.mk *.mk */*.mk ## if not in $(MYOS) nor $(MONOREPO), it includes def.mk def.*.mk */def.mk */def.*.mk *.mk */*.mk
include $(if $(filter-out . myos,$(MYOS)),$(wildcard def.mk def.*.mk */def.mk */def.*.mk) $(filter-out $(wildcard def.mk def.*.mk */def.mk */def.*.mk stack/*.mk),$(wildcard *.mk */*.mk))) include $(if $(filter-out . myos,$(MYOS)),$(wildcard def.mk def.*.mk */def.mk */def.*.mk) $(filter-out $(wildcard def.mk def.*.mk */def.mk */def.*.mk $(STACK_DIR_NAME)/*.mk),$(wildcard *.mk */*.mk)))
## it includes $(STACK_DIR)/*.mk $(STACK_DIR)/*/*.mk ## it includes $(STACK_DIR)/*.mk $(STACK_DIR)/*/*.mk
include $(foreach stack_dir,$(STACK_DIR),$(wildcard $(stack_dir)/*.mk $(stack_dir)/*/*.mk)) include $(foreach stack_dir,$(STACK_DIR),$(wildcard $(stack_dir)/*.mk $(stack_dir)/*/*.mk))
## it includes $(MAKE_LATEST) ## it includes $(MAKE_LATEST)
+13 -2
View File
@@ -3,7 +3,14 @@
set -eu set -eu
# define MYOS path # define MYOS path
MYOS="$(dirname "$(readlink "$0" || echo "$0")")" ## readlink without -f only followed one level and only an absolute link, so a
## relative or chained symlink pointed the framework at the wrong directory
MYOS="$0"
while [ -L "$MYOS" ]; do
_link="$(readlink "$MYOS")"
case "$_link" in /*) MYOS="$_link" ;; *) MYOS="$(dirname "$MYOS")/$_link" ;; esac
done
MYOS="$(cd "$(dirname "$MYOS")" && pwd -P)"
# load system config: /etc/conf.d/myos (openrc convention) first, then the # load system config: /etc/conf.d/myos (openrc convention) first, then the
# debian-style /etc/default/myos as fallback # debian-style /etc/default/myos as fallback
@@ -17,4 +24,8 @@ MYOS_CONF=/etc/conf.d/myos
# call myos Makefile: a WORKDIR from the config or environment wins over PWD, # call myos Makefile: a WORKDIR from the config or environment wins over PWD,
# so a machine can pin its deployment dir and run myos from anywhere # so a machine can pin its deployment dir and run myos from anywhere
IFS=$'\n'; exec env $(cat "$MYOS_CONF" 2>/dev/null) MYOS=. WORKDIR="${WORKDIR:-${PWD}}" make -esC "${MYOS:-.}" "$@" ## a comment or a blank line in the config used to become the program env(1)
## was asked to run, so the whole command failed
IFS=$'\n'
exec env $(sed -e '/^[[:space:]]*#/d' -e '/^[[:space:]]*$/d' "$MYOS_CONF" 2>/dev/null) \
MYOS=. WORKDIR="${WORKDIR:-${PWD}}" make -esC "${MYOS:-.}" "$@"
+72
View File
@@ -0,0 +1,72 @@
##
# myos, from make.
#
# This file gives a project the myos commands as make targets, without the make
# engine: every target shells out to bin/myos, which is pure POSIX sh. What the
# project keeps from make is what make is actually good at, and the CLI is not:
# its own targets, its own dependencies, and the .mk files of its stacks.
#
# MYOS ?= /usr/local/lib/myos
# include $(MYOS)/share/make/shim.mk
#
# It lives outside make/ on purpose: the legacy engine includes every .mk of
# that directory, and would pull this one in too.
#
# Then `make up STACK=host` and `myos up host` do the same thing, through the
# same code. Variables given on the command line are forwarded, so
# `make up STACK=host DOMAIN=example.org` behaves as expected.
MYOS ?= $(patsubst %/share/make/shim.mk,%,$(lastword $(MAKEFILE_LIST)))
MYOS_BIN ?= $(MYOS)/bin/myos
STACK_DIR_NAME ?= stack
## every directory myos looks for stacks in, so the .mk of a stack installed
## system wide brings its targets along too
STACK_DIR ?= $(subst :, ,$(shell $(MYOS_BIN) env MYOS_PATH --color=never 2>/dev/null | awk '{print $$2}'))
# variable MYOS_ARGS: variables set on the make command line, forwarded to myos
MYOS_ARGS ?= $(foreach v,$(MAKEOVERRIDES),$(v))
.DEFAULT_GOAL := help
## the stack files may add their own targets: that is what make is kept for
include $(foreach dir,$(STACK_DIR),$(wildcard $(dir)/*.mk $(dir)/*/*.mk))
# function make: run a myos command, for the stack .mk files that call it
define make
$(MYOS_BIN) $(MYOS_ARGS) $(1)
endef
# The settings of the stacks, read once and evaluated here.
# A stack keeps its settings in hooks that only myos reads; asking for them one
# at a time costs a process per variable, so they come in a single call.
## Written while this file is read, not as a target: a target would collide
## with the catch-all rule at the bottom, which hands anything else to myos.
MYOS_SETTINGS ?= .myos.settings.mk
MYOS_SETTINGS_FILE := $(shell $(MYOS_BIN) --color=never $(MYOS_ARGS) export --make > $(MYOS_SETTINGS) 2>/dev/null && echo $(MYOS_SETTINGS))
-include $(MYOS_SETTINGS_FILE)
# function myos-var: the value myos resolves for one variable, when a single
# lookup is cheaper than the whole set
myos-var = $(shell $(MYOS_BIN) --color=never $(MYOS_ARGS) env $(1) | awk '{print $$2}')
# target help: List the myos commands
.PHONY: help
help:
@$(MYOS_BIN) help
# target myos: Run an arbitrary myos command, as in `make myos ARGS="up host"`
.PHONY: myos
myos:
@$(MYOS_BIN) $(MYOS_ARGS) $(ARGS)
# make tries to remake every makefile it read, and the catch-all below would
# hand each of them to myos as a command. An empty rule stops that.
$(MAKEFILE_LIST): ;
# target %: Hand anything else to myos
## a target the project defines itself keeps precedence over this rule
%: FORCE
@$(MYOS_BIN) $(MYOS_ARGS) $@ $(ARGS)
.PHONY: FORCE
FORCE: ;
+103
View File
@@ -0,0 +1,103 @@
---
name: myos
description: Deploy, inspect and troubleshoot docker compose stacks on a server or a fleet with the myos CLI (Make Your Own Stack). Use when asked to install myos, to start/stop/debug a stack (a host singleton, an application, a user stack), to add a stack to the catalogue, or to work on the myos framework itself.
---
# myos
myos runs docker compose stacks. It decides which compose files to load, under
which project name, with which variables. Everything else is docker.
## Is this the right tool
Use it when the repository or the host has one of:
- a `stack/` directory holding compose files,
- a `Makefile` that includes `make/include.mk`,
- `/etc/conf.d/myos` or `/etc/default/myos`,
- the `myos` command on PATH.
Otherwise call `docker compose` directly.
## Look before you touch
```sh
myos doctor # requirements, config layers, stack path
myos ls # stacks reachable from here
myos ls --groups # groups and what they expand to
myos env # resolved variables
myos -n up host # print what `up` would run, without running it
```
**Always run `myos -n` before `up`, `down` or `restart` on a `host/` stack.**
Host stacks bind privileged ports and are shared by everything on the machine:
taking `host/fabio` down drops every site the machine serves.
## Working loop
```sh
myos ls # find the stack
myos -n config <stack> # check the file list and the project name
myos up <stack> # create and start
myos ps <stack> # what is running
myos logs <stack> # follow the logs
myos exec <stack> -- sh # a shell in the service named after the stack
myos down <stack> # remove the containers
```
## Naming a stack
A reference is `[<group>/]<name>[:<version>]`, or a path.
Commands chain: `myos build up logs host/fabio` runs the three in order and
stops at the first failure.
```sh
myos up # the current directory, when it holds a compose file
myos up host # a group: expands to host/consul host/fabio ...
myos up host/fabio # one stack
myos up postgres:9.6 # the 9.6 overlay of the postgres stack
myos -C /opt/app up # somewhere else
myos -e master up # in another environment
```
Stacks are looked up along the stack path: `./stack`, `../stack`,
`~/.local/share/myos/stack`, `/usr/local/share/myos/stack`, `/usr/share/myos/stack`.
`myos doctor` prints the resolved path.
A stack found in several of them is **merged**, the project last, so a project
adds `stack/postgres/postgres.local.yml` next to the catalogue's `postgres.yml`
and refines it instead of replacing it. The same goes for the settings hooks.
## Three kinds of stack
| kind | reference | compose project | meaning |
|---|---|---|---|
| host | `host/*` | the hostname | one per machine: binds 80/443, consul, certbot. Shared. Treat as production. |
| user | `User/*` | the user identity | one per user: ssh-agent, personal services |
| app | anything else | `<user>-<env>-<app>` | many per machine, one per user and environment |
Set `MYOS_PROJECT_FORMAT=user-app-env` on a deployment created before the
rename, otherwise its containers and volumes look like a different project.
See `references/conventions.md`.
## Rules
- Check what a stack opens before starting it on a server that faces the
internet: `myos expose <stack>`. A port shown on `0.0.0.0` answers the world,
and on linux the host firewall does not see it, because docker writes its own
rules. A port reported as `unbound` was published without anyone choosing an
address: bind it with `ports: ["${MYOS_BIND_PRIVATE}::<port>"]`.
- Never run `myos clean` on a host stack: it removes images **and volumes**,
including the certificates.
- Secrets belong in a file outside the repository, never in a compose file.
- A stack of the catalogue is shared: change it in `myos-stacks`, not in place
on a server.
- Before upgrading myos on a machine that already runs stacks, pin the naming
(`references/conventions.md`), or every project gets a new name.
## More
- `references/commands.md` — every command, and its equivalent in the old make targets
- `references/conventions.md` — file layout, naming, networks, variables, tags
- `references/troubleshooting.md` — what each failure means and how to fix it
- `references/authoring.md` — writing a stack, and working on myos itself
+78
View File
@@ -0,0 +1,78 @@
# Writing a stack, and working on myos
## A new stack in the catalogue
```
stack/<name>/<name>.yml the services
stack/<name>/<name>.local.yml what only makes sense on a workstation (published ports…)
stack/<name>/<name>.labels.yml the registrator labels, so routing stays optional
stack/<name>/<name>.env plain settings: versions, defaults
stack/<name>/<name>.sh lazy defaults (fabio tags), no make needed
stack/<name>/.env.dist the variables it expects, with defaults
stack/<name>/README.md what it is and what it needs
```
Rules that keep a stack reusable:
- no `container_name`, except in a `host/` stack: it prevents scaling and
collides between users,
- no fixed host port outside `host/`; publish through the load balancer,
- reference variables with a default: `${POSTGRES_VERSION:-16}`,
- attach to `private` to be reachable by the other stacks of the user, to
`public` to be routed,
- name volumes, never bind-mount an absolute path.
Check it before committing:
```sh
myos -n config <name> # the file list and the project
myos config <name> # the rendered yaml
```
## A group
```sh
# stack/<group>.env
mygroup=<name> other/<name>
```
Lowercase, and it may name other groups.
## Working on myos itself
```sh
make test # unit + golden, against both engines, with a mocked docker
make test-golden # golden only
make golden-record # re-record the golden expectations from the make engine
make lint # shellcheck
```
Layout:
```
bin/myos argument parsing, configuration, dispatch
lib/core.sh logging, exit codes, dry run
lib/str.sh strings and version comparison
lib/naming.sh project names, networks, user identity
lib/stack.sh stack path, references, overlays, groups
lib/var.sh variable resolution and lazy defaults
lib/config.sh dotenv, templates, variables of the compose files
lib/hooks.sh the per-stack .env and .sh
lib/compose.sh finding and calling docker compose
lib/tags.sh fabio tags
lib/cmd/<x>.sh one file per command
share/compose/ the networks and volumes overlays myos provides
spec/ shellspec
```
Adding a command: write `lib/cmd/<name>.sh` defining `myos_cmd_<name>`, add it
to the usage text in `bin/myos`, and cover it in `spec/unit`.
Constraints:
- POSIX shell, no bashisms: it has to run under the bash 3.2 of macOS and the
ash of Alpine. `make test` runs on both.
- No `a-z` ranges in a `case` pattern: under a dictionary collation such as
`fr_FR` they also match uppercase. Use `[:lower:]`.
- A function that has to return several values takes them out through printf,
not through a global: a caller inside `$( )` would lose the global.
- Changing what a command prints means updating `spec/golden/expected*/` and
explaining the change in `spec/golden/DELTAS.md`.
+93
View File
@@ -0,0 +1,93 @@
# Commands
```
myos [options] <command> [stack...] [VAR=value...] [-- args...]
```
| option | effect |
|---|---|
| `-C DIR` | work in DIR instead of the current directory |
| `-e ENV` | environment: picks `.env.<env>` and the `<stack>.<env>.yml` overlays |
| `-n`, `--dry-run` | print the commands instead of running them |
| `-v`, `--verbose` | say what myos is doing |
| `-d`, `--debug` | print every command |
| command | effect |
|---|---|
| `up` | create and start; creates the external networks first |
| `down` | remove the containers |
| `start` / `stop` / `restart` | on the existing containers |
| `ps` | what is running |
| `logs` | follow the logs |
| `config` | the resolved compose file |
| `exec` | run a command in a running service |
| `run` | run it in a new container, removed afterwards |
| `scale` | `myos scale <stack> SERVICE=<name> NUM=<n>` |
| `build` / `pull` | images |
| `ls [--groups]` | the stacks and groups myos can see |
| `env [VAR...]` | resolved variables |
| `env-update` | fill the `.env` from the `.env.dist` templates |
| `expose [--strict]` | what the stacks publish, and to whom |
| `cert list\|issue\|renew\|show` | certificates, derived from the route tags |
| `export [--make]` | every setting of the stacks, as `KEY=value` |
| `doctor` | check the installation |
| `version` | the myos version |
## Several commands at once
Commands chain, the way make targets did. Leading words that name commands are
commands; the first word that is not one starts the list of stacks.
```sh
myos build up logs host/fabio # like: make build up logs STACK=host/fabio
myos up ps host
```
They run in order and stop at the first failure. A stack whose name is also a
command name has to be given as `STACK=<name>`, otherwise it is read as a
command.
Anything after `--` goes to docker compose:
```sh
myos logs host/fabio -- --tail 20
myos up postgres -- --force-recreate
```
`exec` and `run` take the service from the stack name, since most stacks name
their main service after themselves. `SERVICE=` picks another one:
```sh
myos exec host/consul -- consul members # service consul, command "consul members"
myos exec host/fabio SERVICE=fabio -- sh
myos run postgres -- psql -l
```
## Exit codes
| code | meaning |
|---|---|
| 0 | fine |
| 1 | the command failed |
| 2 | bad invocation, or an unknown command |
| 3 | stack not found (the message prints the search path) |
| 4 | missing requirement, run `myos doctor` |
The make engine exited 0 on an unknown target, printing only a warning. The CLI
does not: a typo is an error.
## Coming from the make targets
| make | myos |
|---|---|
| `make up STACK=host` | `myos up host` |
| `make print-COMPOSE_FILE` | `myos env COMPOSE_FILE` |
| `make host` | `myos up host` (a bare stack name is not a command) |
| `make build up logs STACK=host/fabio` | `myos build up logs host/fabio` |
| `make stack-host-config` | `myos config host` |
| `make up@master` | `myos -e master up` |
| `make exec SERVICE=php ARGS='ls'` | `myos exec <stack> -- php ls` |
| `make DRYRUN=true up` | `myos -n up` |
`print-VAR`, `stack-<stack>-<command>` and `<command>@<env>` still work.
A project `Makefile` that includes `make/include.mk` keeps working too.
+267
View File
@@ -0,0 +1,267 @@
# Conventions
## A stack is a directory of compose files
For stack `<name>` in environment `<env>`, myos loads whichever of these exist,
in this order. A later file overrides the ones before it.
```
<name>.yml the stack
<name>.<env>.yml this environment only
<env>/<name>.yml same, when the stack keeps a directory per environment
<name>.<suffix>.yml an optional overlay, see below
<name>.<suffix>.<env>.yml
<name>.<version>.yml when the reference is <name>:<version>
```
`docker-compose.yml` is read under the same rules, so a stack can keep an
upstream `docker-compose.yml` untouched and add its own `<name>.yml` on top.
Then myos appends its own `share/compose/networks.yml`, and the
`volumes.<suffix>.<env>.yml` of the enabled suffixes.
## Overlay suffixes
Every `COMPOSE_FILE_<X>` variable that is not `false` enables the suffix `<x>`.
| variable | default | loads |
|---|---|---|
| `COMPOSE_FILE_APP` | true | `<name>.app.yml` |
| `COMPOSE_FILE_LABELS` | true | `<name>.labels.yml`, the registrator labels |
| `COMPOSE_FILE_NETWORKS` | true | `<name>.networks.yml` |
| `COMPOSE_FILE_SSH` | true | `<name>.ssh.yml` |
| `COMPOSE_FILE_VOLUMES` | true | `<name>.volumes.yml` |
| `COMPOSE_FILE_WWW`, `_DNS`, `_HOME`, `_LOG`, `_BACKUP` | false | the matching overlay, and the framework bind mount |
A value that is neither `true` nor `false` also enables `<suffix>.<value>`:
`COMPOSE_FILE_WWW=nginx` loads `<name>.www.yml` **and** `<name>.www.nginx.yml`.
## Project names
| scope | project | why |
|---|---|---|
| `host/*` | `$HOSTNAME` | a singleton of the machine: one consul, one fabio on 80/443 |
| `User/*` | the user identity derived from the mail address | one per user |
| `cluster/*` | the stack name | one namespace per swarm |
| anything else | `<user>-<env>-<app>` | one per user, environment and app |
`MYOS_PROJECT_FORMAT=user-app-env` restores the older order. **Any deployment
created before the rename must set it**, in its `.env` or in
`/etc/conf.d/myos`; otherwise the containers and volumes of the old project
are orphaned and the stack comes back up empty.
## Networks
| network | name | scope |
|---|---|---|
| `default` | `_<project>` | the project. The leading underscore keeps it first alphabetically, so it is the first interface attached and service names never resolve across stacks. |
| `private` | `<user>-<env>` | external, shared between the stacks of one user and environment |
| `public` | `<hostname>` | external, where the load balancer reaches the services |
`myos up` creates the external ones when they are missing.
## Routing: registrator, consul, fabio
A service is published by labels, which registrator copies into consul, and on
which fabio routes:
```yaml
labels:
- SERVICE_8000_NAME=${COMPOSE_SERVICE_NAME}-kong-8000
- SERVICE_8000_TAGS=${SUPABASE_KONG_SERVICE_8000_TAGS:-urlprefix-supabase.localhost/*}
- SERVICE_8000_CHECK_TCP=true
- SERVICE_8001_IGNORE=true
```
Registrator only sees ports that are `expose`d, and consul only routes a
service whose check passes.
Tag variables follow `<STACK>_SERVICE_<port>_<KEY>`, with `PATH`, `OPTS`,
`URIS`, `NAME`, and the fabio options `allow`, `auth`, `deny`, `prepend`,
`proto`, `register`, `strip`:
```sh
HOST_NGINX_SERVICE_443_PROTO='https tlsskipverify=true'
DUNITER_V2S_SERVICE_9944_STRIP=/ws
```
## Configuration
Layers, the last one wins:
```
defaults < /etc/conf.d/myos, /etc/default/myos < ~/.config/myos/config
< <workdir>/.env < <workdir>/.env.<env> < environment < VAR=value on the command line
```
All of them are dotenv: `KEY=value`, `#` comments, optional quotes. They are
parsed, never sourced, so a value may contain a `#` or a `$(...)` without
breaking anything or being executed. The make engine included `.env` as a
makefile, where both broke.
## Where a stack lives
The same stack may exist in several directories of the stack path. They are all
loaded, least specific first, so the project has the last word:
```
/usr/local/share/myos/stack/postgres/postgres.yml the catalogue
./stack/postgres/postgres.local.yml the project refines it
./stack/postgres/postgres.sh and may redefine a default
```
Nothing has to be copied to change one setting: a value in the project `.env`
already wins over any default a stack ships.
## Per-stack settings
A stack keeps its own settings next to its compose files:
| file | for |
|---|---|
| `<name>.env` | plain values: versions, defaults |
| `<name>.env.<env>` | the same, for one environment |
| `<name>.sh` | values that have to be computed |
| `<name>.mk` | the legacy make snippet; still read for its groups |
A `.sh` hook declares **lazy defaults**: a function named
`myos_default_<VARIABLE>`, called only when the variable has no value, and
called again at each reference. That is the make `?=` on a recursive variable,
in shell:
```sh
# stack/host/fabio.sh
myos_default_HOST_FABIO_VERSION() { printf '1.6.3'; }
myos_default_HOST_FABIO_SERVICE_9998_NAME() { printf 'fabio'; }
myos_default_HOST_FABIO_SERVICE_9998_AUTH() { printf 'default'; }
myos_default_HOST_FABIO_SERVICE_9998_TAGS() { myos_tagprefix HOST_FABIO 9998; }
```
Two things follow, and they are the point:
- a value given anywhere (environment, `.env`, command line) wins over the
default, without the hook having to say so;
- the default is computed against the values current **at the moment it is
read**, so a `DOMAIN` set in a `.env` loaded later is taken into account.
The prefix is not decoration: a bare function named `host` or `test` would be
indistinguishable from the program of that name, and myos would run it.
Helpers available in a hook: `myos_tagprefix`, `myos_urlprefix`, `myos_uri`,
`myos_url`, `myos_envprefix`, `myos_servicenvs`, `myos_var`, `myos_lower`,
`myos_upper`, and `myos_default NAME 'body'` when the name is built at run time.
## Templates: .env.dist
A stack may ship a `.env.dist` listing the variables it expects, with their
defaults. `myos env-update` writes the missing ones into the `.env`, expanding
`${VAR}` against the current values and running `$(command)`:
```sh
# stack/demo/.env.dist
DEMO_IMAGE=alpine:${DEMO_VERSION}
DEMO_VERSION=3.20
DEMO_SECRET=$(openssl rand -hex 16)
```
A line may refer to a variable defined further down. A variable that already
has a value keeps it: the `.env` records choices, it never overwrites them, and
running the command twice changes nothing.
## What a stack publishes, and to whom
On linux docker writes its own firewall rules, so a port published with
`ports: ["8080:80"]` answers the internet whatever the host firewall says. The
portable answer is to publish where you mean to, which behaves the same on
linux and on macOS and needs no privilege:
```yaml
services:
app:
ports:
- "${MYOS_BIND_PRIVATE}::8080" # this host only, reached through fabio
gateway:
ports:
- "${MYOS_BIND_PUBLIC}:443:443" # the internet, on purpose
peer:
ports:
- "${MYOS_BIND_MESH}::7946" # the private network between the hosts
```
| scope | address | for |
|---|---|---|
| `private` | `127.0.0.1` | everything the load balancer reaches for you. The default. |
| `public` | `0.0.0.0` | a load balancer, a public DNS or mail service |
| `mesh` | the mesh interface | services shared between the hosts of a fleet |
`MYOS_BIND_PUBLIC`, `MYOS_BIND_PRIVATE` and `MYOS_BIND_MESH` override the
addresses; `MYOS_MESH_IFACE` names the interface when it is not one of
easytier, tun0, tailscale0, mycelium or wg0.
There is nothing else to declare: the scope **is** the binding the file asks
for. A port written `- 80` or `- "9000:9000"` is *unbound*, which means docker
opens it on every address and nobody chose that.
```sh
myos expose # what each stack publishes, on which address
myos expose --strict # exits 1 when a port is published without a binding
```
The command reads the compose files as written **and** the resolved
configuration, and shows both: the binding the stack asked for, and the address
it ends up on. Resolving first would lose the difference, since
`${MYOS_BIND_PRIVATE}` and a hand-written `127.0.0.1` both become `127.0.0.1`,
and an unbound port becomes `0.0.0.0` exactly like a deliberate public one.
The split of responsibility: the **scope** belongs to the stack, in its compose
file; the **address** of a scope belongs to the host, in its configuration.
## Certificates
A site gets a certificate by being routed, not by being written down a second
time. `myos cert` reads the same `urlprefix-` tags fabio routes on, and decides
what to ask for:
```sh
myos cert list # what would be asked for, and over which challenge
myos cert issue # ask for it
myos cert renew # what is close to expiry, for a cron
myos cert show # what exists, and when it expires
```
| a tag routes | myos asks for | challenge |
|---|---|---|
| `app.example.org` | a certificate for that name | http-01 |
| `*.ipns.example.org` | `ipns.example.org` **and** `*.ipns.example.org` | dns-01 |
That is the whole of "per site or wildcard as needed": a wildcard is asked for
where a tag uses one, and it absorbs the concrete names it covers. A wildcard
covers one label, so `*.example.org` absorbs `a.example.org` but not
`a.b.example.org`, which keeps its own certificate.
`MYOS_CERT_MODE=per-site` never asks for a wildcard, which keeps everything on
http-01 and needs no DNS credentials. `wildcard` asks for one per domain.
The issuer is [dehydrated](https://github.com/dehydrated-io/dehydrated), a
shell script, in the `host/dehydrated` stack. It answers http-01 itself on a
port bound to the loopback, which fabio routes
`/.well-known/acme-challenge/` to. A wildcard needs dns-01, so point
`HOST_DEHYDRATED_DNS_HOOK` at your provider's script; it receives dehydrated's
own hook arguments.
Certificates land where fabio looks for them, `<name>-cert.pem` and
`<name>-key.pem` under `/host/certs`, written to a temporary name and moved, so
fabio never reads half a file.
## Groups
A group is a lowercase name whose value lists stacks. It can live in a `.env`,
in the environment, in `<path>/<group>.env`, or in a legacy `<group>.mk`:
```sh
host=host/consul host/fabio host/registrator
testing=drone/drone drone/gc
```
Groups expand recursively. Only lowercase names are considered, so an
environment variable can never be mistaken for a group.
+64
View File
@@ -0,0 +1,64 @@
# Troubleshooting
Start with `myos doctor`, then `myos -n <command>` to see what would run.
## `stack not found: <name>` (exit 3)
The reference is not on the stack path, which the message prints.
- `myos ls` shows what is reachable.
- The catalogue may not be installed: `git clone <myos-stacks> /usr/local/share/myos`.
- A stack of the project is only found from the project: use `myos -C /path/to/project`.
## `docker compose >= 2.24.4 not found` (exit 4)
Install the compose plugin, or a `docker-compose` binary of that version.
myos no longer falls back to a compose image.
## `unknown command: <x>` (exit 2)
Check the spelling against `myos help`. The make engine accepted any target
and exited 0 after printing a warning, so typos used to look like successes.
## The containers came back under a different name
The default project name changed from `<user>-<app>-<env>` to
`<user>-<env>-<app>`. The old containers and volumes are still there, under the
old project. Put `MYOS_PROJECT_FORMAT=user-app-env` in the `.env` of the
deployment (or in `/etc/conf.d/myos`) and bring it up again.
Check first: `myos env COMPOSE_PROJECT_NAME` against `docker ps --format '{{.Names}}'`.
## `network <name> declared as external, but could not be found`
The `private` or `public` network is missing. `myos up` creates them; a bare
`docker compose up` does not. Or create it by hand:
`docker network create <user>-<env>`.
## A service is up but not routed
In order: the port must be `expose`d (registrator ignores what it cannot see),
the labels must be on the service, the consul check must pass, and only then
does fabio route the `urlprefix-` tag.
```sh
myos config <stack> | grep -A5 labels # what the labels resolve to
myos exec host/consul -- consul catalog services
myos logs host/registrator
```
## A variable is empty in the container
myos only passes the variables the compose files actually mention. Check with
`myos env` and `myos config <stack>`. A variable set in a `.env` of another
directory is not read: only the workdir's `.env` is.
## On macOS with Colima
The daemon lives in a VM: a bind mount only works for a path the VM shares, and
`host.docker.internal` is the way back to the host. `myos doctor` prints the
`DOCKER_HOST` in use.
## Something changed after upgrading myos
`spec/golden/DELTAS.md` in the myos repository lists every intentional
difference between the make engine and the CLI, with the reason.
+40
View File
@@ -0,0 +1,40 @@
# Benchmark of the engines
Same work, four engines, five runs, median. Docker is the mock of
`spec/support/bin`, the catalogue is `myos-stacks` reached through
`$HOME/.local/share/myos/stack`, the environment is `env.sh`.
sh spec/bench/run.sh
`justfile` is a prototype of just as the engine: shebang recipes that source
`lib/*.sh` once. `go/main.go` is a prototype of the core in Go: stack path,
groups, compose files, project name, dry-run command; `export` runs one `sh`
per stack directory to evaluate the shell hooks.
## Results, 2026-09-05, Mac Studio M2 Ultra
| work | make | sh (bin/myos) | just | go |
|---|---:|---:|---:|---:|
| fixed cost, empty target | 312 ms | 50 ms | 170 ms | 23 ms |
| `up` 1 / 3 stacks, no hooks in the stack | 792 / 2201 | 177 / 332 | 204 / 268 | 23 / 23 |
| `up` 1 / 3 stacks, real catalogue with hooks | — | 353 / 855 | (prototype does not load hooks) | (idem) |
| `export`, 80 settings of the `host` group | — | 1477 | 1314 | 720 (1 sh) |
| same, `MYOS_VAR_MEMO=1` | — | 1503 | 1318 | 774 |
| one computed setting (`HOST_FABIO_SERVICE_9998_TAGS`) | — | ~43 ms net (69 26) | | |
Reference points: `sh -c :` 24 ms, `just --version` 27 ms, sourcing `lib/*.sh` +2 ms.
## What it says
- The engine's own cost: go flat at 23 ms; sh 177 ms + ~63 ms per stack; just
204 ms + ~32 ms per stack; make 792 ms + ~700 ms per stack (it re-reads
itself for every stack).
- The shell hooks cost ~40 ms per computed setting, on every engine: 80
settings ≈ 0.7 s even from Go, which runs the very same `sh`. Memoisation
changes nothing, because the cost is not repeated lookups: each `tagprefix`
spawns 15-20 command substitutions for distinct, mostly empty, variables.
- `bin/myos` doubles that to 1.5 s by loading the hooks of a directory once
per stack reference instead of once per directory: `host/consul`,
`host/fabio` and `host/registrator` share `stack/host/_stack.sh`.
- just's fixed cost (170 ms for a shebang recipe, against 27 ms for `just
--version`) is its own overhead of writing and running the recipe script.
+16
View File
@@ -0,0 +1,16 @@
#!/bin/sh
# bench.sh LABEL N -- CMD... run CMD N times, print the median wall time in ms
# Hermetic: docker is the mock of spec/support/bin, config comes from the
# environment only, HOME points at the fixture catalogue.
set -u
label=$1; n=$2; shift 2; [ "$1" = "--" ] && shift
i=0; times=""
while [ "$i" -lt "$n" ]; do
s=$(python3 -c 'import time;print(int(time.time()*1e6))')
"$@" >/dev/null 2>&1
e=$(python3 -c 'import time;print(int(time.time()*1e6))')
times="$times $(( (e - s) / 1000 ))"
i=$((i + 1))
done
median=$(printf '%s\n' $times | sort -n | awk '{a[NR]=$1} END {print a[int((NR+1)/2)]}')
printf '%-44s %6s ms (runs:%s)\n' "$label" "$median" "$times"
+5
View File
@@ -0,0 +1,5 @@
# the hermetic environment every engine runs in
export PATH=/Users/aya/dev/myos/spec/support/bin:/Users/aya/.local/bin:/usr/bin:/bin
export HOME=/tmp/myos-bench/home WORKDIR=/tmp/myos-bench/wd MYOS_ROOT=/Users/aya/dev/myos
export USER=tester HOSTNAME=testhost DOMAIN=example.test ENV=local DRYRUN=true
export MYOS_CONF=/dev/null MYOS_PROJECT_FORMAT=user-app-env DOCKER_MACHINE=x86_64 DOCKER_SYSTEM=Linux
+238
View File
@@ -0,0 +1,238 @@
// A prototype of the myos core in Go, just large enough to be benchmarked
// fairly against the other engines: stack path, group expansion, compose file
// resolution across every directory of the path, project name, and the
// dry-run compose command. Same rules as lib/stack.sh and lib/naming.sh.
//
// export delegates the shell hooks to ONE sh per stack directory, which is
// what a Go engine would do to keep the developer contract in shell.
package main
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
)
func env(k, def string) string {
if v := os.Getenv(k); v != "" {
return v
}
return def
}
// stackPath: the directories stacks are looked up in, project first
func stackPath(workdir string) []string {
home := env("HOME", "/nonexistent")
root := env("MYOS_ROOT", ".")
prefix := filepath.Dir(filepath.Dir(root))
var out []string
seen := map[string]bool{}
for _, d := range []string{workdir, filepath.Join(workdir, ".."), filepath.Join(home, ".local/share"), filepath.Join(prefix, "share"), "/usr/local/share", "/usr/share"} {
for _, c := range []string{filepath.Join(d, "stack"), filepath.Join(d, "myos/stack")} {
if st, err := os.Stat(c); err == nil && st.IsDir() {
if r, err := filepath.EvalSymlinks(c); err == nil {
c = r
}
if !seen[c] {
seen[c] = true
out = append(out, c)
}
}
}
}
return out
}
// groupValue: the list a lowercase group name expands to, from <g>.env,
// <g>/<g>.env or <g>/_stack.env along the path
func groupValue(path []string, name string) string {
if strings.ContainsAny(name, "/:.") || strings.ToLower(name) != name {
return ""
}
if v := os.Getenv(name); v != "" {
return v
}
for _, d := range path {
for _, f := range []string{filepath.Join(d, name+".env"), filepath.Join(d, name, name+".env"), filepath.Join(d, name, "_stack.env")} {
b, err := os.ReadFile(f)
if err != nil {
continue
}
for _, line := range strings.Split(string(b), "\n") {
if strings.HasPrefix(line, name+"=") {
return strings.Trim(strings.TrimPrefix(line, name+"="), "\"")
}
}
}
}
return ""
}
func expand(path []string, refs []string, depth int) []string {
var out []string
for _, r := range refs {
if v := groupValue(path, r); v != "" && depth < 16 {
out = append(out, expand(path, strings.Fields(v), depth+1)...)
} else {
out = append(out, r)
}
}
return out
}
func stackName(ref string) string {
r := strings.TrimSuffix(ref, "/")
if i := strings.LastIndex(r, ":"); i >= 0 {
r = r[:i]
}
return strings.TrimSuffix(filepath.Base(r), ".yml")
}
// stackDirs: every directory of the path holding the stack, least specific first
func stackDirs(path []string, ref string) []string {
r := strings.TrimSuffix(ref, "/")
if i := strings.LastIndex(r, ":"); i >= 0 {
r = r[:i]
}
name := stackName(ref)
var found []string
for _, d := range path {
var hit string
if st, err := os.Stat(filepath.Join(d, r)); err == nil && st.IsDir() {
hit = filepath.Join(d, r)
} else if _, err := os.Stat(filepath.Join(d, r+".yml")); err == nil {
hit = filepath.Dir(filepath.Join(d, r))
} else if st, err := os.Stat(filepath.Join(d, name)); err == nil && st.IsDir() {
hit = filepath.Join(d, name)
}
if hit != "" {
found = append([]string{hit}, found...)
}
}
return found
}
func exists(p string) bool { _, err := os.Stat(p); return err == nil }
// composeFiles: the files that exist, in the order the framework loads them
func composeFiles(dir string, names, suffixes []string, envName string) []string {
var out []string
for _, e := range []string{"yml", "yaml"} {
for _, n := range names {
for _, f := range []string{
filepath.Join(dir, n+"."+e), filepath.Join(dir, n+"."+envName+"."+e),
filepath.Join(dir, envName, n+"."+e), filepath.Join(dir, envName, n+"."+envName+"."+e)} {
if exists(f) {
out = append(out, f)
}
}
for _, s := range suffixes {
for _, f := range []string{filepath.Join(dir, n+"."+s+"."+e), filepath.Join(dir, n+"."+s+"."+envName+"."+e)} {
if exists(f) {
out = append(out, f)
}
}
}
}
}
return out
}
func scope(ref string) string {
switch strings.SplitN(ref, "/", 2)[0] {
case "host":
return "host"
case "User", "user":
return "user"
case "cluster":
return "cluster"
}
return "app"
}
func projectName(sc, user, envName, app string) string {
switch sc {
case "host":
return env("HOST_COMPOSE_PROJECT_NAME", env("HOSTNAME", "localhost"))
case "user":
return user
case "cluster":
return strings.ToLower(app)
}
n := strings.NewReplacer(".", "", "-", "", "_", "").Replace(strings.ToLower(app))
if env("MYOS_PROJECT_FORMAT", "user-env-app") == "user-app-env" {
return user + "-" + n + "-" + envName
}
return user + "-" + envName + "-" + n
}
func main() {
if len(os.Args) < 2 {
fmt.Fprintln(os.Stderr, "usage: myos-go <noop|up|export> [stack...]")
os.Exit(2)
}
workdir := env("WORKDIR", ".")
envName := env("ENV", "local")
user := env("USER", "tester")
path := stackPath(workdir)
suffixes := []string{"app", "labels", "networks", "ssh", "volumes", "latest"}
switch os.Args[1] {
case "noop":
return
case "up":
refs := expand(path, os.Args[2:], 0)
byProject := map[string][]string{}
var order []string
for _, ref := range refs {
app := stackName(ref)
var files []string
for _, d := range stackDirs(path, ref) {
files = append(files, composeFiles(d, []string{"docker-compose", app}, suffixes, envName)...)
}
p := projectName(scope(ref), user, envName, app)
if _, ok := byProject[p]; !ok {
order = append(order, p)
}
byProject[p] = append(byProject[p], files...)
}
for _, p := range order {
files := append(byProject[p], filepath.Join(env("MYOS_ROOT", "."), "share/compose/networks.yml"))
var b strings.Builder
b.WriteString("docker compose")
for _, f := range files {
b.WriteString(" -f " + f)
}
fmt.Printf("%s -p %s up -d\n", b.String(), p)
}
case "export":
// one sh per stack directory evaluates its hooks and prints every value
refs := expand(path, os.Args[2:], 0)
seen := map[string]bool{}
var dirs []string
for _, ref := range refs {
for _, d := range stackDirs(path, ref) {
if !seen[d] {
seen[d] = true
dirs = append(dirs, d)
}
}
}
sort.Strings(dirs)
root := env("MYOS_ROOT", ".")
for _, d := range dirs {
script := fmt.Sprintf(`for m in core str var tags naming stack config compose hooks; do . %s/lib/$m.sh; done
[ -f %s/_stack.sh ] || exit 0
myos_stack_hooks %s _
for v in $(sed -n 's/^myos_default_\([A-Za-z_][A-Za-z0-9_]*\)().*/\1/p' %s/_stack.sh | sort -u); do printf '%%s=%%s\n' "$v" "$(myos_var "$v")"; done`, root, d, d, d)
cmd := exec.Command("sh", "-c", script)
cmd.Env = os.Environ()
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
_ = cmd.Run()
}
}
}
+52
View File
@@ -0,0 +1,52 @@
# just as the engine: the interface is a justfile, the logic stays in lib/*.sh.
# Recipes are shebang recipes, so a whole body runs in ONE sh that sources
# lib/ once; what is measured is just's own overhead on top of the shell.
set export
MYOS_ROOT := env_var_or_default("MYOS_ROOT", "/Users/aya/dev/myos")
WORKDIR := env_var_or_default("WORKDIR", justfile_directory())
ENV := env_var_or_default("ENV", "local")
USER := env_var_or_default("USER", "tester")
HOSTNAME := env_var_or_default("HOSTNAME", "testhost")
DOMAIN := env_var_or_default("DOMAIN", "example.test")
DRYRUN := env_var_or_default("DRYRUN", "true")
# a recipe that does nothing: the fixed cost of just + one sh + sourcing lib/
noop:
#!/bin/sh
for m in core str var tags naming stack config compose hooks; do . $MYOS_ROOT/lib/$m.sh; done
# up STACKS: resolve every stack of the groups, group by compose project,
# print one compose command per project (what bin/myos does)
up +stacks:
#!/bin/sh
for m in core str var tags naming stack config compose hooks; do . $MYOS_ROOT/lib/$m.sh; done
rows=""
for ref in $(myos_group_expand {{stacks}}); do
files=""
for d in $(myos_stack_dirs "$ref"); do
files="$files $(myos_compose_files "$d" "docker-compose $(myos_stack_name "$ref")" "$(myos_compose_suffixes)" "$ENV" | tr '\n' ' ')"
done
app=$(myos_stack_name "$ref")
project=$(myos_project_name "$(myos_scope "$ref")" "$USER" "$ENV" "$app")
rows="$rows
$project|$files"
done
for project in $(printf '%s\n' "$rows" | sed '/^$/d' | cut -d'|' -f1 | awk '!s[$0]++'); do
files=$(printf '%s\n' "$rows" | awk -F'|' -v p="$project" '$1==p {print $2}' | tr ' ' '\n' | sed '/^$/d' | awk '!s[$0]++')
fargs=""; for f in $files $MYOS_ROOT/share/compose/networks.yml; do fargs="$fargs -f $f"; done
echo "docker compose$fargs -p $project up -d"
done
# export STACKS: every setting the hooks of the stacks declare
export +stacks:
#!/bin/sh
for m in core str var tags naming stack config compose hooks; do . $MYOS_ROOT/lib/$m.sh; done
refs=$(myos_group_expand {{stacks}})
for ref in $refs; do
for d in $(myos_stack_dirs "$ref"); do myos_stack_hooks "$d" "$(myos_stack_name "$ref")"; done
done
names=$(for ref in $refs; do for d in $(myos_stack_dirs "$ref"); do
[ -f "$d/_stack.sh" ] && sed -n 's/^myos_default_\([A-Za-z_][A-Za-z0-9_]*\)().*/\1/p' "$d/_stack.sh"; done; done | sort -u)
for v in $names; do printf '%s=%s\n' "$v" "$(myos_var "$v")"; done
+39
View File
@@ -0,0 +1,39 @@
#!/bin/sh
# the full matrix: 5 runs each, median, every engine on the same work
. /tmp/myos-bench/env.sh; cd "$WORKDIR"
B=/tmp/myos-bench/bench.sh; N=5
MK="make -esC $MYOS_ROOT MYOS=. WORKDIR=$WORKDIR"
SH="$MYOS_ROOT/bin/myos"
JU="just --justfile /tmp/myos-bench/justfile"
GO=/tmp/myos-bench/myos-go
S1="host/consul"; S2="host/consul host/fabio"; S3="host/consul host/fabio host/registrator"
echo "== cout fixe : demarrage + cible vide"
$B "make noop" $N -- $MK FORCE
$B "sh noop (myos version)" $N -- $SH version
$B "just noop (parse + 1 sh + source lib/)" $N -- $JU noop
$B "go noop" $N -- $GO noop
echo
echo "== up : 1 / 2 / 3 stacks, dry-run"
$B "make up 1" $N -- $MK up STACK="$S1"
$B "make up 2" $N -- $MK up STACK="$S2"
$B "make up 3" $N -- $MK up STACK="$S3"
$B "sh up 1" $N -- $SH up host/consul
$B "sh up 2" $N -- $SH up host/consul host/fabio
$B "sh up 3" $N -- $SH up host/consul host/fabio host/registrator
$B "just up 1" $N -- $JU up host/consul
$B "just up 2" $N -- $JU up host/consul host/fabio
$B "just up 3" $N -- $JU up host/consul host/fabio host/registrator
$B "go up 1" $N -- $GO up host/consul
$B "go up 2" $N -- $GO up host/consul host/fabio
$B "go up 3" $N -- $GO up host/consul host/fabio host/registrator
echo
echo "== export : les 80 reglages du groupe host (evaluation des hooks shell)"
$B "sh export, hooks tels quels" $N -- $SH export STACK=host
$B "just export, hooks tels quels" $N -- $JU export host
$B "go export, hooks tels quels (1 sh/repertoire)" $N -- $GO export host
echo
echo "== export : memes hooks, evalues en une passe (MYOS_VAR_MEMO=1)"
MYOS_VAR_MEMO=1 $B "sh export, memoise" $N -- $SH export STACK=host
MYOS_VAR_MEMO=1 $B "just export, memoise" $N -- $JU export host
MYOS_VAR_MEMO=1 $B "go export, memoise" $N -- $GO export host
+2
View File
@@ -0,0 +1,2 @@
APP_SERVICE_8080_TAGS=urlprefix-app.${DOMAIN}/*
POSTGRES_PASSWORD=changeme
+2
View File
@@ -0,0 +1,2 @@
MYOS ?= /usr/local/lib/myos
-include $(MYOS)/make/include.mk
+4
View File
@@ -0,0 +1,4 @@
services:
app:
environment:
DEBUG: "true"
+13
View File
@@ -0,0 +1,13 @@
services:
app:
image: alpine:3.20
command: sleep infinity
environment:
APP_DOMAIN: ${APP_DOMAIN}
DOMAIN: ${DOMAIN}
labels:
- SERVICE_8080_NAME=${COMPOSE_SERVICE_NAME}-app-8080
- SERVICE_8080_TAGS=${APP_SERVICE_8080_TAGS:-urlprefix-app.localhost/*}
networks:
- default
- private
+5
View File
@@ -0,0 +1,5 @@
services:
db:
image: postgres:16-alpine
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
+3
View File
@@ -0,0 +1,3 @@
services:
web:
image: nginx:alpine
@@ -0,0 +1,6 @@
services:
myos:
image: ${USER_DOCKER_IMAGE:-myos:latest}
container_name: ${USER_COMPOSE_PROJECT_NAME:-user}
volumes:
- ${USER_DOCKER_VOLUME:-user}:/tmp/ssh-agent
+1
View File
@@ -0,0 +1 @@
default ?= postgres redis
@@ -0,0 +1,3 @@
services:
drone:
image: drone/drone:1.6
@@ -0,0 +1,2 @@
DRONE_SERVER_HOST ?= drone.$(DOMAIN)
ENV_VARS += DRONE_SERVER_HOST
@@ -0,0 +1,7 @@
services:
drone:
image: drone/drone:${DRONE_VERSION:-latest}
environment:
DRONE_SERVER_HOST: ${DRONE_SERVER_HOST:-drone.localhost}
networks:
- private
@@ -0,0 +1,4 @@
services:
nginx:
volumes:
- dns:/dns
@@ -0,0 +1,4 @@
services:
nginx:
volumes:
- www:/var/www
@@ -0,0 +1,6 @@
services:
nginx:
image: nginx:alpine
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-nginx
ports:
- "8080:80"
@@ -0,0 +1,3 @@
services:
postgres:
image: postgres:9.6
@@ -0,0 +1,4 @@
services:
postgres:
ports:
- "5432:5432"
@@ -0,0 +1,13 @@
services:
postgres:
image: postgres:${POSTGRES_VERSION:-latest}
environment:
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
labels:
- SERVICE_5432_NAME=${COMPOSE_SERVICE_NAME}-postgres-5432
networks:
- private
volumes:
- postgres:/var/lib/postgresql/data
volumes:
postgres:
@@ -0,0 +1,5 @@
services:
redis:
image: redis:alpine
networks:
- private
+1
View File
@@ -0,0 +1 @@
testing ?= drone/drone redis
+12
View File
@@ -0,0 +1,12 @@
services:
consul:
image: hashicorp/consul:1.15
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-consul
network_mode: host
restart: always
environment:
CONSUL_HTTP_TOKEN: ${HOST_CONSUL_HTTP_TOKEN}
volumes:
- consul:/consul/data
volumes:
consul:
+13
View File
@@ -0,0 +1,13 @@
services:
fabio:
image: fabiolb/fabio:1.6.3
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-fabio
depends_on: [consul]
ports:
- "80:80"
- "443:443"
environment:
FABIO_REGISTRY_CONSUL_ADDR: ${DOCKER_HOST_INET4:-127.0.0.1}:8500
FABIO_LOG_ACCESS_TARGET: ${HOST_FABIO_LOG_ACCESS:-}
networks:
- public
+10
View File
@@ -0,0 +1,10 @@
# Stack host: consul + fabio (80/443) + registrator, one instance per host.
host ?= host/consul host/fabio host/registrator
.PHONY: host
host:
$(call make,up STACK="$(host)")
.PHONY: host-down
host-down:
$(call make,down STACK="$(host)")
+9
View File
@@ -0,0 +1,9 @@
services:
registrator:
image: gliderlabs/registrator:master
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-registrator
network_mode: host
depends_on: [consul]
command: -internal=false -useIpFromLabel SERVICE_ADDRESS consul://127.0.0.1:8500
volumes:
- ${DOCKER_SOCKET_LOCATION:-/var/run/docker.sock}:/tmp/docker.sock
+2
View File
@@ -0,0 +1,2 @@
MYOS ?= /usr/local/lib/myos
include $(MYOS)/share/make/shim.mk
+12
View File
@@ -0,0 +1,12 @@
services:
consul:
image: hashicorp/consul:1.15
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-consul
network_mode: host
restart: always
environment:
CONSUL_HTTP_TOKEN: ${HOST_CONSUL_HTTP_TOKEN}
volumes:
- consul:/consul/data
volumes:
consul:
+13
View File
@@ -0,0 +1,13 @@
services:
fabio:
image: fabiolb/fabio:1.6.3
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-fabio
depends_on: [consul]
ports:
- "80:80"
- "443:443"
environment:
FABIO_REGISTRY_CONSUL_ADDR: ${DOCKER_HOST_INET4:-127.0.0.1}:8500
FABIO_LOG_ACCESS_TARGET: ${HOST_FABIO_LOG_ACCESS:-}
networks:
- public
+7
View File
@@ -0,0 +1,7 @@
host ?= host/consul host/fabio
# a target of the project, on top of the myos commands: this is what make is
# kept for, and what the shim leaves alone
.PHONY: host-certs
host-certs:
@echo "would renew the certificates of $(host)"
+9
View File
@@ -0,0 +1,9 @@
services:
registrator:
image: gliderlabs/registrator:master
container_name: ${HOST_COMPOSE_PROJECT_NAME:-localhost}-registrator
network_mode: host
depends_on: [consul]
command: -internal=false -useIpFromLabel SERVICE_ADDRESS consul://127.0.0.1:8500
volumes:
- ${DOCKER_SOCKET_LOCATION:-/var/run/docker.sock}:/tmp/docker.sock

Some files were not shown because too many files have changed in this diff Show More