Configuration
How a Dockflow project is configured, with dockflow.yml, Compose files, environments, variables, secrets and the two interpolation syntaxes.
A project is configured by three things:
- one
dockflow.ymlfor the deploy settings; - standard Compose files for the app;
- secrets, kept out of the repository.
This page explains how they fit together. Every key is in the dockflow.yml reference and the Compose reference.
The files
x-dockflow keysdockflow.ymlsits at the root of the project. Commands look for it in the current directory, then in its parents.-C <dir>points at another project. Every key is checked: an unknown key is an error, not a warning.- Compose files are standard Compose, and run as they are with
docker compose up. Dockflow's per-service settings (routes, rollout, health, stateful, jobs) go underx-dockflow, a key Compose ignores. By default, Dockflow reads the first ofcompose.yaml,compose.yml,docker-compose.yamlanddocker-compose.yml.compose: [a.yml, b.yml]lists several files, merged in order as Compose merges them. - Secrets come from CI variables, the
DOCKFLOW_SECRETSvariable, or a local.env.dockflowfile. They never need to be in the repository.
To check the configuration:
dockflow validatechecks all of it without contacting a server.dockflow config showprints the resolved configuration and services, with secrets masked.dockflow list servicesshows each service as it will be deployed.
A complete example
version: 1
app: shop
on_failure: rollback
releases:
keep: 10
health:
endpoints:
- url: https://${DOMAIN}/
environments:
production:
vars:
DOMAIN: shop.example.com
staging:
vars:
DOMAIN: staging.shop.example.com
on_failure: fail
releases:
keep: 3services:
web:
build: .
environment:
DATABASE_URL: postgres://shop:${DB_PASSWORD}@db:5432/shop
APP_ENV: ${DOCKFLOW_ENV}
LOG_LEVEL: ${LOG_LEVEL:-info}
depends_on:
db:
condition: service_healthy
x-dockflow:
routes:
- domains: ["${DOMAIN}"]
port: 3000
health:
http: { path: /healthz }
db:
image: postgres:17
environment:
POSTGRES_USER: shop
POSTGRES_PASSWORD: ${DB_PASSWORD}
healthcheck:
test: ["CMD-SHELL", "pg_isready -U shop"]
volumes: [pgdata:/var/lib/postgresql/data]
x-dockflow:
stateful: true
volumes:
pgdata:Where each value comes from:
DOMAINcomes from the environment'svars.DOCKFLOW_ENVis built in.LOG_LEVELfalls back toinfo.DB_PASSWORDis a secret:PRODUCTION_DB_PASSWORDwhen deploying production,STAGING_DB_PASSWORDfor staging.
Names
| Name | Rule | Example |
|---|---|---|
app | Lowercase letters, digits and hyphens, starting with a letter, at most 24 characters | shop, api-gateway |
| Environment | Lowercase letters and digits, starting with a letter, at most 12 characters: no hyphen or underscore | production, staging2 |
| Variable and secret keys | Uppercase letters, digits and underscores, starting with a letter | DB_PASSWORD |
On the server, an environment of an app is the Compose project <app>-<env> (shop-production), and its containers, networks and volumes are named after it.
Environments
environments lists where the app is deployed. It needs at least one. Each environment is one server, with its own values.
dockflow deploy production
dockflow deploy pr # a unique prefix of an environment's name
dockflow deploy # when dockflow.yml has a single environmentIn CI, deploy can also take the environment from the git tag. 1.2.0 selects the environment whose ci.tag_suffix is empty (production by default). 1.2.0-staging selects the one whose suffix is staging. See CI.
The server of an environment comes from the <ENV>_CONNECTION secret, or from server: in the environment. See the Quickstart for both.
Some keys exist only inside an environment:
| Key | Description |
|---|---|
server | The server, when no connection string is used: host, port, user, host_key, name |
compose | The Compose files of this environment, replacing the top-level list |
profiles | The Compose profiles to enable |
vars | The environment's values |
deploy.timeout | Replaces the computed health timeout of every service, and bounds jobs |
ci.tag_suffix | The tag suffix that selects this environment in CI |
Values
Every ${NAME} is looked up in three places, in this order. The first match wins.
1. Built-in variables
| Variable | Value |
|---|---|
DOCKFLOW_APP | The app name |
DOCKFLOW_ENV | The environment name |
DOCKFLOW_STACK | <app>-<env>, the Compose project name |
DOCKFLOW_VERSION | The release's version label |
DOCKFLOW_GIT_SHA | The full commit |
DOCKFLOW_GIT_BRANCH | The branch |
DOCKFLOW_SERVER | The server's name (main unless set) |
2. Secrets
For the environment production, the key X is looked up as PRODUCTION_X, then as X, in each source in turn:
- The process environment and
DOCKFLOW_SECRETS. From the process environment, only variables prefixed with the environment's name are read (PRODUCTION_DB_PASSWORD).PATH,HOMEand the rest are never used.DOCKFLOW_SECRETSholds a JSON object of keys and values, prefixed or not. In GitHub Actions,DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}passes every secret of the repository at once. .env.dockflow, at the project root, for deploys from your machine.- The environment's SOPS file,
secrets/<env>.sops.yaml(orsecrets_file), encrypted with age and kept in git. See Secrets in a SOPS file.
PRODUCTION_CONNECTION=eyJ2IjoyLCJzZXJ2ZXJzIjpb…
PRODUCTION_DB_PASSWORD=correct-horse-battery-staple
STAGING_DB_PASSWORD="a value with spaces"The file has one KEY=value per line:
- Lines starting with
#are comments, and anexportprefix is accepted. - Values are taken literally. There is no variable expansion, so a password containing
$is safe. - Quotes around a value are removed, and double-quoted values understand
\n.
.env.dockflow holds secrets and, often, the connection string, which gives root on the server. Add it to .gitignore.
To hand a secret to a container as a file rather than an environment variable, use Compose secrets:: see Secrets and configs.
3. The environment's vars
version: 1
app: shop
environments:
production:
vars:
DOMAIN: shop.example.com
WORKERS: "4"Use vars for values that are not secret and differ per environment. They are taken literally: a vars value is not interpolated itself.
Missing values
A variable with no value and no default is an error, in every syntax. validate, plan and deploy stop with exit code 3, and say which variable is missing where. Nothing is ever deployed with an empty value by accident.
To make a variable optional, give it a default: ${LOG_LEVEL:-info}, or ${SENTRY_DSN:-} for an empty one.
Only what the environment uses is interpolated. The sections of other environments, and the top-level keys the environment overrides, are left alone. So a variable used only under environments.staging does not need a value when you deploy production.
Two syntaxes
Each kind of file keeps its own syntax:
| Where | Syntax |
|---|---|
Compose files, and the string values of dockflow.yml | Compose interpolation: ${VAR}, ${VAR:-default}, ${VAR:?message}, and $$ for a literal $ |
Files listed under templates: | Jinja: {{ VAR }}, {{ VAR | default('x') }}, {% if %}, {% for %}, {% raw %} |
| Hook commands | None: values arrive as environment variables |
Compose interpolation is Compose's own, so a Compose file stays valid for docker compose up. In dockflow.yml, it applies to string values, except version, app, the names of environments, mapping keys and vars.
Jinja is for other files full of $ that Compose interpolation would take for variables, such as an nginx configuration or a shell script.
- Only the files listed under
templates:are rendered. - They are rendered in a temporary copy of the project, before builds and uploads. Your source tree is never modified.
- The same names are top-level variables:
{{ DOMAIN }},{{ DOCKFLOW_ENV }}. An undefined variable is an error here too.
version: 1
app: shop
templates:
- src: config/app.conf.tpl
dest: config/app.conf
hooks:
before_build:
- run: ./scripts/generate-keys.sh
secrets: [JWT_PRIVATE_KEY]
notify:
- url: https://hooks.example.com/dockflow
secret: ${WEBHOOK_SECRET}
environments:
production:
vars:
DOMAIN: shop.example.com
staging:
vars:
DOMAIN: staging.shop.example.com
hooks: {}{# config/app.conf.tpl #}
server_name {{ DOMAIN }};
{% if DOCKFLOW_ENV == "production" %}
log_level warn;
{% else %}
log_level debug;
{% endif %}Hook commands are never interpolated. A hook gets the built-in variables and the environment's vars as environment variables, plus the secrets it lists under secrets:. Above, generate-keys.sh reads $JWT_PRIVATE_KEY. See Templates and Hooks.
Per-environment overrides
Every top-level key except version, app, compose and environments can also be set inside an environment. There it replaces the top-level value for that environment. Values are not merged: an environment that sets hooks has only its own hooks.
| Key | Typical override |
|---|---|
on_failure | ignore on staging, rollback on production |
releases | Fewer releases kept on staging |
registry | Another registry, or other credentials |
build | Build on the server for one environment |
health | Public checks on each environment's own URL |
templates, hooks, uploads, notify | Different lists per environment; hooks: {} removes every hook |
In the example above, staging has no hooks, and keeps the top-level templates and notify.
compose is listed with the environment-only keys, but it works the same way: inside an environment, it replaces the top-level list.
The version label
Every release carries a version label, available as ${DOCKFLOW_VERSION}. It is --version on deploy, the git tag in CI, the short commit (with -dirty for uncommitted changes), or the time of the deploy outside git. See Releases and rollbacks.
A service whose spec contains ${DOCKFLOW_VERSION}, ${DOCKFLOW_GIT_SHA} or ${DOCKFLOW_GIT_BRANCH} changes on every commit, so it is restarted by every deploy, even when nothing else changed.
Masking secrets
Dockflow knows every secret value of the environment. It replaces each occurrence with *** in what it shows or stores: events and the --json stream, build output, hook and job output, error messages, and dockflow config show. The connection string's private key is masked the same way.
PRODUCTION_DB_PASSWORD=s3cret dockflow config show production"DATABASE_URL=postgres://shop:***@db:5432/shop",Two exceptions:
dockflow exec,runandsshconnect your terminal to a command you run yourself. Their output is the command's own, not masked.- A value interpolated into a container's environment is part of the container, as
docker inspectshows on the server. It is also part of the release record, readable by root only.
Secrets are never written to your project's files or passed on a command line.
See Security for the full model.
The switch and edges
The Caddy container Dockflow runs in front of every app, how routes and TLS work, and the edge modes that decide how traffic reaches it.
Deploying from CI
Deploy from GitHub Actions, GitLab CI, Jenkins, Buildkite or Bitbucket Pipelines, with the environment and version taken from the Git tag and secrets from CI variables.