Dockflow

How it works

The pieces of a Dockflow deploy, from the CLI on your machine to the agent, the switch, the registry and the state on the server.

Dockflow is one binary with two roles: the CLI on your machine, and a short-lived agent on the server. One SSH connection links them and carries the Docker Engine API. Nothing else runs on the server's host: everything Dockflow installs there is a container.

your laptop or CI runner                     the server (Linux, Docker Engine 27+)
┌────────────────────────┐                   ┌────────────────────────────────────────────┐
│ dockflow (CLI)         │      SSH          │ Docker Engine (/var/run/docker.sock)       │
│  · reads dockflow.yml, │ ────────────────► │                                            │
│    Compose, secrets    │  Engine API over  │  dockflow-run-<id>   the agent, one per run│
│  · builds images       │  stream forwarding│  dockflow-switch     Caddy: routes, TLS    │
│  · pushes to registry  │  SSH tunnel to    │  dockflow-registry   127.0.0.1:5959        │
│  · renders events      │  127.0.0.1:5959   │  <app>-<env>-*       your app's containers │
└────────────────────────┘                   │                                            │
                                             │  /var/lib/dockflow   state: releases, locks│
                                             └────────────────────────────────────────────┘

The CLI

The dockflow CLI runs on your machine or a CI runner. It:

Everything that can be checked without a server is checked here. dockflow validate stops at this point.

For services with build:, the CLI builds the images and pushes them. It builds with buildx bake or Podman, or on the server with build.builder: remote. See Building images.

SSH and the Engine API

The CLI opens one SSH connection to the server. It verifies the server's host key against the pinned fingerprint or your known_hosts. Nothing is trusted on first use.

Through that connection, the CLI reaches the server's Docker socket with OpenSSH stream-local forwarding. It talks to Docker through the Engine API, the same API the docker CLI uses. If the server refuses stream forwarding, the CLI falls back to docker system dial-stdio over SSH.

No shell command runs on the server during a deploy. Why: see ADR-0005.

The CLI checks the Docker version when it connects: Docker Engine 27 or later is required.

The agent

Every operation that changes the server runs in an agent: a container named dockflow-run-<run-id>, created for one run and stopped when the run ends.

  1. If the server does not have the image dockflow/agent:<version>, the CLI creates it with the Engine API's image import, from the Linux binary of its own version. This happens once per version and per server. See The agent binary.
  2. The CLI creates the agent container. It mounts the Docker socket, /var/lib/dockflow and the switch's admin socket.
  3. The CLI writes one request on the agent's standard input: the desired state of the app, the secret values the run needs, and the files to upload. Then it closes stdin. Secrets never appear in the container's configuration, environment or logs.
  4. The agent applies the request through the local Docker socket, and writes its progress to stdout as events, which the CLI renders.

The run belongs to the agent, not to the CLI. If the CLI disconnects (the laptop sleeps, the network drops, the CI runner is killed), the agent goes on and finishes the run or rolls it back. dockflow attach follows a run in progress again, or replays a finished one.

To stop a run:

  • Ctrl+C asks the agent to stop at the next step and roll back.
  • A second Ctrl+C detaches the CLI. The agent finishes its rollback.
  • dockflow cancel does the same as Ctrl+C, from another terminal.

The CLI never kills the agent.

Finished agent containers are kept for dockflow attach and inspection. Each new run removes all but the 10 most recent.

Not every command changes the server:

  • Commands that only read the state files (status, releases, version <env>, lock status) run an agent too, because /var/lib/dockflow is readable by root only. That agent takes no lock and changes nothing.
  • Commands that need only Docker, such as logs and ps, call the Engine API directly.
  • exec and ssh connect your terminal to the container or the server.

The switch

Every server that routes traffic runs one switch, dockflow-switch. It is the official Caddy image, pinned by digest, installed by dockflow server init. The switch sits in front of every app on the server. It terminates TLS with the default edge, and sends each request to the containers of the release currently serving that route.

During a deploy, the agent moves traffic from the old containers to the new ones without reloading Caddy, so no connection is dropped. See The switch and edges.

The server-local registry

Built images need to reach the server's Docker daemon. By default they go through a registry that runs on the server, dockflow-registry, published on 127.0.0.1:5959 only:

  1. The CLI opens an SSH tunnel to the registry and pushes the image. Layers the registry already has are skipped.
  2. The agent pulls the image by digest into the daemon.

The registry is never reachable from outside the server, and needs no credentials. To use an external registry instead, set registry: in dockflow.yml.

Images used by the kept releases stay on the server, so a rollback never needs a download.

The state

Everything Dockflow knows about a server lives on that server, under /var/lib/dockflow, readable by root only and written only by agents:

  • the server's settings (edge, switch and registry images);
  • the routes of each app environment;
  • per app environment (<app>-<env>): the lock, the release records, the current and previous release, and the files of uploads.

There is no database, and nothing is stored on your machine. The details are in Server layout.

Containers also carry Dockflow's labels (dockflow.release=<seq>) and Compose's own labels (com.docker.compose.project, com.docker.compose.service). So Docker itself shows what is running, and docker compose -p <app>-<env> ps sees the app.

Losing the server loses the release history along with the apps. Dockflow does not replicate it anywhere.

Locks

Each app environment has a lock. A run that changes it (deploy, rollback, restart, stop, scale, destroy) holds the lock for its whole duration. The lock is a flock on a file under /var/lib/dockflow.

  • When the lock is held, another run fails at once with exit code 5, and says who holds it. For CI pipelines that may overlap, dockflow deploy --wait-lock 10m waits up to 10 minutes instead. A lock held by the watchdog (for a restart, or to recover a run that died) is always waited for, up to 3 minutes.
  • When the holder dies, the kernel releases the lock. There is no lease to expire and no stale lock to clear by hand. The watchdog finishes or rolls back the run that died (Interrupted runs).
  • Manual locks. For a maintenance window, dockflow lock acquire production -m "database migration" blocks every run until dockflow lock release. deploy --force deploys anyway.

Read-only commands take no lock. Different apps, and different environments of one app, deploy independently. Changes to the switch's configuration go one at a time, under a server-wide lock.

Plan, then apply

A deploy is computed before it is applied:

  1. Desired state. The CLI converts dockflow.yml, the environment's Compose files, variables, secrets and the digests of built images into the app the server should run.
  2. Actual state. The agent reads what the server runs: the app's containers, networks, volumes and routes, and its current release.
  3. Plan. Dockflow compares the two and produces an ordered list of actions, each with its reason: create networks and volumes, apply uploads, run before jobs, create or update services in depends_on order, remove services that left the Compose files, apply routes, run after jobs.
  4. Apply. On a terminal, deploy shows the plan and asks before applying it. The agent then applies exactly the plan you saw. If the server changed in between, it refuses with plan.stale.

dockflow plan prints the plan without applying it, and takes no lock.

  • Each service's spec has a hash. A service whose hash is unchanged keeps its containers and is listed as unchanged.
  • A plan with nothing to do creates no release and exits 0.
  • The plan always starts from what is actually running, so the next run repairs a run that was interrupted. See Releases and rollbacks.

Events and --json

Every operation reports its progress as a stream of events: the run started, the plan, each action, containers becoming healthy, the switch, the release, the outcome. The agent writes them, and the CLI renders them. Each event has a stable type, a level, a human-readable msg and typed data.

With --json, the CLI prints the events as they are, one JSON object per line (JSON Lines), for scripts and CI:

dockflow deploy production --json | jq -c 'select(.type == "service.healthy")'
{"v":1,"ts":"2026-10-26T14:03:42.118Z","run":"01jbx6k3m8w0v7c9q2r5t4y6z8","seq":17,"type":"service.healthy","level":"info","msg":"web: 2/2 healthy","data":{"service":"web","healthy":2,"desired":2}}

The last event of a run is always run.finished, with the outcome and the exit code. Read commands such as status and releases print a single JSON document instead. -v shows debug events in the human output.

No event carries a secret value: every occurrence of a secret in a message, a log line or an error is replaced by ***.

The types and their fields are in the events reference, and the exit codes in Exit codes.

Edit on GitHub

Migrating from the earlier Dockflow

Move a project and a server from the earlier Dockflow, built on Docker Swarm, to this one, by hand.

Releases and rollbacks

What a release records, how a failed deploy rolls back on its own, what dockflow rollback restores, and how an interrupted run is repaired.

On this page

The CLISSH and the Engine APIThe agentThe switchThe server-local registryThe stateLocksPlan, then applyEvents and --json