Dockflow

Rollouts

How Dockflow replaces the containers of a service, the three strategies and how each service gets one, health gating, draining, timeouts and what happens on failure.

A rollout replaces the containers of one service with those of the new release. For each service, Dockflow:

  • picks a strategy from its Compose definition;
  • waits for the new containers to be healthy before they get traffic;
  • keeps the old containers until the whole deploy succeeds, so it can go back to them.
compose.yml
services:
  web:                     # routed: switch
    build: .
    deploy:
      replicas: 2
    healthcheck:
      test: ["CMD", "wget", "-q", "--spider", "http://127.0.0.1:3000/healthz"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 20s
    x-dockflow:
      routes:
        - domains: ["${DOMAIN}"]
          port: 3000
      rollout:
        drain: 60s
      health:
        http:
          path: /healthz
          status: 200-299

  worker:                  # not routed: start-first
    build: .
    command: ["./worker"]
    x-dockflow:
      health:
        grace: 20s

  mailer:                  # chosen in Compose: stop-first
    image: ghcr.io/acme/mailer:2.1
    deploy:
      update_config:
        order: stop-first

  db:                      # stateful, and a fixed host port: stop-first
    image: postgres:17
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    ports:
      - "127.0.0.1:5432:5432"
    volumes: [pgdata:/var/lib/postgresql/data]
    x-dockflow:
      stateful: true

volumes:
  pgdata:

dockflow list services shows the strategy each service gets:

SERVICE  IMAGE                    REPLICAS  STRATEGY     ROUTES
db       postgres:17              1         stop-first
mailer   ghcr.io/acme/mailer:2.1  1         stop-first
web      shop/web                 2         switch       shop.example.com/
worker   shop/worker              1         start-first

Strategies

StrategyStepsDowntime
switchStart the new containers next to the old ones, wait until they are healthy, move the route's traffic to them, let the old ones drain, stop themNone
start-firstStart the new containers, wait until they are healthy, move the routes if any, stop the old onesNone. Old and new versions run side by side for a moment: a worker must tolerate it
stop-firstStop the old containers (with their stop_grace_period), start the new ones, wait until they are healthyBrief: from the stop to the new container being healthy

Containers are named <app>-<env>-<service>-<release>-<n>, so the containers of a new release never collide with those of the previous one. Inside the app's networks, every container also answers to its service name. Other services keep reaching db or api by name, as with Compose.

How a service gets its strategy

The first that applies:

  1. x-dockflow.rollout.strategy, if set;
  2. Compose's deploy.update_config.order (start-first or stop-first), if set;
  3. stop-first, if something below forces it;
  4. switch, if the service has routes (x-dockflow.routes);
  5. start-first otherwise.

These force stop-first, because two containers of the service cannot run at the same time:

The service…Why
is x-dockflow.stateful: trueA database must not have two instances writing to the same volume. It also runs a single replica
has container_nameTwo containers cannot share a name. It also runs a single replica
publishes a fixed host port (8080:80, 127.0.0.1:5432:5432)Two containers cannot bind the same port. A port published without a host port (- "80") does not count
uses network_mode: hostIts ports are the host's. It cannot have routes either

Asking for another strategy on such a service is an error, as is switch on a service without routes:

error: service game: strategy start-first is impossible because it publishes fixed host ports; use stop-first

A web service behind the switch does not need to publish ports at all: the switch reaches it on the app's network. Removing ports: from a routed service lets it use switch and deploy with no downtime.

Example: a stateful service on a fixed public port

A game server keeps its world on a volume and is reached on a fixed port, without the switch. It gets stop-first. The port filter opens its port once firewall.public_ports lists it:

compose.yml
services:
  game:
    image: itzg/minecraft-server
    environment:
      EULA: "TRUE"
    ports:
      - "25565:25565"
    volumes: [world:/data]
    stop_grace_period: 60s
    x-dockflow:
      stateful: true
volumes:
  world: {}
dockflow.yml
firewall:
  public_ports:
    - { port: 25565, protocol: tcp }

Each deploy stops the old container, giving it stop_grace_period to save, then starts the new one on the same volume. The game server is down from the stop until the new container is healthy.

Without the firewall entry, the port stays published but only the server and its containers reach it, and the plan warns about it. To back the world up while it runs, see backups.

Order

Within a deploy, services are created or updated one after the other, in depends_on order, then by name. Each service is healthy before the next one starts, so a dependency is up before its dependents, as with docker compose up.

before jobs run before the services, after jobs after them. See Jobs.

Health gating

A new container gets traffic, and the rollout moves on, only once it is healthy.

Docker's HEALTHCHECK is the main signal: the healthcheck of the Compose file, or the one built into the image. The container is ready when Docker reports it healthy. It fails the rollout as soon as Docker reports it unhealthy.

Without a healthcheck, a container counts as healthy once it has been running for x-dockflow.health.grace, 10 seconds by default. Running is not the same as ready to serve, so validate and plan warn about a routed service without a healthcheck or an internal HTTP check.

Whatever the healthcheck, the rollout fails when a new container:

  • exits;
  • is restarted three times by Docker (a crash loop);
  • is not healthy within its timeout.

Internal HTTP check

x-dockflow.health.http adds an HTTP check that the agent runs against each new container. It runs from inside the app's network, once the container is healthy and before any traffic moves to it. Use it for images without a HEALTHCHECK, or without curl or wget to write one.

KeyTypeDefaultDescription
pathstring/The path requested
portintegerThe first route's portThe container port. Required for a service without routes
statusstring200-399An accepted status, 200, or a range, 200-299
timeoutduration5sThe timeout of one request
retriesinteger10How many requests before the check fails
intervalduration3sThe time between two requests

A failed internal check follows on_failure. Public checks, on full URLs through DNS and TLS after the switch, are health.endpoints in dockflow.yml. Both are covered in Health checks.

Timeouts

Each service has a timeout for its new containers to become healthy, computed from its healthcheck:

  • with a healthcheck: start_period + (interval + timeout) × (retries + 1) + 30s, and at least 60 seconds, with Docker's defaults for the values not set (interval and timeout 30 seconds, retries 3);
  • without one: the grace period plus 60 seconds.

The web service above gets 20s + (5s + 3s) × 6 + 30s = 98 seconds.

deploy.timeout on an environment replaces the computed timeout of every service, and also bounds each job, which otherwise gets 30 minutes:

dockflow.yml
version: 1
app: shop

environments:
  production:
    vars:
      DOMAIN: shop.example.com
    deploy:
      timeout: 10m

Drain

With the switch strategy, traffic moves to the new containers in one step: the next request goes to them.

Requests already in flight on the old containers are not cut. The old containers keep running for the drain time, x-dockflow.rollout.drain, 30 seconds by default. Then they are stopped with their stop_grace_period. Connections still open at that point, such as WebSockets, are closed.

The drain starts when the service switches. When several services switch, the run waits once for the longest drain, at the end, not one drain per service.

compose.yml
services:
  web:
    image: nginx:1.27-alpine
    x-dockflow:
      routes:
        - domains: [shop.example.com]
          port: 80
      rollout:
        drain: 5s        # short requests only

start-first and stop-first have no drain. Old containers are stopped with their stop_grace_period (10 seconds by default, as in Compose), and they handle SIGTERM as they would with docker compose up.

What happens on failure

Old containers are stopped, never removed, until the whole run commits. When anything fails before that, the agent:

  1. starts the old containers again if needed, and waits until they are healthy;
  2. moves the routes back to them: for a switch service, traffic is back on the old containers in about a second;
  3. removes the new containers.

The release ends rolled_back, and the command exits with code 8. If the agent itself dies, the watchdog does the same within seconds (Interrupted runs).

A failure of the rollout itself always rolls back: a container that exits, crash-loops, stays unhealthy or times out, or a failed job. A failed HTTP check follows on_failure, which can also leave the release in place or ignore the failure. Releases and rollbacks lists everything that counts as a failure.

Unchanged services are left alone

The plan compares each service's spec (image digest, environment, ports, mounts, networks, healthcheck, replicas, routes…) with the spec of the containers running on the server. A service whose spec did not change is listed as unchanged and keeps its containers. It is not restarted, even if other services of the app are.

Interpolating ${DOCKFLOW_VERSION} or ${DOCKFLOW_GIT_SHA} into a service changes its spec on every commit, so every deploy restarts it. See the version label.

Services without a restart policy (nor deploy.restart_policy) get unless-stopped, so they come back after a crash or a reboot of the server.

dockflow restart restarts services with their own strategy, so routed services restart without downtime.

Edit on GitHub

On this page