Dockflow

Migrating from the earlier Dockflow

Move a project and a server from the earlier Dockflow, built on Docker Swarm, to this one, by hand.

This page moves a project from the earlier Dockflow to this one. The earlier Dockflow is the TypeScript tool built on Docker Swarm, with releases up to 3.1.1; this page calls it 1.x.

This version is a rewrite and shares nothing with 1.x: not the configuration format, not the CLI, not what it keeps on the server. It never reads a 1.x file or server, and has no migrate command. You move a project by hand, with the table below.

Do not deploy this Dockflow to a server that 1.x still manages. 1.x's Traefik holds ports 80 and 443, which the switch needs. This version does not know about 1.x's Swarm stacks and port filter.

The server

The simplest path is a fresh server, prepared with dockflow server init as in the quickstart. Keep the old server running until the new one serves the app, then move the DNS.

To reuse the 1.x server, remove 1.x from it first:

  1. Copy what you want to keep out of /var/lib/dockflow. This version writes its own state there and does not read 1.x's.
  2. Remove the Swarm stacks (docker stack rm), Traefik, and the port filter rules and units.
  3. Leave the swarm: docker swarm leave --force.
  4. Run dockflow server init as for a new server.

If a Swarm stack left behind a network with the name your app uses, the plan refuses it until you remove it.

The data in your named volumes stays where it is, but the volume names may change: this version names volumes <app>-<env>_<volume>. To keep the data, either:

  • copy it across with a backup and dockflow backup restore, or with dockflow cp;
  • or declare the old volume with external: true and its name.

The project

1.xThis version
.dockflow/config.yml and servers.ymlOne dockflow.yml at the project root. An environment is one server.
accessories.yml, deployed as a separate Swarm stackA Compose file listed under compose:. Its services are part of the app's Compose project; mark databases and other services that must not run twice stateful.
{{ current.env.x }}${X} in Compose files and dockflow.yml; {{ X }} in templates. Keys are upper case.
{{ x | default('y') }}${X:-y} in Compose files; unchanged in templates. An undefined variable is an error, never an empty string: write ${X:-} for an optional one.
{{ env }}, {{ project_name }}${DOCKFLOW_ENV}, ${DOCKFLOW_APP}; {{ DOCKFLOW_ENV }} and {{ DOCKFLOW_APP }} in templates. See built-in variables.
The nginx plugin and its vhost proxying to a published portx-dockflow.routes on the service. Either the switch is the TLS edge and gets the certificates, or the host's nginx stays in front of it: server init --edge external and the nginx plugin. See behind nginx.
<ENV>_<SERVER>_CONNECTIONThe output of dockflow server connection (or of server init), stored as <ENV>_CONNECTION: PRODUCTION_CONNECTION.
health_checks.endpoints with remote: truex-dockflow.health.http on the service.
health_checks.endpoints with a public URLhealth.endpoints in dockflow.yml.
hooks.pre-deploy running docker exechooks.before_deploy with an exec entry and its service.
Secrets in CIUnchanged: DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}, the PRODUCTION_* variables and .env.dockflow work as before. Per-server prefixes become per-environment ones: PRODUCTION_MANAGER_X becomes PRODUCTION_X. See secrets as CI variables.
build.dockerfile relative to the Compose file's folder (context: ../.. with dockerfile: Dockerfile.app in .dockflow/docker/)Relative to the build context, as Compose defines it: dockerfile: .dockflow/docker/Dockerfile.app.
deploy failing on a held lock in CI, worked around with a concurrency groupdeploy --wait-lock 10m.

There is no Swarm, so a few Compose keys that only Swarm used change meaning:

  • deploy.placement, deploy.endpoint_mode and deploy.rollback_config are ignored with a warning;
  • deploy.mode: global runs one container;
  • external secrets and configs are refused: give them a file, environment or content source.

The Compose reference lists every key.

Step by step

  1. Install this Dockflow next to, or instead of, 1.x: see installation. Both binaries are called dockflow, so check that the one on your PATH, and in CI, is this one.
  2. Write dockflow.yml from the old configuration with the table above, and move what accessories.yml deployed into a Compose file. dockflow init writes a starting dockflow.yml when the directory has none.
  3. Run dockflow validate production until it passes: it checks the files and every variable without a server.
  4. Prepare the server and store its connection string as PRODUCTION_CONNECTION in your CI (and in .env.dockflow to deploy by hand).
  5. Run dockflow plan production, then dockflow deploy production.
  6. Replace the CI workflow: dockflow init --ci github (or gitlab) writes one, and the CI guide has the others.
  7. Once the app runs and its data is in place, move the DNS, and retire the old server or what remains of 1.x on it.

A complete example

A backend with Postgres, which 1.x deployed as an app and an accessory. Both services also join a network shared with another app:

compose.yml
services:
  backend:
    build: .
    environment:
      QUARKUS_DATASOURCE_JDBC_URL: jdbc:postgresql://postgres:5432/app
      QUARKUS_DATASOURCE_USERNAME: app
      QUARKUS_DATASOURCE_PASSWORD: ${DB_PASSWORD}
      PAYMENT_API_KEY: ${PAYMENT_API_KEY}
    networks: [default, shared]
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8081/q/health"]
      start_period: 40s
    x-dockflow:
      routes:
        - domains: [api.example.com]
          port: 8081
          max_body: 25MB

  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: app
    volumes: [postgres_data:/var/lib/postgresql/data]
    networks:
      default: {}
      shared: { aliases: [backend-db] }
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app"]
    x-dockflow:
      stateful: true
      backup: { type: postgres, keep: { daily: 14, weekly: 8 } }

networks:
  shared: { external: true }   # dockflow server network create shared

volumes:
  postgres_data: {}
dockflow.yml
version: 1
app: backend
hooks:
  before_build:
    - script: .dockflow/hooks/pre-build.sh
      secrets: [JWT_PRIVATE_KEY]
environments:
  production: {}       # the server in PRODUCTION_CONNECTION, the secrets in CI

See several apps on one server for the shared network, and backups for the database.

Edit on GitHub

On this page