Deploying from CI

The same dockflow binary runs on your laptop and on a CI runner. A typical setup builds the images on every branch and deploys when you push a tag.

In CI, Dockflow recognises the provider and reads the tag, branch and commit of the run. It takes its secrets from CI variables.

# .github/workflows/deploy.yml (the deploy job)
- run: curl -fsSL https://raw.githubusercontent.com/Shawiizz/dockflow/main/install.sh | sh -s -- --version v1.0.0-alpha.6
- run: dockflow deploy --wait-lock 10m
  env:
    DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

Pushing the tag 1.2.0 deploys version 1.2.0 to production. Pushing 1.2.0-staging deploys version 1.2.0 to staging. Complete workflows for each provider are at the end of this page.

What Dockflow reads from the CI providerCopy Anchor Link

Dockflow checks the environment variables of the run in this order, and uses the first provider that matches:

What each value is used for:

• The tag selects the environment and gives the release its version label. See below.
• The branch and the commit fill DOCKFLOW_GIT_BRANCH and DOCKFLOW_GIT_SHA when the checkout cannot tell (CI checkouts are often detached or shallow). When git answers, its values win. Without a tag, the version label is the short commit.
• The actor and the provider are recorded as the release's author, which dockflow releases shows. They also name the holder of the environment's lock while the run lasts. On Jenkins, BUILD_USER_ID comes from the Build User Vars plugin; without it, the author is the runner's system user.

Without a terminal, Dockflow never prompts, so deploy applies its plan without --yes.

Tags select the environment and the versionCopy Anchor Link

In a run started by a tag, dockflow deploy without an environment splits the tag at its last hyphen:

A suffix that names no environment is an error (exit code 2): Dockflow never guesses.

The default suffix is empty for production, and the environment's own name for every other environment. Set ci.tag_suffix to change it:

version: 1
app: shop
environments:
  production: {}        # tags 1.2.0
  staging: {}           # tags 1.2.0-staging
  preview:
    ci:
      tag_suffix: rc    # tags 1.2.0-rc

A pre-release tag with a hyphen, such as 1.2.0-rc.1, is read the same way: its suffix is rc.1. When your tags don't follow this pattern, name the environment on the command line.

An environment named on the command line always wins over the tag. dockflow deploy staging in a tag build deploys to staging, and the version label still comes from the tag. --version <label> replaces the version label.

Secrets as CI variablesCopy Anchor Link

Every secret of an environment is a CI variable prefixed with the environment's name in upper case. For production:

Give the connection string as this variable, not as --connection "$PRODUCTION_CONNECTION": an argument is visible in the process list to every user of the runner, which matters on shared or self-hosted runners. When a step must pass it explicitly, use --connection @<file> or --connection - with the string on standard input.

Among the process variables, Dockflow reads only those that start with the environment's prefix. A CI variable named DB_PASSWORD alone is ignored, so a variable meant for staging never reaches production by accident.

You can also pass a set of secrets as one JSON object in DOCKFLOW_SECRETS. On GitHub Actions, this gives Dockflow every secret of the repository:

env:
  DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

GitHub may hold the first run of a workflow that hands every secret to a job this way, until someone approves it on the run's page in the Actions tab. To avoid that, name each secret instead: PRODUCTION_DB_PASSWORD: ${{ secrets.PRODUCTION_DB_PASSWORD }}.

Inside DOCKFLOW_SECRETS, the key X is looked up as PRODUCTION_X first, then as X. An unprefixed key there is a value shared by every environment.

Process variables and DOCKFLOW_SECRETS come before .env.dockflow, the file you keep out of Git for local deploys. Secret values are masked as *** in everything Dockflow prints or stores: events, build and hook output, errors. See Secrets for the rest.

Overlapping runs: --wait-lockCopy Anchor Link

In CI, two runs often overlap, such as a push to main and a tag pushed right after. Let the second one wait:

dockflow deploy --wait-lock 10m

Each environment has one lock. Without --wait-lock, a run that starts while another holds the lock fails at once with exit code 5. With it, the run waits up to that long for the other one to finish, and emits a lock.waiting event that names the holder.

A lock taken by hand with dockflow lock acquire (a maintenance window) is never waited for. The deploy fails with exit code 5 unless you pass --force.

Building on every branchCopy Anchor Link

dockflow build builds the images of the services with build:, without deploying. It needs no server: it builds for the runner's platform (--platform changes it) and checks that every image builds.

dockflow build production

In CI, dockflow build takes the environment from the tag, as deploy does. On a branch, name the environment when dockflow.yml has several. --only web,worker builds some services only. See Building images for every option.

A runner that builds needs Docker with buildx (or Podman, with build.engine: podman). A runner that only deploys services with public images needs no Docker at all: Dockflow reaches the server's Docker over SSH.

Build in one job, deploy in anotherCopy Anchor Link

With --push, dockflow build connects to the server and sends the images where a deploy would, built for the server's platform. A later job deploys them with --skip-build, without building again. Both jobs must check out the same commit: it gives the version label, and so the image tag.

dockflow build production --push        # job 1
dockflow deploy production --skip-build # job 2, same commit

To have a person review the plan before it is applied, save it in one job and apply it in another, after an approval step:

dockflow plan production --out plan.json   # job 1: keep plan.json as an artifact
dockflow deploy --plan plan.json           # job 2, once approved

The second job builds nothing and applies exactly the reviewed plan. If anything changed in between, on the server or in the project, it fails with plan.stale (exit code 6) and changes nothing.

Complete workflowsCopy Anchor Link

Each workflow builds on branches and deploys on tags, with the CLI pinned to a version. The GitHub and GitLab ones also back up on a schedule, as dockflow init --ci writes them.

Replace production with your environment. Add the secrets your Compose files use (PRODUCTION_DB_PASSWORD here) as CI variables.

name: Deploy

on:
  push:
    branches: ["**"]
    tags: ["*"]
  schedule:
    - cron: "17 3 * * *"
  workflow_dispatch:

jobs:
  build:
    if: github.event_name == 'push' && github.ref_type == 'branch'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dockflow
        run: curl -fsSL https://raw.githubusercontent.com/Shawiizz/dockflow/main/install.sh | sh -s -- --version v1.0.0-alpha.6
      - name: Build
        run: dockflow build production
        env:
          DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

  deploy:
    if: github.event_name == 'push' && github.ref_type == 'tag'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dockflow
        run: curl -fsSL https://raw.githubusercontent.com/Shawiizz/dockflow/main/install.sh | sh -s -- --version v1.0.0-alpha.6
      - name: Deploy
        run: dockflow deploy --wait-lock 10m
        env:
          DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

  # the services that declare x-dockflow.backup, then the server's state
  backup:
    if: github.event_name != 'push'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Install dockflow
        run: curl -fsSL https://raw.githubusercontent.com/Shawiizz/dockflow/main/install.sh | sh -s -- --version v1.0.0-alpha.6
      - name: Back up
        run: dockflow backup run production --wait-lock 10m && dockflow backup run production --server
        env:
          DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

The GitHub ActionCopy Anchor Link

On GitHub, the Shawiizz/dockflow action installs the CLI and runs it:

• On a pull request, it posts the plan of the change as a comment, so a reviewer sees what merging would change on the server. The comment is updated on each push.
• Elsewhere, it deploys.

name: Deploy

on:
  pull_request:
  push:
    tags: ["*"]

permissions:
  contents: read
  pull-requests: write # the plan's comment

jobs:
  dockflow:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Shawiizz/[email protected]
        with:
          env: production # the environment a pull request's plan is for
          command: ${{ github.event_name == 'push' && 'deploy --wait-lock 10m' || '' }}
        env:
          DOCKFLOW_SECRETS: ${{ toJSON(secrets) }}

The comment looks like this:

Dockflow plan: shop to productionCopy Anchor Link

Version 4f2c1a9, server main.

	Change	Target	Strategy	Why
~	service.update	web	start-first	image changed
	upload.apply	/etc/nginx/sites-enabled/shop.conf		changed

The action installs cosign first, then the CLI with install.sh --require-signature: the binary is checked against the release's SHA256SUMS, and SHA256SUMS against its Sigstore signature from the repository's release workflow. If either does not verify, the step fails and nothing runs. With version: none, it installs neither.

The plan needs the connection secret to read the server's state. GitHub gives no secrets to pull requests from forks: their plan fails, and the comment says so. A failed plan fails the job.

The action computes the plan again at deploy time. After a merge, the commit (and maybe the server) changed since the review, so the action deploys the merged code, not the reviewed plan. To apply exactly a reviewed plan, use dockflow plan --out and dockflow deploy --plan, as in Build in one job, deploy in another.

Deploying from a branchCopy Anchor Link

Tags are not required. To deploy every push to main to staging, name the environment:

dockflow deploy staging --wait-lock 10m

The version label is then the short commit, such as 3f9c2ab.

Exit codesCopy Anchor Link

Every failure has its own exit code, so a CI step can tell a refused configuration (3) from a held lock (5), a failed build (7) or a deploy that was rolled back (8). See Exit codes.