Airflow

Overview

Apache Airflow is an open-source platform for programmatically authoring, scheduling and monitoring workflows. This template deploys an Airflow 3.x stack using the CeleryExecutor: one webserver workload running the API server, scheduler, dag-processor and triggerer, Celery workers that execute tasks, Redis as the task broker and PostgreSQL as the metadata database. Workers run at a fixed replica count or, optionally, are scaled by KEDA on the queue length; DAGs are placed on a shared volume or delivered by an optional git-sync sidecar. The admin password, the JWT signing secret and the Fernet key are not template values. They come from a dictionary secret you create before installing, so they never pass through Helm. The web UI ships closed to the internet: it can trigger arbitrary code and decrypt every credential stored in an Airflow Connection, so you reach it over a port-forward until you choose to expose it.

This template deploys into an existing GVC; it does not create one. Every resource is pinned to the single location named by the location value, which must already be a location of that GVC. Other locations the GVC has are left alone: nothing Airflow-related starts in them. There is no multi-location mode, because the shared Airflow home provisions one volume per location and the database and broker are each a single volume.

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-airflow-webserver`	API server, scheduler, dag-processor and triggerer in one container (stateful, 1 replica); serves the UI and REST API on port `8080`	—
Workload	`RELEASE_NAME-airflow-celery-worker`	Celery workers executing DAG tasks (stateful); fixed `airflow.celeryWorker.replicas`, or scaled by KEDA	—
Workload	`RELEASE_NAME-airflow-postgres`	PostgreSQL metadata database (stateful, 1 replica, port `5432`)	—
Workload	`RELEASE_NAME-airflow-redis`	Redis Celery broker (stateful, 1 replica, port `6379`)	—
Volume set	`RELEASE_NAME-airflow-vs`	Shared Airflow home (`/opt/airflow`: DAGs and task logs) mounted by the webserver and every worker	—
Volume set	`RELEASE_NAME-airflow-postgres-vs`	`ext4` volume holding the metadata database	—
Volume set	`RELEASE_NAME-airflow-redis-vs`	`ext4` volume holding the broker’s data	—
Secret	`RELEASE_NAME-airflow-webserver-startup`	Opaque secret holding the webserver’s startup script	—
Identity	`RELEASE_NAME-airflow-identity`	Identity shared by the webserver, workers and database	—
Policy	`RELEASE_NAME-airflow-policy`	`reveal` on your database credentials secret, the startup script, your auth secret and, when set, your git token secret	—
Policy	`RELEASE_NAME-airflow-gvc-policy`	`view` on the one GVC you install into, so the webserver can check its location and KEDA settings at boot	—
Container	`git-sync` (sidecar on the webserver and worker workloads)	Clones a Git repository onto the shared volume and keeps it in sync	`gitSync.enabled: true`

Prerequisites

Two secrets must exist before you install, and the GVC must already have the location you configure. Secrets are org-level, so no GVC flag is involved.

Confirm the GVC has the location you will configure

location must name a location the GVC already has. Read the current list under spec.staticPlacement.locationLinks:

cpln gvc get GVC_NAME -o yaml

Set location to one of those names (the default is aws-us-east-1). To add a location, see the GVC reference — every workload in a GVC runs in every location it has, so add locations to a shared GVC deliberately.

Create the auth secret

A dictionary secret holding exactly three keys — jwtSecret, fernetKey and adminPassword:

cpln secret create-dictionary --name AIRFLOW_AUTH_SECRET_NAME \
  --entry jwtSecret="$(openssl rand -base64 48)" \
  --entry fernetKey="$(openssl rand -base64 32 | tr '+/' '-_')" \
  --entry adminPassword="$(openssl rand -base64 18)"

Set airflow.auth.secretName to this name.

Key	What it is
`jwtSecret`	Signs Airflow API access tokens — anyone holding it can mint one.
`fernetKey`	Encrypts the Connections and Variables your DAGs use: database passwords, cloud keys, API tokens. Must be 32 random bytes in URL-safe base64, which is what the command above produces. It cannot be rotated.
`adminPassword`	The web UI and API password for the `airflow.admin.username` login.

Read the secret back later

Pass -o yaml — a bare cpln secret reveal prints only a summary table, not the values:

cpln secret reveal AIRFLOW_AUTH_SECRET_NAME -o yaml

Create the database credentials secret

A dictionary secret holding username, password and database. The bundled PostgreSQL initializes from it, and the webserver and workers connect with it:

cpln secret create-dictionary --name my-airflow-db-credentials \
  --entry username=airflow \
  --entry password="$(openssl rand -hex 24)" \
  --entry database=airflow

Set its name in postgres.config.credentialsSecretName. Secret names are org-wide, so give each Airflow release its own secret. The three values are part of Airflow’s connection URLs, so use letters, digits, - and _ only. PostgreSQL reads the secret only when its data is first initialized — see Rotating the Database Password.

Optional: create a git token secret for a private DAG repository

Only when gitSync.enabled is true and the repository is private. An opaque secret with encoding plain whose payload is a personal access token; the sidecar sends it as the Git password:

printf '%s' "YOUR_GIT_TOKEN" | cpln secret create-opaque --name GIT_TOKEN_SECRET_NAME --encoding plain -f -

Set gitSync.auth.secretName to this name. Leave it empty for a public repository or when git-sync is off — setting it while git-sync is disabled is rejected at render.

Optional: enable KEDA on the GVC

Only when you set keda.enabled: true. KEDA is a GVC-level setting this template cannot turn on for you; see Keda. Export the GVC, set spec.keda.enabled: true in the exported file, apply it and read it back:

cpln gvc get GVC_NAME -o yaml-slim > gvc.yaml

cpln apply -f gvc.yaml

cpln gvc get GVC_NAME -o yaml

Always start from the exported file. cpln apply replaces a GVC’s spec rather than merging into it, so a hand-written file carrying only spec.keda removes the GVC’s locations.

Create the auth and database credentials secrets before installing, or the deployment wedges silently. A secretName pointing at a secret that does not exist installs “successfully” and then never starts. No container runs, so cpln logs returns zero lines. The one place the reason appears is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-airflow-webserver --gvc GVC_NAME -o yaml

The secret AIRFLOW_AUTH_SECRET_NAME no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed.

The command is get-deployments — plain cpln workload get has no versions key. Once the secret exists the workload recovers on its own, slowly; force a redeployment to skip the wait:

cpln workload force-redeployment RELEASE_NAME-airflow-webserver --gvc GVC_NAME

Generate the Fernet key once and keep it for the life of the install. It encrypts every Connection and Variable Airflow stores. A different key does not fail loudly — it makes all of them unreadable, and there is no re-encrypt path short of re-entering each one by hand. Back the secret up somewhere you will still have it later.

Installation

Install from the marketplace registry, naming the location your GVC has and the two secrets you created:

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/airflow \
  --version 2.1.0 --gvc GVC_NAME \
  --set location=aws-us-east-1 \
  --set airflow.auth.secretName=AIRFLOW_AUTH_SECRET_NAME \
  --set postgres.config.credentialsSecretName=my-airflow-db-credentials

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Location

location: aws-us-east-1 # the ONE location Airflow runs in; must already be a location of the GVC you install into

Every workload in the release is pinned here. GVC locations you do not name run nothing — their deployments read This workload location is deactivated because maxScale is set to 0., which is what a healthy install looks like, not a fault.

PostgreSQL Metadata Database

postgres:
  image: postgres:18
  resources:
    minCpu: 250m
    maxCpu: 500m
    minMemory: 512Mi
    maxMemory: 1024Mi
  config:
    # REQUIRED PREREQUISITE SECRET — CREATE IT BEFORE YOU INSTALL. If it does not
    # exist the deployment WEDGES silently and `cpln logs` returns nothing at all.
    # A `dictionary` secret holding exactly three keys: username, password and
    # database. Postgres reads them only when its data directory is first
    # initialized. Upgrading from 2.0.0: give it a NEW name (not
    # RELEASE-airflow-config) — see `Upgrading from 2.0.0` in the README.
    credentialsSecretName: my-airflow-db-credentials
  volumeset:
    capacity: 10 # initial capacity in GiB (minimum is 10)

The metadata database credentials come from your prerequisite dictionary secret (see Prerequisites), named by postgres.config.credentialsSecretName. PostgreSQL reads them only when the data directory is first initialized, so editing the secret afterwards does not change the database password — see Rotating the Database Password.

Redis Broker

redis:
  image: redis:7.4
  resources:
    cpu: 250m
    memory: 512Mi
  volumeset:
    capacity: 10 # initial capacity in GiB (minimum is 10)

airflow:
  webserver:
    image: apache/airflow:3.0.3
    resources:
      cpu: 2000m
      memory: 3Gi
  celeryWorker:
    image: controlplanecorporation/celery:v1
    replicas: 1 # fixed worker count; ignored when keda.enabled is true
    resources:
      cpu: 256m
      memory: 512Mi
  webPort: 8080 # port serving the Airflow UI and REST API

  auth:
    secretName: my-airflow-auth # REQUIRED prerequisite dictionary secret: jwtSecret, fernetKey, adminPassword
    jwtExpirationTime: 86400 # API token lifetime in seconds (Airflow's own default)

  admin:
    username: admin # its password is `adminPassword` in the secret above

  scheduler:
    dagDirListInterval: 10 # how often to rescan the DAG folder (seconds)
    minFileProcessInterval: 10 # minimum interval between processing a DAG file (seconds)

  celery:
    workerConcurrency: 1 # tasks each worker runs concurrently

The webserver runs the API server, scheduler, dag-processor and triggerer in one container pinned to a single replica, so airflow.celeryWorker.replicas (or KEDA) is the only horizontal scaling knob: you scale task execution, not scheduling. The admin account is provisioned by Airflow’s SimpleAuthManager, which has no SSO or LDAP support; the webserver rewrites its password file from the secret on every container start.

Storage

volumeset:
  airflow:
    capacity: 10 # shared Airflow home (DAGs + logs), GiB (minimum is 10)

The Airflow home is a shared filesystem mounted at /opt/airflow by the webserver and every Celery worker, so they read the same DAGs and write to the same log directory. The database and broker volumes are sized separately under postgres.volumeset.capacity and redis.volumeset.capacity.

Access

firewallConfig:
  inboundAllowCIDR: [] # [] = closed to the internet; add CIDRs (e.g. - 203.0.113.0/24) to expose the UI

internalAccess:
  type: same-gvc # options: same-gvc, same-org, workload-list
  workloads: [] # only with workload-list, e.g. - //gvc/GVC_NAME/workload/WORKLOAD_NAME

firewallConfig.inboundAllowCIDR applies to the webserver only. With the shipped [] the canonical endpoint returns 403 and the UI is reachable only over a port-forward; a change takes up to a few minutes to take effect. internalAccess controls which workloads inside the org may reach Airflow’s own workloads:

Type	Description
`same-gvc`	Allow access from all workloads in the same GVC
`same-org`	Allow access from all workloads in the org
`workload-list`	Allow access only from the workloads listed in `internalAccess.workloads`, plus this release’s own four workloads, which the template always adds

Under workload-list you only list your own clients: the template merges in the webserver, workers, database and broker so they can keep reaching each other, and allows the KEDA operator onto the broker when keda.enabled is true.

DAG Delivery With git-sync

gitSync:
  enabled: false
  repo: "" # git repository URL (e.g. https://github.com/YOUR_ORG/dags)
  branch: main
  period: 60s # how often to sync
  subPath: "" # optional subfolder within the repo containing DAGs
  auth:
    secretName: "" # OPTIONAL opaque secret (encoding plain) holding a personal access token; empty = public repo

When enabled, a git-sync sidecar runs on the webserver and worker workloads, clones the repository under /opt/airflow/dags-sync on the shared volume, and the DAGs folder on both tiers points at the synced checkout (plus gitSync.subPath when set). gitSync.repo and gitSync.branch are required when git-sync is on.

KEDA Autoscaling

keda:
  enabled: false # requires KEDA enabled on the GVC first — see Prerequisites
  minScale: 1 # minimum number of Celery workers; 0 = scale to zero when idle
  maxScale: 3 # maximum number of Celery workers
  scaleToZeroDelay: 300 # idle time before scaling to zero (seconds)
  listLength: 3 # Redis queue length that triggers a scale-up
  cooldownPeriod: 1 # cooldown between scaling events (seconds)
  initialCooldownPeriod: 1 # cooldown after startup before scaling begins (seconds)
  pollingInterval: 4 # how often KEDA queries Redis (seconds)

KEDA scales the Celery workers on the Redis queue length, including down to zero between runs. With it off, workers run at the fixed airflow.celeryWorker.replicas.

Enable KEDA on the GVC before you set keda.enabled: true. Getting the order wrong is accepted without any error: the worker workload is created, never autoscales, and sits at keda.minScale forever — at minScale: 0 that is zero workers, and every task queues with nothing reporting a problem. On a fresh install the webserver checks the GVC at boot and refuses to start with [airflow] FATAL: keda.enabled is true in your values, but GVC ... has spec.keda.enabled = false; on an established install the same condition is logged as a warning and Airflow keeps running. Enabling KEDA on the GVC clears the refusal on the webserver’s next restart. See Prerequisites for the commands.

Connecting

What	Address	Notes
UI and REST API (closed default)	`cpln port-forward RELEASE_NAME-airflow-webserver 8080:8080 --gvc GVC_NAME`, then `http://localhost:8080`	The tunnel works while the canonical endpoint returns `403`
UI and REST API (public)	`status.canonicalEndpoint` from `cpln workload get RELEASE_NAME-airflow-webserver --gvc GVC_NAME -o yaml`	Returns `403` until you add a CIDR to `firewallConfig.inboundAllowCIDR`
Metadata database (in-GVC)	`RELEASE_NAME-airflow-postgres.GVC_NAME.cpln.local:5432`	Credentials are the `username`, `password` and `database` keys of the secret named by `postgres.config.credentialsSecretName`
Redis broker (in-GVC)	`RELEASE_NAME-airflow-redis.GVC_NAME.cpln.local:6379`	No authentication
Login	`airflow.admin.username` and the `adminPassword` key of your auth secret	`cpln secret reveal AIRFLOW_AUTH_SECRET_NAME -o yaml`

Verify the webserver is serving through the tunnel:

cpln port-forward RELEASE_NAME-airflow-webserver 8080:8080 --gvc GVC_NAME

curl http://localhost:8080/api/v2/monitor/health

First Run

Install with the UI closed

Keep the shipped firewallConfig.inboundAllowCIDR: [].

Tunnel to the UI and sign in

cpln port-forward RELEASE_NAME-airflow-webserver 8080:8080 --gvc GVC_NAME

Open http://localhost:8080 and log in with airflow.admin.username and the adminPassword from your auth secret. There is no setup wizard; the admin account already exists.

Expose it only if you need to

Run a Helm upgrade with your own CIDR ranges in firewallConfig.inboundAllowCIDR, then allow a few minutes for the firewall change to take effect before testing the canonical endpoint.

API Access

Airflow 3.x issues JWT access tokens signed with your jwtSecret. Request one with the admin credentials and pass it on the v2 API:

curl -X POST http://localhost:8080/auth/token \
  -H "Content-Type: application/json" \
  -d '{"username": "admin", "password": "ADMIN_PASSWORD"}'

curl http://localhost:8080/api/v2/dags \
  -H "Authorization: Bearer ACCESS_TOKEN"

Tokens expire after airflow.auth.jwtExpirationTime seconds.

Operations

Deploying DAGs

With git-sync off, DAG files go in /opt/airflow/dags on the shared volume; the dag-processor picks up a new file within airflow.scheduler.dagDirListInterval seconds. Copy a local folder in through the webserver container (--stdin is required, and the archive is base64-encoded because cpln workload exec is not binary-safe):

tar -cf - -C ./dags . | base64 | cpln workload exec RELEASE_NAME-airflow-webserver --gvc GVC_NAME \
  --container airflow --stdin --quiet -- sh -c 'base64 -d | tar -xf - -C /opt/airflow/dags'

With gitSync.enabled: true, commit DAGs to the configured repository and branch instead; the sidecar syncs them every gitSync.period.

Backing Up

The chart schedules no backups and no volume set snapshots. The metadata database — every Connection, Variable, DAG run and task history — is the one thing to back up; the broker is transient and DAGs live in your repository or on the shared volume. Two options: Take a snapshot of the database volume set:

cpln volumeset snapshot create RELEASE_NAME-airflow-postgres-vs --gvc GVC_NAME --snapshot-name airflow-before-upgrade

Or dump the database with pg_dump, which exists in the postgres:18 image. The dump is base64-encoded in transit because cpln workload exec corrupts binary output:

cpln workload exec RELEASE_NAME-airflow-postgres --gvc GVC_NAME --container postgresql --quiet -- \
  sh -c 'pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc | base64' | base64 -d > airflow.dump

Back up the auth secret too: the Fernet key inside it is the only way to read the dump’s Connections and Variables.

Restoring a Backup

This restore path has not been exercised end to end against this template. The commands are built from the chart’s configuration (container postgresql, credentials in POSTGRES_USER / POSTGRES_DB) and the tools present in the pinned images, but verify them on a throwaway release before relying on them.

Restore a pg_dump archive into the metadata database, then restart the webserver and workers so they re-read it. Stop triggering DAG runs while the restore runs:

base64 < airflow.dump | cpln workload exec RELEASE_NAME-airflow-postgres --gvc GVC_NAME \
  --container postgresql --stdin --quiet -- \
  sh -c 'base64 -d | pg_restore -U "$POSTGRES_USER" -d "$POSTGRES_DB" --clean --if-exists'

cpln workload force-redeployment RELEASE_NAME-airflow-webserver --gvc GVC_NAME

cpln workload force-redeployment RELEASE_NAME-airflow-celery-worker --gvc GVC_NAME

The restored Connections and Variables are readable only under the Fernet key that encrypted them, so the release must point at the same auth secret the backup was taken under. Restoring a volume set snapshot in place (cpln volumeset snapshot restore) is not verified for this template.

Rotating the Admin Password

The webserver rewrites its SimpleAuthManager password file from the secret on every container start, so changing adminPassword and restarting the webserver is enough. A secret change is not picked up by a running replica, so redeploy after changing it. Export the secret, edit only adminPassword (leave fernetKey untouched), re-apply it and confirm the change:

cpln secret reveal AIRFLOW_AUTH_SECRET_NAME -o yaml > secret.yaml
# edit only the key you are rotating in secret.yaml
cpln apply -f secret.yaml
cpln secret reveal AIRFLOW_AUTH_SECRET_NAME -o yaml

Then restart the webserver:

cpln workload force-redeployment RELEASE_NAME-airflow-webserver --gvc GVC_NAME

jwtSecret can be rotated the same way — redeploy both the webserver and RELEASE_NAME-airflow-celery-worker, and every outstanding API token becomes invalid. Never change fernetKey.

Rotating the Database Password

PostgreSQL keeps the password it was initialized with, and a running workload keeps the values it started with, so a rotation takes three steps in order:

Change it in the database (DB_USERNAME and DB_NAME are the secret’s username and database):

cpln workload exec RELEASE_NAME-airflow-postgres --gvc GVC_NAME --container postgresql -- psql -U DB_USERNAME -d DB_NAME -c "ALTER ROLE \"DB_USERNAME\" PASSWORD 'NEW_PASSWORD'"

Update the password key of the secret: cpln secret edit SECRET_NAME.
Right away, redeploy the webserver and the workers, and wait until get-deployments shows a higher version number for each, alone and ready. Do not redeploy the database: PostgreSQL reads the secret only when it is first initialized, so restarting it only adds downtime.
cpln workload force-redeployment RELEASE_NAME-airflow-webserver --gvc GVC_NAME cpln workload force-redeployment RELEASE_NAME-airflow-celery-worker --gvc GVC_NAME

Connections Airflow already has open keep working after ALTER ROLE; new ones fail until the redeploy finishes, so rotate in a quiet window. Without step 3 the old password stays in use indefinitely: updating a secret does not restart the workloads that read it.

Upgrading from 1.4.x or Earlier

Template version 1.5.0 removed the credentials that 1.4.x took as plain Helm values. Each removed key is rejected at render with a message naming its replacement, so an old values file fails loudly instead of being silently ignored. There is no in-place path from 1.4.x to 2.0.0 — follow Migrating from 1.x, carrying only the new keys:

1.4.x and earlier	1.5.0 and later
`airflow.auth.jwtSecret`	`jwtSecret` key in the prerequisite auth secret
`airflow.auth.fernetKey`	`fernetKey` key in the same secret
`airflow.admin.password`	`adminPassword` key in the same secret
`gitSync.auth.token`	`gitSync.auth.secretName` — an opaque secret you create
`airflow.auth.jwtExpirationDelta`	`airflow.auth.jwtExpirationTime` (seconds)
`airflow.auth.jwtRefreshThreshold`	Removed — Airflow 3.x has no equivalent
`firewallConfig.inboundAllowCIDR` defaulting to `0.0.0.0/0`	`firewallConfig.inboundAllowCIDR: []`, closed

If your 1.4.x install ran on the shipped CHANGE_ME Fernet key, every stored Connection and Variable is encrypted under a value published in a public repository. The key cannot be rotated, so there is no in-place fix: export what you need, install with a freshly generated key, re-enter the Connections and Variables by hand, and rotate every credential they held at its source.

Migrating from 1.x

Template versions through 1.5.0 created their own GVC. Version 2.0.0 deploys into an existing GVC and does not declare one — and Helm deletes what a chart stops declaring.

Never run a plain helm upgrade on a 1.x release onto 2.0.0. The upgrade drops the GVC from the manifest and deletes it, with every workload, volume set and identity inside — your metadata database and your DAGs — while reporting success. The chart refuses to render when your values still carry the gvc key, and the refusal leaves the running release untouched:

Error: execution error at (airflow/templates/workload-webserver.yaml:1:4): airflow 2.0.0: the `gvc`
values key was REMOVED. This chart no longer creates a GVC — it deploys into the GVC you install
into, and `gvc.locations` became the single top-level `location`. DO NOT `helm upgrade` a 1.x
release onto 2.0.0 ...

An upgrade run with no values at all has no gvc key to catch and proceeds, so the rule matters more than the guard. To keep the GVC, follow Upgrading In Place from 1.5.0. The alternative is a new release against an existing GVC, as below.

Pick the GVC and location for the new release

Choose a GVC that already has the location you will set as location, and enable KEDA on it first if you plan to use it. See Prerequisites.

Install 2.0.0 as a new release against the same auth secret

Use a different release name — policy and secret names are org-wide, so a same-named release collides with the 1.x one even in another GVC. Point airflow.auth.secretName at the same auth secret the 1.x release used: the fernetKey must be identical, or every restored Connection and Variable becomes unreadable. With 2.1.0 or later, also point postgres.config.credentialsSecretName at a new database credentials secret (see Prerequisites).

Move the metadata database

Dump from the old release’s database workload, in the GVC the 1.x chart created, and restore into the new one. Both go through base64 because cpln workload exec is not binary-safe:

cpln workload exec OLD_RELEASE_NAME-airflow-postgres --gvc OLD_GVC_NAME --container postgresql --quiet -- \
  sh -c 'pg_dump -U "$POSTGRES_USER" -d "$POSTGRES_DB" -Fc | base64' | base64 -d > airflow.dump

base64 < airflow.dump | cpln workload exec RELEASE_NAME-airflow-postgres --gvc GVC_NAME \
  --container postgresql --stdin --quiet -- \
  sh -c 'base64 -d | pg_restore -U "$POSTGRES_USER" -d "$POSTGRES_DB" --clean --if-exists --no-owner --no-acl'

--no-owner --no-acl let the restore succeed when the new database credentials secret uses a different username from the old release. This data move has not been exercised end to end; see Restoring a Backup.

Move the DAGs

With gitSync.enabled: true there is nothing to do — the sidecar clones onto the new volume. Otherwise copy them off the old shared volume:

cpln workload exec OLD_RELEASE_NAME-airflow-webserver --gvc OLD_GVC_NAME --container airflow --quiet -- \
  sh -c 'tar -cf - -C /opt/airflow/dags . | base64' | base64 -d > dags.tar

base64 < dags.tar | cpln workload exec RELEASE_NAME-airflow-webserver --gvc GVC_NAME \
  --container airflow --stdin --quiet -- sh -c 'base64 -d | tar -xf - -C /opt/airflow/dags'

Verify, then uninstall the old release

Force a redeployment of the new webserver so the scheduler re-reads the restored database, and confirm in the UI that your DAGs, Connections and Variables are present. Only then uninstall the 1.x release — against the GVC you installed it into, where Helm tracks it, not the GVC it created. That uninstall deletes the created GVC and everything still in it.

Values that moved or were removed in 2.0.0:

1.x	2.0.0
`gvc.name`	Gone — the chart deploys into the GVC you install into
`gvc.locations`	`location`, a single top-level location name
`keda.enabled: true` by default	`false` by default, and it requires KEDA enabled on your GVC
—	`airflow.celeryWorker.replicas`, the fixed worker count used when KEDA is off
—	`internalAccess.type` / `internalAccess.workloads`

Upgrading In Place from 1.5.0

Template versions through 1.5.0 created their own GVC, and 2.1.0 deploys into an existing one. A plain cpln helm upgrade across that change deletes the GVC the 1.x release created, with every workload, volume set and identity inside it. These steps mark that GVC “keep” first, so the upgrade leaves it and everything in it in place. It covers a 1.5.0 release (upgrade an older 1.x to 1.5.0 first) that runs in one location; with more than one entry in gvc.locations, use Migrating from 1.x instead. Create the database credentials secret under a new name first, holding the values the database runs with today. 1.5.0 keeps them in RELEASE_NAME-airflow-config, and the upgrade deletes that secret:

cpln secret reveal RELEASE_NAME-airflow-config -o yaml   # username and password in use today
cpln secret create-dictionary --name RELEASE_NAME-airflow-db \
  --entry username='YOUR_CURRENT_USERNAME' --entry password='YOUR_CURRENT_PASSWORD' --entry database='YOUR_CURRENT_DATABASE'

YOUR_CURRENT_DATABASE is postgres.config.database of the 1.5.0 release, airflow unless you changed it. Then mark the GVC and upgrade:

cpln helm get values RELEASE_NAME -o yaml > values-old.yaml
curl -fsSLO https://raw.githubusercontent.com/controlplane-com/templates/main/scripts/keep-gvc.py
python3 keep-gvc.py RELEASE_NAME            # shows what it will change
python3 keep-gvc.py RELEASE_NAME --apply
# write values-new.yaml from values-old.yaml (see below)
cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/airflow --version 2.1.0 --gvc OLD_GVC_NAME -f values-new.yaml

OLD_GVC_NAME is the GVC the 1.x release created (its gvc.name), not the GVC you ran the original install against.
The upgrade output must include Skipped deletion of /org/ORG_NAME/gvc/OLD_GVC_NAME. If it does not, stop: the GVC was not protected.

values-new.yaml is values-old.yaml with these changes:

# gvc: delete the whole block
location: aws-eu-central-1     # the one entry of the old gvc.locations
postgres:
  config:                      # delete username, password and database
    credentialsSecretName: RELEASE_NAME-airflow-db   # the secret created above
keda:
  enabled: true                # what 1.5.0 ran with (its default was true)

If values-old.yaml has no gvc key, the release ran on 1.5.0’s defaults: OLD_GVC_NAME is airflow and location is aws-eu-central-1. Keep keda.enabled as 1.5.0 had it: the GVC it created has KEDA on exactly when that was true, and true against a GVC with KEDA off stops the webserver from starting. The upgrade restarts PostgreSQL, Redis, the webserver and the workers, so Airflow is unavailable for a few minutes. A workload can wait with is not allowed to reveal the secret in get-deployments while access to the new secrets propagates; it clears by itself (see Upgrading from 2.0.0). Afterwards the GVC belongs to you, not the release: cpln helm uninstall leaves it in place, and you delete it with cpln gvc delete OLD_GVC_NAME. Details: the keep-gvc script documentation.

Upgrading from 2.0.0

Template version 2.1.0 removed postgres.config.username, postgres.config.password and postgres.config.database: the metadata database credentials are now a prerequisite secret. Carrying the old keys forward fails the render. 2.0.0 created that secret itself (RELEASE_NAME-airflow-config), so this upgrade deletes it, and the chart refuses that name for the new secret. Before upgrading, create a secret under a new name holding the values your database was initialized with — not new ones, or Airflow loses its database connection:

cpln secret reveal RELEASE_NAME-airflow-config -o yaml   # username and password the database runs with today
cpln secret create-dictionary --name RELEASE_NAME-airflow-db \
  --entry username='YOUR_CURRENT_USERNAME' --entry password='YOUR_CURRENT_PASSWORD' --entry database='YOUR_CURRENT_DATABASE'

YOUR_CURRENT_DATABASE is postgres.config.database of the 2.0.0 release — airflow unless you changed it; the old secret does not hold it. Then upgrade with postgres.config.credentialsSecretName set to the new name and the three old keys removed. An upgrade that passes no values at all uses the default name my-airflow-db-credentials, so create the secret under that name instead if you upgrade that way. The upgrade restarts PostgreSQL, the webserver and the workers (they now read the new secret), so Airflow is briefly unavailable. The new version of a workload can also wait for several minutes, with get-deployments reporting that it is not allowed to reveal the secret, while access to the new secret propagates; the previous version keeps running meanwhile, and it clears by itself (or wait a few minutes, then cpln workload force-redeployment the waiting workload).

Scaling and Availability

Workers scale; the webserver does not. Raise airflow.celeryWorker.replicas for a fixed pool, or enable KEDA to scale on queue length (see KEDA Autoscaling). The webserver, database and broker are each a single replica on a single volume.
Single location by construction. The shared Airflow home provisions one volume per location, so a second location would be a second, disconnected set of DAGs and logs. Run a separate release per location if you need Airflow in two places.
The first Helm upgrade after an install can restart the bundled database and the webserver even when nothing changed. Expect a brief interruption of the UI; later upgrades with unchanged values are clean.
A wide simultaneous fan-out can fail tasks without starting them. The Celery executor’s Redis transport in this worker image has an upstream race when many tasks are dispatched at the same instant; the affected tasks are marked failed with no hostname, no start time and an empty log. Stagger task submission rather than fanning out wide at once.

Troubleshooting

The install succeeded but nothing starts and cpln logs returns no lines

Symptom: every resource reports created, no workload becomes ready, cpln logs is empty.Cause: the auth secret (or the git token secret) does not exist, so no container ever starts.Fix: read the deployment message, which names the missing secret:

cpln workload get-deployments RELEASE_NAME-airflow-webserver --gvc GVC_NAME -o yaml

The secret AIRFLOW_AUTH_SECRET_NAME no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed.

Create the secret exactly as in Prerequisites, then force a redeployment rather than waiting for the automatic recovery.

Every location reports This workload location is deactivated because maxScale is set to 0

Symptom: the install succeeded, all four workloads sit at zero replicas, and cpln workload get-deployments shows that message for every location.Cause: location names a location the GVC does not have. The platform stores the placement without validating it.Fix: compare location with spec.staticPlacement.locationLinks from cpln gvc get GVC_NAME -o yaml, then either add the location to the GVC or upgrade the release with a location the GVC already has. Seeing the message on the GVC’s other locations while the configured one runs is normal.

Webserver log shows FATAL: keda.enabled is true in your values, but GVC has spec.keda.enabled = false

Symptom: the webserver crash-loops on a fresh install with that line in cpln logs.Cause: keda.enabled: true without KEDA enabled on the GVC. The worker workload was accepted anyway and will never autoscale.Fix: enable KEDA on the GVC as in Prerequisites; the webserver passes the check on its next restart. Or set keda.enabled: false and use airflow.celeryWorker.replicas.

Workers never scale and tasks stay queued with no error

Symptom: keda.enabled: true, the worker workload is healthy at keda.minScale (zero workers at minScale: 0), queued tasks never run, nothing is logged.Cause: KEDA is not enabled on the GVC, and the install was onto an existing Airflow home, so the webserver only warned instead of refusing to start. Under internalAccess.type: workload-list the template already allows the KEDA operator onto the broker, so that is not the cause.Fix: read spec.keda.enabled with cpln gvc get GVC_NAME -o yaml and enable it as in Prerequisites.

Tasks fail without ever starting after a large fan-out

Symptom: several tasks of a wide DAG run are marked failed with an empty log, no hostname and no start time, while the rest succeed.Cause: an upstream race in the Celery executor’s Redis transport (module 'redis' has no attribute 'client' from kombu/transport/redis.py) when many tasks are dispatched at the same instant.Fix: stagger task submission, and count completed task instances rather than relying on the UI to flag the failures.

helm upgrade prints the gvc values key was REMOVED

Symptom: upgrading a 1.x release to 2.0.0 fails at render with the message quoted in Migrating from 1.x.Cause: your values still carry the 1.x gvc key. The refusal is deliberate; a plain upgrade would delete the GVC the 1.x chart created.Fix: do not remove the key and retry. Follow Upgrading In Place from 1.5.0, or install 2.0.0 as a new release and move the data as described in Migrating from 1.x.

The canonical endpoint returns 403

Symptom: the webserver is ready but its public URL returns 403.Cause: firewallConfig.inboundAllowCIDR is [] (the default), or a recent change has not taken effect yet.Fix: use the port-forward from First Run, or add your CIDR ranges and allow a few minutes for the change to apply.

Airflow cannot connect to the metadata database after changing the database secret

Symptom: after editing the password key of the database credentials secret and redeploying, the webserver and workers fail to authenticate against PostgreSQL.Cause: PostgreSQL reads POSTGRES_PASSWORD only when the data directory is initialized, so the database still has the old password while Airflow’s connection string carries the new one.Fix: change the password in the database first, as described in Rotating the Database Password, or put the old value back in the secret and redeploy the webserver and workers.

Upgrade fails because postgres.config.username, password and database were REMOVED

Symptom: upgrading a 2.0.0 release fails at render with airflow 2.1.0: postgres.config.username, postgres.config.password and postgres.config.database were REMOVED.Cause: your values still carry the 2.0.0 plaintext database credentials.Fix: create the database credentials secret holding your current values and remove the old keys — see Upgrading from 2.0.0.

Important Notes

Create the auth and database credentials secrets before installing. Without either the deployment wedges with no log output; cpln workload get-deployments is the one place the missing secret is named.
The Fernet key can never be rotated. Generate it once, keep it, and back it up with the auth secret.
Never run a plain helm upgrade on a 1.x release onto 2.0.0 — it deletes the GVC the 1.x chart created and everything in it. See Upgrading In Place from 1.5.0 or Migrating from 1.x.
location must be a location of your GVC. Otherwise the install succeeds and nothing starts anywhere.
Enable KEDA on the GVC before setting keda.enabled: true, or the workers sit at keda.minScale forever with no error.
Rotating the database password needs ALTER ROLE plus a forced redeployment of the webserver and workers — editing the secret alone changes nothing; see Rotating the Database Password.
The UI is closed by default. Reach it with cpln port-forward; a firewall change takes a few minutes to apply.
The first Helm upgrade after an install can restart the database and webserver even with unchanged values; expect a brief interruption.
Changing the admin password means editing the secret and forcing a webserver redeployment; see Rotating the Admin Password.
Uninstalling deletes all three volume sets, including the DAGs and logs on the shared Airflow home. Back up the metadata database first; see Backing Up.

External References

Apache Airflow Documentation

Official Apache Airflow documentation

Fernet Key and Encryption at Rest

How Airflow encrypts Connections and Variables

Airflow Public API Authentication

Requesting and using JWT access tokens

CeleryExecutor

The CeleryExecutor and distributed task execution

KEDA Documentation

Event-driven autoscaling documentation

Airflow Template

View the source files, default values, and chart definition

​Overview

​What Gets Created

​Prerequisites

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​Location

​PostgreSQL Metadata Database

​Redis Broker

​Airflow

​Storage

​Access

​DAG Delivery With git-sync

​KEDA Autoscaling

​Connecting

​First Run

​API Access

​Operations

​Deploying DAGs

​Backing Up

​Restoring a Backup

​Rotating the Admin Password

​Rotating the Database Password

​Upgrading from 1.4.x or Earlier

​Migrating from 1.x

​Upgrading In Place from 1.5.0

​Upgrading from 2.0.0

​Scaling and Availability

​Troubleshooting

​Important Notes

​External References

Apache Airflow Documentation

Fernet Key and Encryption at Rest

Airflow Public API Authentication

CeleryExecutor

KEDA Documentation

Airflow Template

Overview

What Gets Created

Prerequisites

Installation

Configuration

Location

PostgreSQL Metadata Database

Redis Broker

Airflow

Storage

Access

DAG Delivery With git-sync

KEDA Autoscaling

Connecting

First Run

API Access

Operations

Deploying DAGs

Backing Up

Restoring a Backup

Rotating the Admin Password

Rotating the Database Password

Upgrading from 1.4.x or Earlier

Migrating from 1.x

Upgrading In Place from 1.5.0

Upgrading from 2.0.0

Scaling and Availability

Troubleshooting

Important Notes

External References