TiDB

Overview

TiDB is a distributed, MySQL-compatible database designed for horizontal scalability and high availability. It separates compute from storage across three components: a SQL processing layer (tidb-server), a distributed key-value store (TiKV), and a placement driver (PD) that holds cluster metadata and decides where data lives. This template deploys all three tiers across one or more Control Plane locations using PingCAP’s official images, with an optional ProxySQL connection pooler in every location and optional scheduled backups to AWS S3 or Google Cloud Storage. Database credentials are not template values: the root password, application user, password and database name come from a dictionary secret you create before installing, so none of them pass through Helm or land in the release. Root gets its password while the cluster bootstraps, before any TiDB server accepts a connection.

This template deploys into an existing GVC. It does not create one. Every resource lands in the GVC you install into, so cpln workload exec, cpln logs and uninstalling all work against that GVC, and uninstalling can never delete it. Every location you list in locations must already be on that GVC. A GVC location you did not list simply starts nothing — the console shows it as This workload location is deactivated because maxScale is set to 0., which is the expected state, not a fault.

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-pd`	Placement driver quorum (stateful, `pdReplicas` members spread across `locations`)	—
Workload	`RELEASE_NAME-tikv`	TiKV storage nodes (stateful, `locations[].tikvReplicas` per location)	—
Workload	`RELEASE_NAME-server`	MySQL-compatible SQL layer on port `4000` (standard, `locations[].serverReplicas` per location)	—
Workload	`RELEASE_NAME-proxysql`	ProxySQL connection pooler on port `4000` in front of the TiDB servers (standard, `proxysql.replicas` per location)	`proxysql.enabled: true`
Workload	`RELEASE_NAME-tidb-db-init`	Backstop cron job that sets the root password and creates the application database and user if they are missing; runs in the first entry of `locations` only	`autoCreateDatabase.deployInitWorkload: true` (default)
Workload	`RELEASE_NAME-tidb-backup`	Cron job running `br backup full` to your S3 or GCS bucket, in `backup.location` only	`backup.enabled: true`
Volume set	`RELEASE_NAME-tidb-pd-vs`	PD metadata, ext4 on general-purpose SSD, daily snapshots kept 7 days, final snapshot on delete	—
Volume set	`RELEASE_NAME-tidb-tikv-vs`	TiKV data, ext4 on general-purpose SSD, optional autoscaling, daily snapshots kept 7 days, final snapshot on delete	—
Secret	`RELEASE_NAME-tidb-pd-startup`	PD startup script	—
Secret	`RELEASE_NAME-tidb-tikv-startup`	TiKV startup script	—
Secret	`RELEASE_NAME-tidb-server-startup`	tidb-server startup script	—
Secret	`RELEASE_NAME-tidb-db-init`	Database init script	`autoCreateDatabase.deployInitWorkload: true` (default)
Identity	`RELEASE_NAME-tidb-identity`	Shared by every workload; carries the cloud-account binding when backups are on	—
Policy	`RELEASE_NAME-tidb-policy`	`reveal` on the startup-script secrets and on the credentials secret you created	—
Policy	`RELEASE_NAME-tidb-gvc-policy`	`view` on the one GVC you install into, so PD can confirm at boot that the GVC has every listed location	—

Prerequisites

Two things must be in place before you install: a GVC that has every location you intend to use, and a credentials secret. Backups additionally need a bucket and a cloud account — see Backing Up.

Make sure the GVC has every location you will list

The GVC may have more locations than you list — nothing TiDB-related runs in those — but every entry in locations must be on it. Read the GVC’s locations from spec.staticPlacement.locationLinks:

cpln gvc get GVC_NAME -o yaml

The shipped default is a single location, aws-us-east-1. If your GVC uses a different location, set locations at install time (see Installation). This release starts nothing in a GVC location you did not list, but other workloads in that GVC do run in every location the GVC has, so add locations to a shared GVC deliberately.

Create the database credentials secret

Required whenever autoCreateDatabase.enabled is true (the default). A dictionary secret holding exactly four keys — rootPassword, user, password and db. The cluster sets the root password and creates that user and that database while it bootstraps. Secrets are org-level, so no --gvc flag is involved:

cpln secret create-dictionary --name SECRET_NAME \
  --entry rootPassword='YOUR-ROOT-PASSWORD' \
  --entry user=myuser \
  --entry password='YOUR-STRONG-PASSWORD' \
  --entry db=mydb

user must be 1–32 characters, and db 1–64 characters and not ending in a space; the TiDB servers refuse to start otherwise and name the key in their logs. Any characters are allowed in the passwords.Set autoCreateDatabase.credentialsSecretName to this name. Secret names are org-wide, so give each release its own.

Create the secret before installing, or the deployment wedges silently. A credentialsSecretName pointing at a secret that does not exist installs successfully and then never starts. The container never runs, so cpln logs returns zero lines. The one place the reason appears is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-server --gvc GVC_NAME -o yaml

The secret SECRET_NAME no longer exists. Workload updates are paused until the
secret is added or the reference to the secret removed.

Use get-deployments — plain cpln workload get has no versions key. Once the secret exists the deployment recovers on its own after several minutes, or force a redeployment to skip the wait:

cpln workload force-redeployment RELEASE_NAME-server --gvc GVC_NAME

A location the GVC does not have is not caught at install time. The install succeeds, and PD then refuses to bootstrap, restarting with this in its logs:

[tidb-pd] FATAL: locations declared in values are not in GVC 'GVC_NAME': aws-us-west-2
[tidb-pd] GVC 'GVC_NAME' has: aws-us-east-1 aws-us-east-2

Add the missing location to the GVC, or remove it from locations. A PD member that already holds data logs a WARNING instead and keeps serving, so this check never takes down a running cluster.

Installation

Create the credentials secret first, then install into the GVC that has your locations:

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME \
  --set autoCreateDatabase.credentialsSecretName=SECRET_NAME

locations is a list, so when your GVC does not use the default aws-us-east-1 pass it in a values file rather than with --set (a --set locations[0].name=... on its own replaces the whole list element and drops tikvReplicas and serverReplicas):

# values.yaml
locations:
  - name: aws-us-west-2
    tikvReplicas: 3
    serverReplicas: 3
autoCreateDatabase:
  credentialsSecretName: SECRET_NAME

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME -f values.yaml

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Locations

locations:
  - name: aws-us-east-1
    tikvReplicas: 3    # TiKV storage nodes in this location; must be at least 1
    serverReplicas: 3  # TiDB SQL servers in this location; must be at least 1

pdReplicas: 3   # PD members across all locations; odd only (1, 3, 5 or 7); 1 is for testing

Every location listed must already exist in the GVC you install into. tikvReplicas and serverReplicas set the TiKV storage nodes and TiDB SQL servers in that location independently. pdReplicas members are spread evenly across locations, with any remainder going to the first ones. A count of 0 is refused at render — remove the location from the list instead. Clients reach only the TiDB servers in their own location, so run your applications in a location listed in locations. A client in a GVC location this release does not run in cannot connect. Set serverReplicas to 2 or more to keep a location serving when one of its SQL servers fails. The default — one location, three TiKV nodes, three SQL servers, three PD members — survives the loss of a node. It does not survive the loss of a location. For that, use three locations with one PD member and one TiKV node each:

locations:
  - name: aws-us-east-1
    tikvReplicas: 1
    serverReplicas: 2
  - name: aws-us-west-2
    tikvReplicas: 1
    serverReplicas: 2
  - name: aws-eu-central-1
    tikvReplicas: 1
    serverReplicas: 2
pdReplicas: 3
proxysql:
  enabled: true

PD then keeps quorum when one location goes away, and TiKV spreads each region’s three copies one per location. All three locations must be on the GVC. Clients in the surviving locations keep connecting while new leaders are elected; writes pause briefly during the election, so use a client timeout of 60 seconds, or retry, so that pause is a delay rather than an error. Two SQL servers per location behind ProxySQL keep each location serving through the failure of one server.

Images and Resources

images:
  server: pingcap/tidb:v8.5.7
  tikv: pingcap/tikv:v8.5.7
  pd: pingcap/pd:v8.5.7

resources:
  pd:
    cpu: 2
    memory: 4Gi
  server:
    cpu: 2
    memory: 2Gi
  tikv:
    cpu: 2
    memory: 4Gi

Bump images.* together with backup.image: from v8.5.7, br enforces a version match with the cluster. The shipped sizes are for testing. For production, PD wants 4–8 CPU / 8–16Gi, tidb-server 8–16 CPU / 16–32Gi (it scales with concurrent connections), and TiKV 8–16 CPU / 32–64Gi.

Database Initialization

autoCreateDatabase: # Sets the root password at cluster bootstrap and creates the application database and user
  enabled: true
  deployInitWorkload: true # Backstop job for clusters created before 2.2.0; on a fresh 2.2.0 install it finds the work done. Set to false once initialized
  credentialsSecretName: my-tidb-credentials   # REQUIRED prerequisite dictionary secret — create it BEFORE you install
  schedule: "*/5 * * * *"   # how often the init job runs; it fast-exits once the database exists

credentialsSecretName names the prerequisite secret from Prerequisites. With enabled: true, the first TiDB server sets the root password and creates the application user and database while the cluster bootstraps, before it accepts any connection. Every TiDB server then becomes ready only once root refuses an empty password, so root is never reachable without one. With enabled: false, root starts with no password and no user or database is created. Set a root password with ALTER USER before any client can reach the cluster. RELEASE_NAME-tidb-db-init is a backstop: a cron job in the first entry of locations that sets the root password and creates the user and database if they are missing. On a fresh install it finds the work already done and exits, so every run is a no-op and schedule only controls how often it checks. Once the cluster is initialized, upgrade with deployInitWorkload: false to remove the job and its script secret; tidb-server keeps its reference to the credentials secret.

Connection Pooling (ProxySQL)

proxysql:
  enabled: false                   # set true for a ProxySQL pooler tier in every location
  image: proxysql/proxysql:3.0.11
  replicas: 2                      # pooler replicas per location
  resources:
    cpu: 500m
    memory: 512Mi

With proxysql.enabled: true the template runs a ProxySQL connection pooler in every configured location, in front of that location’s TiDB servers. Clients then connect to RELEASE_NAME-proxysql.GVC_NAME.cpln.local:4000 instead of RELEASE_NAME-server, with the same users and passwords (see Connecting).

Requires autoCreateDatabase.enabled: true. ProxySQL authenticates clients with the user, password and rootPassword from the credentials secret; the chart refuses to render with proxysql.enabled on and autoCreateDatabase.enabled off.
Reachable by whoever internal_access.server allows — the pooler follows the SQL tier’s access setting.
Clients must connect with a utf8mb4 charset. Every modern driver defaults to it; the mysql command-line client takes its charset from the OS locale, so pass --default-character-set=utf8mb4.

Storage

volumeset:
  tikv:
    capacity: 10 # initial capacity in GiB (minimum is 10)
    snapshots:
      schedule: "0 3 * * *"    # volume snapshots of every TiKV node, daily at 3am UTC; at most hourly; "" turns them off
      retentionDuration: 7d    # how long each snapshot is kept
    autoscaling:
      enabled: false # Set to true to enable autoscaling
      maxCapacity: 100 # Maximum capacity in GiB
      minFreePercentage: 10 # Minimum free percentage before scaling triggers
      scalingFactor: 1.2 # Multiplier applied to current capacity when scaling
  pd:
    capacity: 10 # initial capacity in GiB (minimum is 10)
    snapshots:
      schedule: "0 3 * * *"    # volume snapshots of every PD member, daily at 3am UTC; at most hourly; "" turns them off
      retentionDuration: 7d    # how long each snapshot is kept

PD only holds cluster metadata, so it has no autoscaling knob. Every PD and TiKV volume is snapshotted on snapshots.schedule and each snapshot is kept for retentionDuration; both volume sets also take a final snapshot when deleted. A schedule more frequent than hourly is refused at render, and "" turns snapshots off. Volume snapshots are per volume and crash-consistent: they protect against losing a single volume. To restore a whole cluster, use a br backup (see Backing Up).

Access

external_access: # Set if client is outside the GVC or in another location
  server_outboundAllowCIDR: []
  tikv_outboundAllowCIDR: [] # Note: when backup.enabled is true, the template automatically allows outbound access (0.0.0.0/0) regardless of this value.
  pd_outboundAllowCIDR: []

internal_access:
  server:
    type: same-gvc # options: same-gvc, same-org, workload-list
    workloads:  # Note: can only be used if type is workload-list
      #- //gvc/GVC_NAME/workload/WORKLOAD_NAME
  tikv:
    type: same-gvc # options: same-gvc, same-org, workload-list
    workloads:  # Note: can only be used if type is workload-list
      #- //gvc/GVC_NAME/workload/WORKLOAD_NAME
  pd:
    type: same-gvc # options: same-gvc, same-org, workload-list
    workloads:  # Note: can only be used if type is workload-list
      #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

`internal_access.*.type`	Who may reach that tier
`same-gvc`	Every workload in the same GVC (default)
`same-org`	Every workload in the org
`workload-list`	Only the workloads listed under `workloads`

With workload-list, list only your clients. Every workload this release creates is always allowed, whatever you set — the three tiers have to reach each other, and each tier’s own replicas have to reach each other.

external_access.*_outboundAllowCIDR opens outbound internet access per tier. When backup.enabled is true, TiKV gets 0.0.0.0/0 outbound regardless, because TiKV uploads backup data to the bucket directly. There is no public inbound access to the MySQL port: reach tidb-server over internal GVC DNS or with cpln port-forward (see Connecting). Allow several minutes for an access change to take effect before concluding it did not apply.

Backup

backup:
  enabled: false
  image: ghcr.io/controlplane-com/backup-images/tidb-backup:8.5.7
  schedule: "0 2 * * *"  # daily at 2am UTC
  activeDeadlineSeconds: 14400  # 4 hours max per backup job
  location: aws-us-east-1  # MUST be one of `locations` above; put it near your bucket

  resources:
    cpu: 1
    memory: 1Gi

  provider: aws # Options: aws or gcp

  aws:
    bucket: my-backup-bucket
    region: us-east-1
    cloudAccountName: my-backup-cloudaccount
    policyName: my-backup-policy
    prefix: tidb/backups

  gcp:
    bucket: my-backup-bucket
    cloudAccountName: my-backup-cloudaccount
    prefix: tidb/backups

backup.location must be one of locations; the chart refuses to render otherwise, because the backup cron runs only there. Bucket, cloud account and access setup for both providers are under Backing Up.

Connecting

Your application must retry — build it in from the start. Restarts, upgrades and failovers are routine, and each one fails a few connections or statements for a few seconds even though the database stays up. Retry a failed connection with a short backoff (about 0.5–1 seconds) for 10–15 seconds, and retry a failed statement, making retried writes idempotent. Connection pools usually replace a broken connection but do not re-run the failed query for you. Use a client timeout of 60 seconds so a brief write pause is a delay rather than an error. Details: Scaling and Availability.

What	Address	Notes
MySQL protocol (applications)	`RELEASE_NAME-server.GVC_NAME.cpln.local:4000`	`user` / `password` from the credentials secret; database `db`
MySQL protocol (root)	`RELEASE_NAME-server.GVC_NAME.cpln.local:4000`	`root` / `rootPassword` from the credentials secret
Pooled MySQL protocol (when `proxysql.enabled`)	`RELEASE_NAME-proxysql.GVC_NAME.cpln.local:4000`	Same users and passwords as above; connect with a utf8mb4 charset
PD HTTP API (cluster state)	`RELEASE_NAME-pd.GVC_NAME.cpln.local:2379`	No authentication; reachable only from inside the GVC
tidb-server status API	`RELEASE_NAME-server.GVC_NAME.cpln.local:10080`	No authentication; reachable only from inside the GVC
Public endpoint	none	The template publishes no public inbound port
Credentials	`cpln secret reveal SECRET_NAME -o yaml`	A bare `cpln secret reveal` prints only a summary table

Always use the fully qualified .GVC_NAME.cpln.local form; the bare workload name is not reliable. Run the client in a location listed in locations — clients reach only the servers in their own location. From a workload in the same GVC, connect with any MySQL client:

mysql -h RELEASE_NAME-server.GVC_NAME.cpln.local -P 4000 -u myuser -p --default-character-set=utf8mb4

With ProxySQL enabled, point the same command at RELEASE_NAME-proxysql.GVC_NAME.cpln.local instead. Without it, clients connect to RELEASE_NAME-server directly. The pingcap/tidb image ships no MySQL client, so to verify from your own machine forward the port and use a local client:

cpln port-forward RELEASE_NAME-server 4000:4000 --gvc GVC_NAME

mysql -h 127.0.0.1 -P 4000 -u myuser -p -e 'SELECT VERSION();'

To see the TiKV stores PD knows about, ask PD from inside the tidb-server container:

cpln workload exec RELEASE_NAME-server --gvc GVC_NAME --container tidb-server -- \
  curl -s http://RELEASE_NAME-pd.GVC_NAME.cpln.local:2379/pd/api/v1/stores

On a multi-location install add --location LOCATION_NAME to pick which location’s replica runs the command. A fresh install takes about 5 minutes to accept connections in one location, and 10 to 15 minutes when its locations span regions or clouds.

Workload	Port	Protocol	Description
`RELEASE_NAME-server`	`4000`	TCP	MySQL-compatible SQL port
`RELEASE_NAME-server`	`10080`	HTTP	tidb-server status and metrics
`RELEASE_NAME-proxysql`	`4000`	TCP	Pooled MySQL-compatible SQL port (when `proxysql.enabled`)
`RELEASE_NAME-pd`	`2379`	TCP	PD client port (HTTP API)
`RELEASE_NAME-pd`	`2380`	TCP	PD peer port
`RELEASE_NAME-tikv`	`20160`	TCP	TiKV data port
`RELEASE_NAME-tikv`	`20180`	TCP	TiKV status port

Operations

Backing Up

When backup.enabled is true, the RELEASE_NAME-tidb-backup cron workload runs TiDB’s br backup full on backup.schedule from backup.location and writes a full cluster snapshot to BUCKET/PREFIX/tidb-TIMESTAMP/. TiKV nodes upload their data to the bucket directly, which is why the template opens TiKV outbound access whenever backups are on. Keep backup.image on the same TiDB version as images.*.

AWS S3
Google Cloud Storage

Create the bucket

Create an S3 bucket. Set backup.aws.bucket to its name and backup.aws.region to its region.

Create a cloud account

If you do not have one, create a cloud account for AWS. Set backup.aws.cloudAccountName to its name.

Create the IAM policy

Create an IAM policy with the JSON below, replacing YOUR_BUCKET_NAME, and set backup.aws.policyName to its name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME",
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

Enable backups

Set backup.enabled: true, backup.provider: aws, backup.location to one of your locations, and backup.aws.prefix to the folder path for backups, then upgrade the release:

cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME -f values.yaml

Create the bucket

Create a GCS bucket. Set backup.gcp.bucket to its name.

Create a cloud account

If you do not have one, create a cloud account for GCP. Set backup.gcp.cloudAccountName to its name.

Grant bucket access

Grant the cloud account’s service account the Storage Admin (roles/storage.admin) role on that bucket. The chart also binds roles/storage.objectAdmin to the release’s identity.

Enable backups

Set backup.enabled: true, backup.provider: gcp, backup.location to one of your locations, and backup.gcp.prefix to the folder path for backups, then upgrade the release:

cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME -f values.yaml

Confirm a backup ran by reading the cron’s output; a successful run ends with Backup completed:

cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-tidb-backup"} |= "Backup completed"' --limit 50 --since 24h

To take a backup now rather than waiting for the schedule, start the cron in backup.location:

cpln workload cron start RELEASE_NAME-tidb-backup --gvc GVC_NAME --location LOCATION_NAME

Switch providers with a fresh install, not an upgrade. An identity’s cloud-account binding is never removed once set, so a release switched from one provider to another keeps the old binding attached even though the chart no longer renders it.

Restoring a Backup

br restore full restores a backup into a new, empty release. It must run inside the GVC with a br that matches the cluster version, and the target release’s TiKV nodes read the backup files from the bucket with the target release’s own identity. The target’s backup cron workload meets all three conditions, so the restore runs there. Rehearse a restore into a scratch release before you need one.

Install an empty target release

Install a new release with a different release name, with autoCreateDatabase.enabled: false so the cluster holds no data before the restore, and with its backup section pointing at the same provider, bucket and cloud account as the backup you are restoring. Until the restore finishes, the target’s root has no password.

Run br restore full from the target's backup workload

List one PD member per entry in the target’s locations: replica-0.RELEASE_NAME-pd.LOCATION_1.GVC_NAME.cpln.local:2379, replica-0.RELEASE_NAME-pd.LOCATION_2.GVC_NAME.cpln.local:2379, and so on — br needs one that answers and finds the rest. Run it in the target’s backup.location. For S3:

cpln workload cron start RELEASE_NAME-tidb-backup --gvc GVC_NAME --location LOCATION_NAME \
  --command /usr/local/bin/br --arg restore --arg full \
  --arg "--pd=replica-0.RELEASE_NAME-pd.LOCATION_1.GVC_NAME.cpln.local:2379,replica-0.RELEASE_NAME-pd.LOCATION_2.GVC_NAME.cpln.local:2379,replica-0.RELEASE_NAME-pd.LOCATION_3.GVC_NAME.cpln.local:2379" \
  --arg "--storage=s3://BUCKET_NAME/PREFIX/tidb-TIMESTAMP" \
  --arg "--s3.region=BUCKET_REGION"

For GCS, use a gcs:// storage URL and add --send-credentials-to-tikv=false — without it the restore fails with You should provide '--gcs.credentials-file':

cpln workload cron start RELEASE_NAME-tidb-backup --gvc GVC_NAME --location LOCATION_NAME \
  --command /usr/local/bin/br --arg restore --arg full \
  --arg "--pd=replica-0.RELEASE_NAME-pd.LOCATION_1.GVC_NAME.cpln.local:2379,replica-0.RELEASE_NAME-pd.LOCATION_2.GVC_NAME.cpln.local:2379,replica-0.RELEASE_NAME-pd.LOCATION_3.GVC_NAME.cpln.local:2379" \
  --arg "--storage=gcs://BUCKET_NAME/PREFIX/tidb-TIMESTAMP" \
  --arg "--send-credentials-to-tikv=false"

Confirm the restore

The run is listed as Successful in the cron’s history, and its output contains Full Restore success summary:

cpln workload cron get RELEASE_NAME-tidb-backup --gvc GVC_NAME

cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-tidb-backup"} |= "Full Restore success summary"' --limit 50 --since 1h

A full restore also restores the source cluster’s users. Root’s password and the application user become the ones the backup was taken with, not any credentials you configured for the target. Connect to the restored cluster with the source cluster’s credentials.

Upgrading from 1.x

Template versions through 1.8.1 created their own GVC, so that GVC is part of the 1.x release’s manifest. Versions 2.0.0 and later do not declare it — and Helm deletes what a chart stops declaring.

Never helm upgrade a 1.x release onto 2.x. The upgrade deletes the GVC the 1.x chart created and every workload, volume set and identity inside it, including your data. The 2.x chart refuses to render when your values still carry a gvc key, so an upgrade run with your old values file fails before touching anything:

Error: execution error at (tidb/templates/identity.yaml:1:4): tidb 2.0.0: the `gvc` values key was
REMOVED. This chart no longer creates a GVC -- it deploys into the GVC you install into ...

That guard cannot see an upgrade run with no values file at all — a 1.x release installed on pure defaults has no gvc key to catch. Do not run the upgrade under any circumstances. Install a new release and move the data across.

Recover the credentials the existing cluster uses

Versions up to 1.7.0 took the credentials as Helm values and wrote them into a chart-owned secret named RELEASE_NAME-tidb-user; from 1.8.0 they live in the secret you created. Read them back with cpln secret reveal SECRET_NAME -o yaml. Any password that came from a pre-1.8.0 default was published in the public template repository — treat it as compromised and choose a new one for the new cluster.

Back up the old cluster

Enable backup on the 1.x release, or run br backup full by hand against its PD endpoint.

Choose the GVC for the new release

Create or pick a GVC that has every location you intend to list in locations. See Prerequisites.

Install 2.2.0 as a new empty restore target

Install into that GVC with a different release name — secret and policy names are org-wide, so a same-named release collides with the 1.x one even in another GVC. Set it up exactly as in step 1 of Restoring a Backup: autoCreateDatabase.enabled: false, and a backup section pointing at the provider, bucket and cloud account the 1.x backup went to.

Restore and cut over

Restore into the new cluster (see Restoring a Backup), then point your applications at the new RELEASE_NAME-server.GVC_NAME.cpln.local:4000 endpoint. The restored cluster keeps the 1.x cluster’s users and passwords; change any compromised password with ALTER USER as in Rotating Credentials.

Uninstall the old release against the GVC you installed it into

Not the GVC it created — the GVC you passed at install time is where Helm tracks the release, and uninstalling from there removes the created GVC with it:

cpln helm uninstall RELEASE_NAME --gvc GVC_NAME

Values keys that moved or were removed since 1.x:

gvc.locations is now the top-level locations, gvc.pdReplicas is now the top-level pdReplicas, and gvc.name is gone.
devMode is gone. locations may hold a single location, and PD’s replication factor is derived from the number of TiKV nodes you configure.
exposeServer is gone. It never published the MySQL port — the only thing the public endpoint would have served was tidb-server’s unauthenticated status API. Reach the server over internal GVC DNS or cpln port-forward.
autoCreateDatabase.database.* is gone (since 1.8.0): the four values are the prerequisite dictionary secret named by autoCreateDatabase.credentialsSecretName.
replicas on a location is now tikvReplicas plus serverReplicas (split in 2.1.0). A count of 0 is refused; remove the location from locations instead.

The guards fire on the mere presence of a removed key, so exposeServer: false and devMode: false fail too, each with an error naming the key and its replacement.

Upgrading from 2.0.0

2.0.0 and 2.2.0 both deploy into an existing GVC, so this is a plain cpln helm upgrade with one values change. Everything in Upgrading from 2.1.0 applies to this upgrade too.

Check the 2.0.0 cluster is healthy

A 2.0.0 bootstrap can leave a TiKV store that never comes up, so the cluster runs without fault tolerance or never serves SQL at all. Upgrading does not repair that. Ask PD for its stores (on a multi-location install add --location LOCATION_NAME):

cpln workload exec RELEASE_NAME-pd --gvc GVC_NAME --replica RELEASE_NAME-pd-0 --container tidb-pd -- sh -c 'curl -s 127.0.0.1:2379/pd/api/v1/stores | grep state_name'

Every store must report "Up". If one is not "Up" and the cluster has never served queries, it holds no data: uninstall it and install 2.2.0 fresh. If it has been serving with a store down, back it up and restore into a fresh 2.2.0 release (see Backing Up and Restoring a Backup).

Split replicas into tikvReplicas and serverReplicas

In every locations entry, replace replicas: N with tikvReplicas: N and serverReplicas: N to keep your current shape exactly:

locations:
  - name: aws-us-east-1
    tikvReplicas: 3
    serverReplicas: 3

The chart refuses to render while a location still sets replicas, so an upgrade with your old values file fails before touching anything:

Error: execution error at (tidb/templates/workload-tikv.yaml:24:20): tidb: location 'aws-us-east-1' sets
`replicas`, which 2.1.0 SPLIT into `tikvReplicas` (TiKV storage nodes) and `serverReplicas` (TiDB SQL servers) ...

Upgrade the release

cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME -f values.yaml

The upgrade restarts every workload once. Replicas in the same location restart one at a time; separate locations restart at the same time, so on a multi-location install writes can pause briefly while PD re-forms. Data is not at risk. ProxySQL stays off unless you set proxysql.enabled: true.

Upgrading from 2.1.0

An in-place cpln helm upgrade with your existing values file; nothing about how clients connect changes.

Check that root refuses an empty password

From a workload in the same GVC, or through cpln port-forward as in Connecting, log in as root with no password:

mysql -h 127.0.0.1 -P 4000 -u root -e 'SELECT 1;'

It must fail with ERROR 1045. If it connects, root has no password: keep autoCreateDatabase.deployInitWorkload: true for the upgrade. Each new TiDB server then waits about 2 minutes before serving and logs a WARNING that root accepts an empty password, and the init job sets the password from your credentials secret.

Plan for one restart of every workload

The upgrade restarts every workload once. Replicas in the same location restart one at a time, but every location restarts at the same time, so with one PD member and one TiKV node per location writes pause for about 75 seconds. Run the upgrade in a quiet period, with clients that retry (see Scaling and Availability). Data is not at risk.

Upgrade the release

cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/tidb \
  --version 2.2.0 --gvc GVC_NAME -f values.yaml

What else changes:

Volume snapshots start. Every PD and TiKV volume is snapshotted daily at 03:00 UTC and kept for 7 days. Set volumeset.tikv.snapshots.schedule and volumeset.pd.snapshots.schedule to "" to keep them off. See Storage.
The init job runs only in the first entry of locations.
ProxySQL no longer uses a startup-script secret, so the upgrade deletes RELEASE_NAME-tidb-proxysql-startup. ProxySQL now also loads credentials that contain a '.
An existing cluster keeps its root password. Root’s password is set at bootstrap only on clusters created by 2.2.0 or later.

Scaling and Availability

What each failure needs. Your application never has to pick another server for anything inside a location: the template and TiDB handle that, and the application needs only the ordinary retry described below. The one thing they cannot handle is the loss of a whole location, because a client reaches only the database in its own location.

Failure	Handled by	What it needs
A TiDB server	The template	`serverReplicas: 2` or more per location. With ProxySQL no queries fail; without it, new connections fail for about 5 seconds
A ProxySQL replica	The template	`proxysql.replicas: 2` (the default)
A TiKV or PD node	TiDB (Raft elects a new leader)	At least 3 TiKV nodes and `pdReplicas: 3` (the default)
A whole location	The template for the database; your application for its clients	The three-location layout in Locations, and your application running in at least 2 of the listed locations behind a public endpoint that spans them. A workload’s canonical endpoint already sends each request to the nearest healthy location

Retry logic your application needs, with or without ProxySQL. Every row above can still fail a connection or a statement for a moment:

Retry failed connections with a short backoff (about 0.5–1 seconds) for 10–15 seconds. Without ProxySQL, this is what covers a TiDB server crash: for several seconds new connections can still reach the server that died. ProxySQL retries those for you, but a ProxySQL replica that dies still drops the connections going through it.
Retry a failed statement, and make retried writes idempotent: a statement running on a process that fails returns an error, and may already have committed.
Set a client timeout of 60 seconds, so a write pause during a leader election, a PD or TiKV restart or an upgrade is a delay rather than an error.
Node loss. The default shape (one location, three TiKV nodes, three PD members) keeps serving through the loss of a node. A TiDB server stays in service while TiKV is partly unavailable; affected statements return errors until TiKV recovers, without restarting the SQL tier.
Location loss. Needs at least three locations with PD spread one per location, as in the Locations example. Clients in the lost location cannot connect until it returns; clients elsewhere keep working, so run your application in at least 2 listed locations.
SQL server loss. Clients reach only the servers in their own location, so serverReplicas: 2 or more is what keeps a location serving through a server failure. Adding ProxySQL in front smooths that over further.
Adding TiKV and tidb-server capacity. Raise locations[].tikvReplicas or locations[].serverReplicas and upgrade. New TiKV stores register with PD and receive data; new tidb-server replicas join behind the same internal address.
PD’s replication factor is fixed at bootstrap. It is the number of TiKV nodes you configure, capped at 3, and PD persists it — scaling TiKV up later does not raise it. Start with at least 3 TiKV nodes if you ever want 3-way replication.
Scaling in (reducing tikvReplicas) and changing pdReplicas on a running cluster are not covered by this template and have not been exercised. TiKV stores hold data; consult the TiDB documentation on scaling in before removing any.
Storage. Enable volumeset.tikv.autoscaling to grow TiKV volumes as they fill. PD volumes are fixed-size.

Rotating Credentials

The credentials secret is applied on first initialization only. Changing it afterwards does not change the cluster — it only changes what the init job and your clients present. Rotate inside TiDB first, then update the secret:

ALTER USER 'myuser'@'%' IDENTIFIED BY 'NEW-PASSWORD';

Write the whole secret back with every key present, then apply it:

# credentials.yaml
kind: secret
name: SECRET_NAME
type: dictionary
data:
  rootPassword: YOUR-ROOT-PASSWORD
  user: myuser
  password: NEW-PASSWORD
  db: mydb

cpln apply -f credentials.yaml

A cpln://secret reference is resolved when a replica starts and is never re-read while it runs, so force a redeployment of tidb-server afterwards so its environment matches:

cpln workload force-redeployment RELEASE_NAME-server --gvc GVC_NAME

With ProxySQL enabled, redeploy the pooler too — it authenticates clients with the credentials from the secret:

cpln workload force-redeployment RELEASE_NAME-proxysql --gvc GVC_NAME

If the init job is still deployed, set autoCreateDatabase.deployInitWorkload: false before rotating — it checks the database with the credentials in the secret on every run.

Troubleshooting

The deployment never starts and the logs are empty

Symptom: cpln logs returns nothing for RELEASE_NAME-server or RELEASE_NAME-tidb-db-init, and the workload never becomes ready.Cause: the credentials secret named by autoCreateDatabase.credentialsSecretName does not exist. The container never runs, so there is nothing to log.Fix: read the message, create the secret, and force a redeployment:

cpln workload get-deployments RELEASE_NAME-server --gvc GVC_NAME -o yaml

The secret SECRET_NAME no longer exists. Workload updates are paused until the
secret is added or the reference to the secret removed.

cpln workload force-redeployment RELEASE_NAME-server --gvc GVC_NAME

PD restarts with FATAL locations declared in values are not in GVC

Symptom: RELEASE_NAME-pd crash-loops and its logs contain:

[tidb-pd] FATAL: locations declared in values are not in GVC 'GVC_NAME': aws-us-west-2
[tidb-pd] GVC 'GVC_NAME' has: aws-us-east-1

Cause: locations names a location the GVC does not have. The platform accepts the install; PD refuses to bootstrap a cluster whose members can never all appear.Fix: add the location to the GVC, or remove it from locations and upgrade. Check the GVC with cpln gvc get GVC_NAME -o yaml (spec.staticPlacement.locationLinks).

A GVC location shows red with This workload location is deactivated because maxScale is set to 0

Symptom: the console marks a location red on every TiDB workload with This workload location is deactivated because maxScale is set to 0.Cause: the GVC has a location you did not list in locations. Nothing TiDB-related runs there by design. The init and backup cron jobs show This workload location is deactivated because suspend is set to true in every location except the one they run in, for the same reason.Fix: none needed. To use that location, add it to locations with tikvReplicas and serverReplicas counts and upgrade.

tidb-server never becomes ready

Symptom: RELEASE_NAME-server stays unready after install.Cause: a TiDB server reports ready only once it has started against the cluster, and that needs a healthy PD and registered TiKV stores. Either tier is still starting, or TiKV cannot reach PD.Fix: check the other tiers in order — PD first, then TiKV:

cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-pd"}' --limit 50 --since 10m
cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-tikv"}' --limit 50 --since 10m

Then confirm what PD sees:

cpln workload exec RELEASE_NAME-server --gvc GVC_NAME --container tidb-server -- \
  curl -s http://RELEASE_NAME-pd.GVC_NAME.cpln.local:2379/pd/api/v1/stores

A fresh install takes about 5 minutes to accept connections in one location, and 10 to 15 minutes when its locations span regions or clouds.

tidb-server restarts with FATAL the credentials secret's user must be 1-32 characters

Symptom: RELEASE_NAME-server restarts repeatedly and its logs contain:

[tidb] FATAL: the credentials secret's 'user' must be 1-32 characters (got 33); fix the secret, then force a redeployment

or the same message for 'db' must be 1-64 characters and not end with a space.Cause: the user or db key in the credentials secret is outside TiDB’s limits. The servers stop before the cluster bootstraps, so nothing is created with a passwordless root.Fix: write the secret back with a valid value (see Rotating Credentials for the cpln apply shape), then force a redeployment:

cpln workload force-redeployment RELEASE_NAME-server --gvc GVC_NAME

TiDB servers log WARNING root accepts an EMPTY password

Symptom: each TiDB server waits about 2 minutes before serving and logs:

[tidb] WARNING: root accepts an EMPTY password on this server; the db-init job sets it (autoCreateDatabase.deployInitWorkload), or set it yourself with ALTER USER

Cause: root has no password. This happens on a cluster created by an earlier version whose root was never given one, or when the credentials secret’s rootPassword is empty.Fix: make sure the credentials secret holds a non-empty rootPassword and keep autoCreateDatabase.deployInitWorkload: true; the init job’s next run sets root’s password from the secret. Or set it yourself with ALTER USER 'root'@'%' IDENTIFIED BY 'YOUR-ROOT-PASSWORD'; and store the same value in the secret.

helm install fails because a snapshots schedule runs more often than hourly

Symptom: rendering fails with

tidb: volumeset.tikv.snapshots.schedule ... must be a 5-field cron with a single fixed minute (snapshots cannot run more often than hourly)

(or the same for volumeset.pd).Cause: volume snapshots can run at most once an hour.Fix: use a schedule with one fixed minute, such as "0 3 * * *", or "" to turn snapshots off.

A GCS restore fails with You should provide gcs.credentials-file

Symptom: br restore full from a GCS backup fails within seconds with You should provide '--gcs.credentials-file' when '--send-credentials-to-tikv' is true.Cause: by default br tries to send its own credentials to TiKV; on this template TiKV reads the bucket with the release’s identity instead.Fix: add --arg "--send-credentials-to-tikv=false" to the restore command, as in Restoring a Backup.

helm upgrade fails at render saying a location sets replicas

Symptom: the upgrade exits before touching anything with tidb: location 'LOCATION_NAME' sets replicas, which 2.1.0 SPLIT into tikvReplicas ... and serverReplicas.Cause: your values file still uses the 2.0.0 replicas key. 2.1.0 sets TiKV nodes and SQL servers separately.Fix: replace replicas: N with tikvReplicas: N and serverReplicas: N in every locations entry. See Upgrading from 2.0.0.

helm install fails with proxysql.enabled requires autoCreateDatabase.enabled

Symptom: rendering fails with tidb: proxysql.enabled requires autoCreateDatabase.enabled.Cause: ProxySQL authenticates clients with the credentials from the secret named by autoCreateDatabase.credentialsSecretName, which is only wired in when autoCreateDatabase.enabled is true.Fix: set autoCreateDatabase.enabled: true with a valid credentials secret, or leave proxysql.enabled: false.

Queries through ProxySQL fail intermittently with ERROR 1273

Symptom: a client works when connected to RELEASE_NAME-server directly, but through RELEASE_NAME-proxysql some queries fail with ERROR 1273 ... latin1_swedish_ci.Cause: the client requests a latin1 charset. TiDB accepts that on a direct connection but rejects it when ProxySQL applies the client charset.Fix: connect with utf8mb4. Modern drivers default to it; for the mysql command-line client pass --default-character-set=utf8mb4.

A client in another GVC location cannot connect

Symptom: an application in the same GVC gets connection failures against RELEASE_NAME-server.GVC_NAME.cpln.local:4000, while clients in other locations work.Cause: the client runs in a GVC location not listed in locations. Clients reach only the TiDB servers (and ProxySQL replicas) in their own location.Fix: run the client in a listed location, or add its location to locations and upgrade.

helm install or upgrade fails at render naming a REMOVED key

Symptom: the command exits before creating anything with an error such as tidb: exposeServer was REMOVED in 2.0.0, the gvc values key was REMOVED, devMode or autoCreateDatabase.database was REMOVED.Cause: your values file still carries a key that 2.0.0 removed. The guards fire on the key’s presence, even when it is set to false.Fix: delete the key and use its replacement from the list in Upgrading from 1.x. If this was an attempt to upgrade a 1.x release in place, stop — that path destroys the 1.x GVC. Install a new release instead.

helm install fails with backup.location is not one of locations

Symptom: rendering fails with tidb: backup.location 'aws-us-west-2' is not one of locations (aws-us-east-1).Cause: the backup cron runs only in backup.location, which must be one of your locations.Fix: set backup.location to a location you configured, preferably the one nearest your bucket.

Clients cannot connect after switching internal access to workload-list

Symptom: your application gets connection refused or timeouts against RELEASE_NAME-server.GVC_NAME.cpln.local:4000 after setting internal_access.server.type: workload-list.Cause: the client workload is missing from internal_access.server.workloads, or the change has not finished taking effect yet.Fix: add the client as //gvc/GVC_NAME/workload/WORKLOAD_NAME, upgrade, and allow several minutes. You never need to list this release’s own workloads — they are always included.

The backup job exits immediately after upgrading TiDB

Symptom: RELEASE_NAME-tidb-backup runs fail right after images.* was bumped, and the br output refuses the cluster version.Cause: br must match the cluster version; from v8.5.7 it enforces the check even with --check-requirements=false.Fix: bump backup.image to the same TiDB version as images.* and upgrade.

The init job keeps running every five minutes

Symptom: RELEASE_NAME-tidb-db-init shows a new run on every tick of autoCreateDatabase.schedule, each logging Database already initialized. Exiting.Cause: expected. The job is a backstop cron in the first entry of locations that fast-exits once the database exists.Fix: none required. To remove it, set autoCreateDatabase.deployInitWorkload: false and upgrade.

Important Notes

Your application must retry failed connections and statements — without it, every upgrade and replica restart shows up as errors in your app. Retry with a short backoff for 10–15 seconds, make retried writes idempotent and use a 60-second client timeout; see Scaling and Availability.
Root has a password from the start only when autoCreateDatabase.enabled is true (the default). With it off, root starts with no password — set one with ALTER USER before clients can reach the cluster.
Each TiDB server start makes one root login attempt with an empty password, to confirm root is protected before serving. If you set FAILED_LOGIN_ATTEMPTS on root, set it above your total number of TiDB servers, or restarting them all locks root.
Upgrading from 2.1.0 restarts every workload once — on the one-PD-and-TiKV-per-location layout, writes pause for about 75 seconds. Check first that root refuses an empty password; see Upgrading from 2.1.0.
Never helm upgrade a 1.x release onto 2.x — it deletes the GVC the 1.x chart created and everything in it. Install a new release: Upgrading from 1.x.
Create the credentials secret before installing. Without it the deployment wedges silently; the only diagnostic is status.versions[].message from cpln workload get-deployments.
Upgrading from 2.0.0 needs one values change: replace each location’s replicas with tikvReplicas and serverReplicas. See Upgrading from 2.0.0.
The GVC must contain every location you list, and may contain more. A missing location is not caught at install: PD exits with FATAL: locations declared in values are not in GVC ....
Run clients in a listed location. Clients reach only the TiDB servers in their own location; use serverReplicas: 2 or more to keep a location serving through a server failure.
Surviving a location loss needs your application in at least 2 locations — see Scaling and Availability.
PD’s replication factor is fixed when the cluster first bootstraps (TiKV node count, capped at 3). Start with at least 3 TiKV nodes if you ever want 3-way replication.
Restore into a new, empty release whose backup section points at the same provider, bucket and cloud account. A full restore brings the source cluster’s users and root password with it; see Restoring a Backup.
Volume snapshots protect single volumes, not the cluster. Use a br backup to restore a whole cluster.
Keep backup.image and images.* on the same TiDB version. br refuses a version mismatch.
Through ProxySQL, connect with a utf8mb4 charset — a latin1 client fails intermittently with ERROR 1273.
There is no public access to the MySQL port. Reach tidb-server over internal GVC DNS or cpln port-forward RELEASE_NAME-server 4000:4000 --gvc GVC_NAME.
Release names must be unique per org — secrets and policies are org-wide, so two releases with the same name collide even in different GVCs.
Credentials apply on first initialization only. Rotate with ALTER USER inside TiDB first, then update the secret, then force a redeployment of tidb-server (and of RELEASE_NAME-proxysql when enabled).
Switch backup providers with a fresh install, not an upgrade — the identity keeps the old cloud-account binding.

External References

TiDB Documentation

Official TiDB documentation

TiDB Architecture

How PD, TiKV and tidb-server fit together

BR Snapshot Backup and Restore

The upstream br backup full and br restore full procedure

Backup Image Source

Source for the TiDB backup container image

TiDB Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

What Gets Created

Prerequisites

Installation

UI

CLI

Terraform

Pulumi

Configuration

Locations

Images and Resources

Database Initialization

Connection Pooling (ProxySQL)

Storage

Access

Backup

Connecting

Operations

Backing Up

Restoring a Backup

Upgrading from 1.x

Upgrading from 2.0.0

Upgrading from 2.1.0

Scaling and Availability

Rotating Credentials

Troubleshooting

Important Notes

External References

TiDB Documentation

TiDB Architecture

BR Snapshot Backup and Restore

Backup Image Source

TiDB Template

​Overview

​What Gets Created

​Prerequisites

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​Locations

​Images and Resources

​Database Initialization

​Connection Pooling (ProxySQL)

​Storage

​Access

​Backup

​Connecting

​Operations

​Backing Up

​Restoring a Backup

​Upgrading from 1.x

​Upgrading from 2.0.0

​Upgrading from 2.1.0

​Scaling and Availability

​Rotating Credentials

​Troubleshooting

​Important Notes

​External References

TiDB Documentation

TiDB Architecture

BR Snapshot Backup and Restore

Backup Image Source

TiDB Template

Overview

What Gets Created

Prerequisites

Installation

Configuration

Locations

Images and Resources

Database Initialization

Connection Pooling (ProxySQL)

Storage

Access

Backup

Connecting

Operations

Backing Up

Restoring a Backup

Upgrading from 1.x

Upgrading from 2.0.0

Upgrading from 2.1.0

Scaling and Availability

Rotating Credentials

Troubleshooting

Important Notes

External References