ClickHouse

Overview

ClickHouse is a column-oriented analytical database for real-time querying and data warehousing at scale. This template deploys ClickHouse in single-node or cluster mode — the mode is derived from the locations list — with object storage (AWS S3, GCS, Azure Blob Storage, or Hetzner Object Storage) as the primary store for table data and a persistent volume for metadata and a local read cache. The database password is not a template value. ClickHouse reads it from a dictionary secret you create before installing, so it never passes through Helm or lands in the release.

This template deploys into an existing GVC that you already have. It does not create or manage a GVC. Pass the GVC with --gvc GVC_NAME at install time. Every location you list in locations must already be on that GVC, and a GVC location you did not list simply starts nothing.

Deployment Modes

The locations list is the topology, not just placement: its length selects the mode, and a location’s position in the list is that shard’s number.

`locations`	Mode	Keeper
1 location, `replicas: 1`	Single-node — development, staging, lower-traffic workloads	Not deployed
1 location, `replicas` > 1	Single-shard cluster — one shard, N replicas, survives a replica loss	1 member, so no fault tolerance
3 or more locations	Multi-shard cluster — one shard per location	3 members across the first three locations, quorum 2
2 locations	Not supported — refused at render	—

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-clickhouse-server`	ClickHouse server (stateful); the replica count per location comes from `locations`. Native protocol on port `9000`, HTTP on `8123`, inter-server on `9009`	—
Workload	`RELEASE_NAME-clickhouse-keeper`	ClickHouse Keeper (stateful), the Raft coordination service; one replica in each of the first three locations. Client port `9181`, Raft port `9234`	Cluster modes only
Volume set	`RELEASE_NAME-clickhouse-server-vs`	Server metadata, `store/` and object-storage disk metadata at `/var/lib/clickhouse` (10 GiB by default); a final snapshot is kept for 7 days on deletion	—
Volume set	`RELEASE_NAME-clickhouse-keeper-vs`	Keeper Raft log and snapshots (10 GiB by default); a final snapshot is kept for 7 days on deletion	Cluster modes only
Secret	`RELEASE_NAME-clickhouse-server-startup`	Server startup script, mounted as a file	—
Secret	`RELEASE_NAME-clickhouse-keeper-startup`	Keeper startup script, mounted as a file	Cluster modes only
Secret	`RELEASE_NAME-clickhouse-s3-config`	Storage configuration for the selected `provider`; the name ends in `-s3-config`, `-gcs-config`, `-azure-config` or `-hetzner-config` and exactly one is created	—
Identity	`RELEASE_NAME-clickhouse-identity`	Workload identity for both workloads; carries the cloud account binding and IAM policy when `provider` is `aws`	—
Policy	`RELEASE_NAME-clickhouse-policy`	Grants the identity `reveal` on the release’s own secrets and on the prerequisite secrets you name	—
Policy	`RELEASE_NAME-clickhouse-gvc-policy`	Grants the identity `view` on the one GVC you install into, so each container can confirm at start that the GVC has every location in `locations`	—

Each server container also mounts two scratch volumes — a local filesystem cache at /var/lib/clickhouse_cache and temporary spill space at /var/lib/clickhouse-tmp. Table data lives in your bucket in every mode; the volume set holds metadata and cache only. The template creates no credential secret: the password exists only in the secret you create.

Prerequisites

Three things must be in place before you install: a GVC with the right locations, a credentials secret, and object storage for the provider you choose. Secrets are org-level, so no --gvc flag is involved when creating them.

Have a GVC that contains every location you will list

The GVC must already exist and must have every location you put in locations. It may have more locations than you list — nothing ClickHouse-related runs in those. Check what a GVC has before installing; the locations are under spec.staticPlacement.locationLinks:

cpln gvc get GVC_NAME -o yaml

To add a missing location, see cpln gvc add-location. Add locations to a shared GVC deliberately — other workloads in that GVC run in every location it has.

Create the credentials secret

A dictionary secret holding exactly two keys, password and database. ClickHouse authenticates as its built-in default user, so there is no username key:

cpln secret create-dictionary --name CREDENTIALS_SECRET_NAME \
  --entry password='YOUR-STRONG-PASSWORD' \
  --entry database=mydatabase

Set database.credentialsSecretName to the name you used. Secret names are org-wide, so give each release its own.

Set up object storage for one provider

Object storage is required in every mode, including single-node — there is no local-only shape. Pick one provider under Object Storage below and complete its steps, then set provider to aws, gcp, azure or hetzner. AWS is the only keyless option; the other three each need their own prerequisite dictionary secret.

Read a secret back later

Pass -o yaml; a bare cpln secret reveal prints only a summary table, not the values:

cpln secret reveal CREDENTIALS_SECRET_NAME -o yaml

Create every prerequisite secret before installing, or the deployment wedges silently. The chart refuses to render when a required credentialsSecretName is blank, but a name pointing at a secret that does not exist installs “successfully” and then never starts. The container never runs, so cpln logs returns zero lines — there is nothing to log. The one place the reason appears is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-clickhouse-server --gvc GVC_NAME -o yaml

The secret CREDENTIALS_SECRET_NAME no longer exists. Workload updates are paused until the
secret is added or the reference to the secret removed.

Use get-deployments — plain cpln workload get has no versions key. Once the secret exists the deployment recovers on its own after several minutes, or immediately with a forced redeployment:

cpln workload force-redeployment RELEASE_NAME-clickhouse-server --gvc GVC_NAME

Object Storage

Complete the steps for exactly one provider. Only the active provider’s values are used.

AWS S3

Access comes from a Control Plane cloud account through the workload identity, so there is no key to store.

Create an S3 bucket. Set aws.bucket to its name and aws.region to its region.
If you do not have one yet, create a Control Plane cloud account for the AWS account holding the bucket. Set aws.cloudAccountName to its name.
Create an IAM policy scoped to exactly that bucket, replacing YOUR_BUCKET_NAME, and set aws.policyName to its name:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME",
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

The template attaches that policy to the workload identity; the bucket in your policy is the only storage ClickHouse can reach.

GCS

ClickHouse reaches GCS over its S3-compatible interface, which requires an interoperability HMAC key. A cloud account is not required.

Create a GCS bucket. Set gcp.bucket to its name.
In the Google Cloud console, open Cloud Storage → Settings → Interoperability and choose Create a key for a service account. Create a new service account (or pick one) and grant it the Storage Object Admin (roles/storage.objectAdmin) role.
Store the HMAC pair in a dictionary secret holding exactly the keys accessKeyId and secretAccessKey, and set gcp.credentialsSecretName to its name:

cpln secret create-dictionary --name GCS_CREDENTIALS_SECRET_NAME \
  --entry accessKeyId=YOUR_HMAC_ACCESS_KEY \
  --entry secretAccessKey=YOUR_HMAC_SECRET

Or do the same with the gcloud CLI:

gcloud config set project YOUR_PROJECT_ID

gcloud storage buckets create gs://YOUR_BUCKET_NAME --location=NAM4

gcloud iam service-accounts create clickhouse-storage

gcloud projects add-iam-policy-binding $(gcloud config get-value project) \
  --member="serviceAccount:clickhouse-storage@$(gcloud config get-value project).iam.gserviceaccount.com" \
  --role="roles/storage.objectAdmin"

gsutil hmac create clickhouse-storage@$(gcloud config get-value project).iam.gserviceaccount.com

Azure Blob Storage

ClickHouse uses Azure’s native Blob Storage SDK, authenticating with a storage account access key. A cloud account is not required.

In the Azure Portal, go to Storage accounts → Create. Use Standard performance, LRS redundancy, and leave hierarchical namespace off. Set azure.storageAccount to the account name.
Inside the storage account, go to Containers → + Container and create a container with the access level Private. Set azure.container to its name.
Go to Security + networking → Access keys and copy either key1 or key2.
Store the key in a dictionary secret holding exactly the key accountKey, and set azure.credentialsSecretName to its name:

cpln secret create-dictionary --name AZURE_CREDENTIALS_SECRET_NAME \
  --entry accountKey=YOUR_ACCOUNT_KEY

Or do the same with the Azure CLI:

az storage account create \
  --name YOUR_STORAGE_ACCOUNT \
  --resource-group YOUR_RESOURCE_GROUP \
  --sku Standard_LRS

az storage container create \
  --name clickhouse-data \
  --account-name YOUR_STORAGE_ACCOUNT

az storage account keys list \
  --account-name YOUR_STORAGE_ACCOUNT \
  --resource-group YOUR_RESOURCE_GROUP \
  --query "[0].value" -o tsv

Hetzner Object Storage

Hetzner Object Storage is S3-compatible and authenticates with an access key pair. A cloud account is not required. Available regions are nbg1 (Nuremberg), hel1 (Helsinki) and fsn1 (Falkenstein).

In the Hetzner Cloud console, go to Object Storage and create a bucket. Set hetzner.bucket to its name and hetzner.region to its region.
Go to Security → S3 Credentials and choose Generate credentials. Save the access key and secret key immediately — the secret is not shown again.
Store the pair in a dictionary secret holding exactly the keys accessKeyId and secretAccessKey, and set hetzner.credentialsSecretName to its name:

cpln secret create-dictionary --name HETZNER_CREDENTIALS_SECRET_NAME \
  --entry accessKeyId=YOUR_ACCESS_KEY \
  --entry secretAccessKey=YOUR_SECRET_KEY

Installation

With the prerequisites in place, install into your existing GVC. The default provider is AWS, so a default install needs the credentials secret and the four aws values:

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/clickhouse \
  --version 3.0.0 --gvc GVC_NAME \
  --set database.credentialsSecretName=CREDENTIALS_SECRET_NAME \
  --set aws.bucket=BUCKET_NAME \
  --set aws.region=BUCKET_REGION \
  --set aws.cloudAccountName=CLOUD_ACCOUNT_NAME \
  --set aws.policyName=POLICY_NAME

The default locations entry is aws-us-east-1 with one replica. If your GVC does not have that location, or you want a cluster, set locations (and any other provider’s values) in a values file passed with -f, as shown in the CLI guide. Or follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Locations

# This chart deploys into the GVC you install into — it does NOT create one.
# Every location listed here MUST already exist in that GVC, or the nodes refuse
# to initialise with a named error (see Prerequisites in the README).
# Extra locations in the GVC are fine: nothing ClickHouse-related runs in them.
#
# Deployment mode is derived from this list:
#   1 location,  replicas: 1   → single-node (no Keeper)
#   1 location,  replicas: >1  → single-shard cluster (Keeper required)
#   3 or more locations        → multi-shard cluster, one shard per location
#   2 locations                → NOT supported
locations:
  - name: aws-us-east-1
    replicas: 1

Each entry pairs a location with a replica count, and the list length selects the mode — see Deployment Modes. A location’s position in the list is its shard number, and Keeper runs in the first three. Two locations, a duplicate location, an empty list and replicas: 0 are all refused at render.

GVC locations you did not list show as inactive in the console, with the deployment message This workload location is deactivated because maxScale is set to 0. That is what a healthy install looks like in a shared GVC, not a fault.

Provider and Object Storage

provider: aws # Options: aws, gcp, azure, or hetzner

aws: # If enabled, all fields below are required - See README for guidance
  bucket: my-clickhouse-bucket # Name of your S3 bucket
  region: us-east-1 # Region of your S3 bucket
  cloudAccountName: my-clickhouse-cloudaccount # Name of your Cloud Account
  policyName: my-clickhouse-s3-policy # Name of your pre-created policy to allow access to the S3 bucket

gcp: # If enabled, all fields below are required - See README for guidance
  bucket: my-clickhouse-gcs-bucket # Name of your GCS bucket
  # REQUIRED PREREQUISITE SECRET — a `dictionary` secret holding exactly
  # `accessKeyId` and `secretAccessKey` (the GCS interoperability HMAC pair).
  # They reach ClickHouse as AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY rather
  # than being written into the disk XML, because a cpln:// reference inside a
  # mounted config file is never resolved.
  credentialsSecretName: my-clickhouse-gcs-credentials

azure: # If enabled, all fields below are required - See README for guidance
  storageAccount: myclickhousestorage # Name of your Azure Storage Account
  container: clickhouse-data # Name of your Blob Storage container
  # REQUIRED PREREQUISITE SECRET — a `dictionary` secret holding exactly
  # `accountKey` for your Azure Storage Account.
  credentialsSecretName: my-clickhouse-azure-credentials

hetzner: # If enabled, all fields below are required - See README for guidance
  bucket: my-clickhouse-hetzner-bucket # Name of your Hetzner Object Storage bucket
  region: nbg1 # Region of your bucket. Options: nbg1, hel1, fsn1
  # REQUIRED PREREQUISITE SECRET — a `dictionary` secret holding exactly
  # `accessKeyId` and `secretAccessKey` for your Hetzner Object Storage.
  credentialsSecretName: my-clickhouse-hetzner-credentials

Set provider and fill in the matching section; the setup steps for each are under Object Storage. Only the active provider’s section is read, its storage configuration secret is the only one created, and its credentials secret (for gcp, azure and hetzner) is the only one the policy grants reveal on.

Switch providers with a fresh install, not an upgrade. A cloud account binding on the workload identity is never removed once set, so a release switched from one provider to another keeps the old provider’s binding attached even though the chart no longer renders it.

Cluster and Database

# Used in cluster modes only. Must be a bare identifier: letters, digits and
# underscores, not starting with a digit — it becomes an XML element name and is
# used unquoted in `ON CLUSTER` DDL.
clusterName: my_cluster

database: # Automatically create a database on initialization using the default user
  # REQUIRED PREREQUISITE SECRET — CREATE IT BEFORE YOU INSTALL.
  # A `dictionary` secret holding exactly two keys: `password` and `database`.
  # ClickHouse has no separate username here — it uses the built-in `default`
  # user, so there is no `username` key. If the secret does not exist at install
  # time the deployment WEDGES silently; see Prerequisites in the README.
  credentialsSecretName: my-clickhouse-credentials

clusterName is the name used in ON CLUSTER DDL and Distributed tables in cluster modes; anything other than a bare identifier is refused at render. The database key of your credentials secret names the database created at first start (with ON CLUSTER in cluster modes), and the password key is the password every client connection uses.

Renaming clusterName on an existing install orphans its Distributed tables. They keep pointing at the old cluster name and fail with Code: 701 ... Requested cluster 'my_cluster' not found (CLUSTER_DOESNT_EXIST). Recreate them after a rename.

Volumes

volumeset:
  server:
    capacity: 10 # initial capacity in GiB (minimum is 10)
  keeper:
    capacity: 10 # initial capacity in GiB (minimum is 10) - cluster modes only

The volume sets hold metadata and Keeper state, not table data — that lives in your bucket. Both take a final snapshot when deleted, kept for 7 days.

Server and Keeper

server:
  image: clickhouse/clickhouse-server:25.10
  resources:
    cpu: 2
    memory: 2Gi
  internalAccess:
    type: same-gvc # options: same-gvc, same-org, workload-list, none
    workloads: [] # required when type is workload-list; list only your clients -- this release's
    # own server and keeper workloads are added automatically, e.g. //gvc/GVC_NAME/workload/WORKLOAD_NAME

# ─── Keeper (cluster modes only) ──────────────────────────────────────────────
keeper: # cluster modes only
  image: clickhouse/clickhouse-keeper:25.10
  resources:
    cpu: 2
    memory: 2Gi
  internalAccess:
    type: same-gvc # options: same-gvc, same-org, workload-list, none
    workloads: [] # required when type is workload-list; list only your clients -- this release's
    # own server and keeper workloads are added automatically, e.g. //gvc/GVC_NAME/workload/WORKLOAD_NAME

keeper is only read in cluster modes. Neither workload is exposed publicly, and the template has no public access option; internalAccess controls which workloads can reach each component:

`internalAccess.type`	Description
`same-gvc`	Allow access from all workloads in the same GVC (default).
`same-org`	Allow access from all workloads in the same organization, including other GVCs.
`workload-list`	Allow access only from the workloads listed in `workloads`.
`none`	No internal access at all.

With workload-list, list only your clients. The server and Keeper reach each other over the same internal firewall, so the chart adds this release’s own server and Keeper workloads to every list it renders. Access changes can take a few minutes to take effect.

Connecting

Target	Address	Notes
Native protocol (`clickhouse-client`, drivers)	`RELEASE_NAME-clickhouse-server.GVC_NAME.cpln.local:9000`	From workloads allowed by `server.internalAccess`.
HTTP interface	`RELEASE_NAME-clickhouse-server.GVC_NAME.cpln.local:8123`	Same scope as above.
A specific server replica	`replica-INDEX.RELEASE_NAME-clickhouse-server.LOCATION_NAME.GVC_NAME.cpln.local:9000`	`INDEX` starts at `0`.
Keeper (cluster modes)	`replica-0.RELEASE_NAME-clickhouse-keeper.LOCATION_NAME.GVC_NAME.cpln.local:9181`	One member in each of the first three locations.
Public endpoint	None	The template exposes no public access.
Username	`default`	The built-in user; there is no other.
Password and database	`cpln secret reveal CREDENTIALS_SECRET_NAME -o yaml`	The `password` and `database` keys.

Always use the fully qualified .GVC_NAME.cpln.local form; the bare workload name is not reliable. From another workload in the same GVC:

clickhouse-client --host RELEASE_NAME-clickhouse-server.GVC_NAME.cpln.local \
  --port 9000 --user default --password 'PASSWORD' --query 'SELECT version()'

To verify a fresh install without a client workload, run clickhouse-client inside the server container:

cpln workload exec RELEASE_NAME-clickhouse-server --gvc GVC_NAME --container server -- \
  clickhouse-client --user default --password 'PASSWORD' --query 'SELECT version()'

Or reach the HTTP interface from your machine through a tunnel — it works even though the workload is closed to the internet:

cpln port-forward RELEASE_NAME-clickhouse-server 8123:8123 --gvc GVC_NAME

echo 'SELECT 1' | curl 'http://default:PASSWORD@localhost:8123/' -d @-

Tables in Cluster Modes

Use ReplicatedMergeTree plus a Distributed table. A plain MergeTree in a multi-shard cluster is single-copy and is not covered by the cluster’s availability story.

CREATE TABLE events_local ON CLUSTER my_cluster (id UInt64, ts DateTime)
ENGINE = ReplicatedMergeTree('/clickhouse/tables/{shard}/events', '{replica}')
ORDER BY id;

CREATE TABLE events ON CLUSTER my_cluster AS events_local
ENGINE = Distributed(my_cluster, currentDatabase(), events_local, rand());

{shard} and {replica} come from each node’s <macros>, which the chart derives from the location’s position in locations and the replica’s index.

Operations

Backing Up

The template ships no backup job. Table data lives in your bucket and the volume set holds only metadata and the object-storage disk metadata that points at those objects, so a volume set snapshot on its own is not a backup of your data — merges continually replace the objects the snapshot’s metadata refers to. The volume sets do take a final snapshot when deleted, kept for 7 days, which protects against an accidental uninstall but not against data loss inside ClickHouse. For a restorable copy, use ClickHouse’s own BACKUP statement to write to a separate bucket or path, as documented in the upstream backup and restore guide:

BACKUP TABLE mydatabase.events_local TO S3('https://BACKUP_BUCKET.s3.amazonaws.com/clickhouse-backups/events', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY');

Backing up with BACKUP ... TO S3(...) has not been exercised on this template. The statement is upstream’s documented form; run it against your own install and confirm the objects land before relying on it. In cluster modes, run it on each shard’s replica (the statement backs up the replica it runs on), or back up the Distributed table’s underlying local table per shard.

Restoring a Backup

No restore path has been verified on this template. Neither RESTORE ... FROM S3(...) nor restoring a volume set snapshot has been exercised on this chart, and a volume set snapshot alone cannot restore table data because that data is in your bucket, not on the volume. Rehearse a backup and restore on a scratch release before relying on either.

Upstream’s documented form restores into the same or a different table name:

RESTORE TABLE mydatabase.events_local FROM S3('https://BACKUP_BUCKET.s3.amazonaws.com/clickhouse-backups/events', 'ACCESS_KEY_ID', 'SECRET_ACCESS_KEY');

See the upstream backup and restore guide for incremental backups and the Azure destination form.

Rotating the Password

The server startup script writes the password key of your credentials secret into ClickHouse’s user configuration every time a replica starts, so a rotation takes effect when the replicas restart with the new value. Workloads resolve a secret only at replica start, so updating the secret alone changes nothing on a running replica.

Update the secret

Apply the whole secret manifest with the new password. Keep the database key unchanged — it names the database created at first start, and a different value makes the next start create an additional database rather than renaming the existing one:

kind: secret
name: CREDENTIALS_SECRET_NAME
type: dictionary
data:
  password: NEW-STRONG-PASSWORD
  database: mydatabase

cpln apply -f secret.yaml

Force a redeployment of the server

cpln workload force-redeployment RELEASE_NAME-clickhouse-server --gvc GVC_NAME

In cluster modes the password is also what the replicas use to talk to each other, so plan the redeployment as a query interruption until every replica has restarted. Keeper does not use the password and needs no redeployment.

Update your clients

Point every application at the new password. The old one is rejected once a replica has restarted.

This procedure follows from how the chart’s startup script applies the secret; it has not been exercised end to end on this template. Confirm the new password with the verify snippet under Connecting after the redeployment completes.

Upgrading from 2.x

Template versions through 2.8.0 created their own GVC, so that GVC is part of a 2.x release’s manifest. Version 3.0.0 does not declare a GVC, and Helm deletes what a chart stops declaring.

An in-place helm upgrade from any 2.x release to 3.0.0 is not supported and destroys the deployment. The upgrade would delete the GVC the 2.x chart created and every workload, volume set and identity inside it — including your ClickHouse metadata and Keeper state — while reporting success.The chart refuses to render when your values still carry the 2.x gvc key, so an upgrade that passes your old values file stops before a single resource is touched:

clickhouse 3.0.0: the `gvc` values key was REMOVED. This chart no longer creates a GVC -- it deploys
into the GVC you install into, and `gvc.locations` moved to the top-level `locations`. DO NOT `helm
upgrade` a 2.x release onto 3.0.0: the upgrade drops `kind: gvc` from the manifest and Helm deletes
what a chart no longer declares, which DESTROYS that GVC and every workload, volumeset and identity
inside it -- including your ClickHouse metadata and Keeper state. Install 3.0.0 as a NEW release
against an existing GVC, reload your data, then uninstall the old release. See `Migrating from 2.x`
in the README.

The guard cannot cover an upgrade run with no values at all. A 2.x release installed on pure defaults has no gvc key for the chart to see, so nothing fires. Do not run cpln helm upgrade on a 2.x release against the 3.0.0 chart under any circumstances. The only supported path is a new release against an existing GVC, reloading the data, then removing the old release.

What changed between 2.x and 3.0.0:

	2.x	3.0.0
GVC	Created by the chart (`gvc.name`)	Not created — install into an existing GVC with `--gvc`
Locations	`gvc.locations`	Top-level `locations`
Internal access	`server.internal_access`, `keeper.internal_access`; the `workloads` list was never applied, so `type: workload-list` blocked all traffic	`server.internalAccess`, `keeper.internalAccess`; the `workloads` list is applied, and the release’s own workloads are added automatically
Database credentials (2.5.0 and earlier)	`database.name`, `database.password` values, written into a chart-owned secret	`database.credentialsSecretName` — a dictionary secret you create (already so from 2.6.0)
GCS and Hetzner credentials (2.6.x and earlier)	`gcp.accessKeyId`, `gcp.secretAccessKey`, `hetzner.accessKeyId`, `hetzner.secretAccessKey` values	`gcp.credentialsSecretName`, `hetzner.credentialsSecretName` (already so from 2.7.0)
Azure credentials (2.7.0 and earlier)	`azure.accountKey` value	`azure.credentialsSecretName` (already so from 2.8.0)
A location with `replicas: 0`	Silently accepted	Refused at render — remove the location from `locations` instead

Any removed key still present in your values is refused at render with a message naming the replacement.

Recover the password the existing cluster uses

Read it from your current values file or, for a release installed with 2.5.0 or earlier, from the secret that version created:

cpln secret reveal RELEASE_NAME-clickhouse-db-config -o yaml

Any password that came from a pre-2.6.0 default was published in the public template repository — treat it as compromised and choose a new one for the new deployment.

Choose the GVC for the new release

Create or pick a GVC and make sure it has every location you intend to list in locations. See Prerequisites.

Install 3.0.0 as a new release into that GVC

Use a different release name — secret names are org-wide, so a same-named release collides with the 2.x one even in another GVC. Give the new release its own bucket or container: the chart writes under a fixed path, so two releases sharing one bucket would share that path too.

Reload your data

Do not try to reuse the old release’s volume set. It holds metadata whose shard identity and Keeper paths belong to the old topology, and it cannot be moved between releases.

Cut over, then uninstall the old release

Point your applications at the new endpoint, then uninstall the old release against the GVC you originally installed it into — not the GVC it created. That is where Helm tracks the release, and uninstalling from there removes the created GVC with it.

Scaling and Availability

locations[].replicas adds replicas to a shard; adding locations adds shards. A location’s position in locations is its shard number and the first three host Keeper, so settle the topology before loading data — existing replicated table paths and macros are derived from that position.
Keeper is the availability floor. The three-member group in a multi-shard cluster tolerates the loss of one member; the single-shard cluster has one member and tolerates none. If a majority of the Keeper locations are missing from the GVC, the containers exit with a named error rather than waiting for an election that can never complete.
With one replica per shard, losing a shard fails every distributed query, not just the rows on that shard. Add replicas per location if partial results are not acceptable.
A cpln helm upgrade restarts every replica in every location at once, including the first upgrade after an install even when nothing changed. Plan each one as a query interruption.
A node failure reschedules a replica and reattaches the same volume set; table data in the bucket is unaffected.
Keep every location and the bucket in the same cloud provider and region family. Cross-region traffic to object storage is billed on every query that misses the local cache.

Troubleshooting

Workload never becomes ready and cpln logs returns nothing

Symptom: the install succeeds, RELEASE_NAME-clickhouse-server never becomes ready, and cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-clickhouse-server"}' --limit 50 --since 10m returns zero lines.Cause: a credentialsSecretName names a secret that does not exist, so the container is never started. The message is only visible in status.versions[].message:

cpln workload get-deployments RELEASE_NAME-clickhouse-server --gvc GVC_NAME -o yaml

The secret CREDENTIALS_SECRET_NAME no longer exists. Workload updates are paused until the
secret is added or the reference to the secret removed.

Fix: create the secret as in Prerequisites. The deployment recovers on its own within several minutes, or run cpln workload force-redeployment RELEASE_NAME-clickhouse-server --gvc GVC_NAME.

Container exits with FATAL: locations declared in values are not in GVC

Symptom: the install succeeds, but a fresh node restarts repeatedly with this in its logs:

[clickhouse] FATAL: locations declared in values are not in GVC 'GVC_NAME': aws-eu-central-1
[clickhouse] GVC 'GVC_NAME' has: aws-us-east-1 aws-us-east-2
[clickhouse] Refusing to initialise. Add the missing location(s) to the GVC, or remove them from 'locations' in your values.

Cause: locations names a location the GVC does not have. The platform does not validate this at install time, so each container checks its GVC at start. A node that is already initialized logs a WARNING instead and keeps serving, so this check never takes down a running cluster.Fix: add the missing location to the GVC (see Prerequisites) or remove it from locations, then upgrade.

Container exits with FATAL: Keeper quorum is unreachable

Symptom: in a cluster mode, the server or Keeper containers exit with:

[clickhouse-keeper] FATAL: Keeper quorum is unreachable. 1 of 3 configured Keeper locations (aws-us-east-1 aws-us-west-2 aws-eu-central-1) exist in GVC 'GVC_NAME' (which has: aws-us-east-1).
[clickhouse-keeper] A Raft group of 3 needs floor(3/2)+1 members. Refusing to start rather than waiting for an election that can never complete.

Cause: Keeper members are replica 0 of the first three entries in locations, and fewer than a majority of those locations exist in the GVC.Fix: add the missing locations to the GVC, or remove them from locations so the first three entries are all locations the GVC has.

A location shows zero replicas and 'deactivated because maxScale is set to 0'

Symptom: cpln workload get-deployments RELEASE_NAME-clickhouse-server --gvc GVC_NAME -o yaml shows a location with no replicas and the message This workload location is deactivated because maxScale is set to 0.Cause: that GVC location is not in locations. This is expected — the template runs nothing in GVC locations you did not declare. If every location shows it, none of your locations entries matches a location on the GVC, so nothing starts and there is no container to log an error.Fix: nothing, when the location is one you did not intend to use. Otherwise compare locations against spec.staticPlacement.locationLinks from cpln gvc get GVC_NAME -o yaml and correct whichever side is wrong.

Install or upgrade is refused at render with a clickhouse validation message

Symptom: cpln helm install or cpln helm upgrade stops before creating anything with one of:

2 locations is not supported. Use 1 location for single-node mode or 3+ locations for cluster mode.

clickhouse: clusterName 'my-cluster' is not a bare identifier. It becomes an XML element name and is used
unquoted in `ON CLUSTER` DDL, so it must match ^[A-Za-z_][A-Za-z0-9_]*$ -- letters, digits and underscores,
not starting with a digit. 'my_cluster' is valid; 'my-cluster' is not.

Cause: the chart validates locations, clusterName, provider and every credentialsSecretName at render time, and refuses shapes that cannot work.Fix: follow the message — use one or three-plus locations, a bare-identifier clusterName, a supported provider, and a non-empty secret name for the active provider. Your running release is unaffected until the upgrade succeeds.

helm upgrade is refused with 'the gvc values key was REMOVED'

Symptom: an upgrade of a 2.x release stops at render time with the message quoted under Upgrading from 2.x.Cause: your values still carry the 2.x gvc key. The refusal is deliberate — the upgrade it blocks would delete the GVC the 2.x chart created.Fix: do not retry the upgrade with the key removed. Install 3.0.0 as a new release against an existing GVC, reload the data, and remove the old release as described in Upgrading from 2.x.

Bootstrap fails with FATAL: Keeper quorum was not reachable within 300s

Symptom: in a cluster mode, the first server replica in the first location exits with:

[clickhouse] FATAL: Keeper quorum was not reachable within 300s.
[clickhouse] Database 'mydatabase' was NOT created. Check that every Keeper location exists in the GVC and that its replica-0 is ready.

Cause: the bootstrap replica creates the database ON CLUSTER, which needs Keeper. It waits a bounded time for Keeper quorum rather than looping forever.Fix: check that RELEASE_NAME-clickhouse-keeper is ready in each of the first three locations with cpln workload get-deployments RELEASE_NAME-clickhouse-keeper --gvc GVC_NAME -o yaml, and read its logs with cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-clickhouse-keeper"}' --limit 50 --since 10m. Once Keeper is up the server replica is restarted automatically and bootstraps the database.

Distributed queries fail with Code: 701 CLUSTER_DOESNT_EXIST

Symptom: queries against a Distributed table fail with Code: 701 ... Requested cluster 'my_cluster' not found (CLUSTER_DOESNT_EXIST).Cause: clusterName was changed on an existing install. Distributed tables keep the cluster name they were created with.Fix: drop and recreate the Distributed tables with the new cluster name, or restore the previous clusterName.

Changing internalAccess appears to have no effect

Symptom: right after a helm upgrade that changed server.internalAccess or keeper.internalAccess, clients still get the old answer.Cause: firewall changes take a few minutes to propagate.Fix: wait and re-test before concluding the setting is broken.

Important Notes

Never run cpln helm upgrade on a 2.x release against 3.0.0 — it deletes the GVC the 2.x chart created and everything in it. Install a new release: Upgrading from 2.x.
Create the credentials secret before installing. A missing one wedges the deployment with no log output; cpln workload get-deployments is the one command that names it.
The GVC must contain every location you list, and may contain more. Undeclared GVC locations run nothing.
Two locations is not supported. Use one (single-node or single-shard) or three or more.
Object storage is required in every mode, including single-node. There is no local-only shape.
The credentials secret has no username key — ClickHouse authenticates as its built-in default user.
Rotating the password requires a forced redeployment of the server workload to take effect.
Switch object-storage providers with a fresh install, not an upgrade.
Renaming clusterName orphans existing Distributed tables (Code: 701). Recreate them after a rename.
A helm upgrade restarts every replica in every location at once. Plan it as a query interruption.
Uninstalling deletes the volume sets (a final snapshot is kept for 7 days) but leaves your bucket and your secrets in place.

External References

ClickHouse Documentation

Official ClickHouse documentation

ClickHouse Keeper

Raft coordination for replicated tables

Data Replication

The ReplicatedMergeTree engine family

Distributed Table Engine

Fan a query out across shards

Backup and Restore

BACKUP and RESTORE to object storage

ClickHouse Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

Deployment Modes

What Gets Created

Prerequisites

Object Storage

AWS S3

GCS

Azure Blob Storage

Hetzner Object Storage

Installation

UI

CLI

Terraform

Pulumi

Configuration

Locations

Provider and Object Storage

Cluster and Database

Volumes

Server and Keeper

Connecting

Tables in Cluster Modes

Operations

Backing Up

Restoring a Backup

Rotating the Password

Upgrading from 2.x

Scaling and Availability

Troubleshooting

Important Notes

External References

ClickHouse Documentation

ClickHouse Keeper

Data Replication

Distributed Table Engine

Backup and Restore

ClickHouse Template

​Overview

​Deployment Modes

​What Gets Created

​Prerequisites

​Object Storage

​AWS S3

​GCS

​Azure Blob Storage

​Hetzner Object Storage

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​Locations

​Provider and Object Storage

​Cluster and Database

​Volumes

​Server and Keeper

​Connecting

​Tables in Cluster Modes

​Operations

​Backing Up

​Restoring a Backup

​Rotating the Password

​Upgrading from 2.x

​Scaling and Availability

​Troubleshooting

​Important Notes

​External References

ClickHouse Documentation

ClickHouse Keeper

Data Replication

Distributed Table Engine

Backup and Restore

ClickHouse Template

Overview

Deployment Modes

What Gets Created

Prerequisites

Object Storage

AWS S3

GCS

Azure Blob Storage

Hetzner Object Storage

Installation

Configuration

Locations

Provider and Object Storage

Cluster and Database

Volumes

Server and Keeper

Connecting

Tables in Cluster Modes

Operations

Backing Up

Restoring a Backup

Rotating the Password

Upgrading from 2.x

Scaling and Availability

Troubleshooting

Important Notes

External References