Overview
Redis is an in-memory data store used as a cache, session store, message broker and queue backend. This template deploys one Redis master-replica cluster whose members span several Control Plane locations, with a Redis Sentinel instance in every location that monitors the master and elects a replacement from any location when the current one is lost. It is not a sharded Redis Cluster and not one independent Redis per region: every instance in every location belongs to the same replication group. Oneengine value runs both tiers on Valkey, the BSD-licensed fork of Redis, instead of Redis; leaving it alone changes nothing.
global.locations, and every one of them must already be a location of that GVC. A GVC location you do not list starts nothing: its deployment reads This workload location is deactivated because maxScale is set to 0. For a Sentinel cluster inside a single location, use the Redis template instead.How Many Locations You Need
A failover needs a majority of Sentinels to agree, and Sentinel runs exactly one instance per location. That arithmetic decides what the cluster survives.floor((N-1)/2) losses, so an even count buys nothing over the odd count below it.
What Gets Created
Prerequisites
An existing GVC with at least two locations
global.locations; every name you list must appear under spec.staticPlacement.locationLinks:Optional: password secrets
redis.passwordSecretName and sentinel.passwordSecretName to the names you used. Leave either value empty to run that tier without a password. See Authentication.Optional: public TCP access
Optional: scheduled backups
Installation
Put your location list in a values file — the default list names three AWS locations your GVC is unlikely to have — and install into the GVC that has them:--set: --set global.locations[0].name=aws-us-east-1 replaces the whole list with that single entry, and the install is refused with redis-multi-location requires at least 2 entries in global.locations.
UI
CLI
Terraform
Pulumi
Configuration
Locations
- Every entry must already be a location of the GVC you install into. At least two are required; an odd count is what buys automatic failover — see How Many Locations You Need.
- The first entry is where the initial master is seeded and where the backup job runs.
global.locations[].replicasis deliberately not read — setredis.replicasPerLocationinstead, which applies to every location. A standalone install that setsreplicasis refused at render withglobal.locations[].replicas is not read by redis-multi-location.- To add or remove a location, edit this list and run
cpln helm upgrade. Every instance restarts (see Scaling and Availability).
[redis] line with the result. On a fresh data directory a mismatch is fatal — the container exits and logs [redis] FATAL: naming the missing location — so a misconfigured cluster never bootstraps. On an already-initialised cluster the same conditions log [redis] WARNING: and the instance keeps serving, so a location that has genuinely gone away cannot take the cluster down with it. Read them with:
Redis or Valkey
engine picks the server both tiers run and is the only difference between the two shapes — topology, replica counts, configuration, Sentinel behaviour, secrets, firewall and backups are identical. Setting engine: valkey makes redis.image and sentinel.image inert and runs the Redis and Sentinel workloads on valkeyImage, so a Valkey server behind a Redis Sentinel cannot be produced by accident.
redis-server, redis-cli and redis-sentinel compatibility symlinks, so every command, configuration directive and connection string on this page is unchanged on either engine, and redis.serverCommand stays redis-server. Two things to know when running Valkey:
INFOreportsredis_version:7.2.4for client compatibility. Readserver_nameandvalkey_versionto see what is really running; a client library that version-gates onredis_versionbelieves it is talking to Redis 7.2.- Use a Debian-based
valkeyImagetag. Both tiers assemble their configuration in a way the-alpineimages do not support, and an-alpinetag fails at start withBad directive or wrong number of arguments. Theredis-alpinetags fail the same way throughredis.imageandsentinel.image. Valkey 9.x tags are reachable throughvalkeyImagebut have not been tested with this template.
Redis
redis.volumeset.initialCapacity GiB volume at /data, with both AOF and RDB persistence on, so data survives restarts, redeployments and upgrades. redis.extraArgs is appended to the server command line and overrides the equivalent directive in the base configuration.
Sentinel
mymaster, marks it down after 5 seconds without a response and promotes a replica once the quorum agrees. It keeps the elected master and its peer list on its own 10 GiB volume, so a Sentinel that restarts rejoins knowing which instance is the master rather than starting from its static configuration.
Authentication
redis.passwordSecretName— applied as bothrequirepassandmasterauthon every Redis instance, and used by Sentinel to authenticate to the master. Empty means no Redis password at all.sentinel.passwordSecretName— applied asrequirepasson every Sentinel. Independent of the Redis password; empty means Sentinel itself is unauthenticated.
redis.passwordSecretName through a helm upgrade has the same effect as the forced redeployment above, because every Redis instance restarts.
Public Access
- Enable the dedicated load balancer on your GVC. Arbitrary TCP ports need one, and it is a paid feature. This template deploys into a GVC you already own, so it cannot turn that on for you; a domain pointed at a GVC without one is created but never serves traffic.
- Add the DNS records before the deploy that turns public access on. Domain ownership is verified when the domain resource is first created; if the records are missing the deploy fails and the error lists the exact TXT and CNAME records required (see Required DNS Records). Add them, then upgrade the release. Disable any DNS proxying — TCP traffic must pass straight through.
- Decide who may connect.
firewall.externalInboundAllowCIDRdefaults to0.0.0.0/0when public access is on; set it to the CIDRs your clients actually come from. Do not pin it to a single/32unless that address is genuinely static — when your office, VPN or NAT egress rotates its IP you lose access to your own cluster from outside.
--no-auth-warning -a "$PASSWORD" to either command when the matching tier has a password.
Firewall
workload-list, list only your clients. The same list governs Redis-to-Redis replication and Sentinel’s monitoring of both tiers, so the Redis, Sentinel and backup workloads are appended for you; a client that is not listed is refused. Firewall changes take a few minutes to take effect — re-test before concluding a rule did not apply.
Backup
Connecting
mymaster rather than at a specific instance: a write sent to a replica is refused with READONLY You can't write against a read only replica. Sentinel answers with the master’s in-GVC hostname (replica-N.RELEASE_NAME-redis.LOCATION_NAME.GVC_NAME.cpln.local), so Sentinel-aware clients must run inside Control Plane; from outside, use the public per-instance ports.
To verify the cluster, ask the Sentinel in one location where the master is, then ask that location’s first Redis instance to respond. The commands are the same on both engines; add --no-auth-warning -a PASSWORD to redis-cli when the matching tier has a password:
redis-cli INFO replication against any instance shows its role and, on a replica, master_link_status:up once it has synced. To confirm which engine is running, redis-cli INFO server answers with server_name:valkey and a valkey_version line on Valkey and with neither on Redis.
Shell-style scripts inside the GVC can follow the master the same way the instances themselves do:
Operations
Backing Up
Backups are off by default. Whenbackup.enabled is true, a cron workload (RELEASE_NAME-redis-backup) runs on backup.schedule in the first declared location only, takes an RDB snapshot and uploads it to your bucket as PREFIX/redis-YYYY-MM-DDTHH-MM-SSZ.rdb.gz. The job authenticates to the bucket through the Redis identity’s cloud account binding — no static keys. It runs in one location because a cron workload otherwise fires in every location of its GVC and would write one copy of the same dump per location into the same bucket.
AWS S3
- Create an S3 bucket. Set
backup.aws.bucketto its name andbackup.aws.regionto its region. - Create a cloud account for your AWS account if you do not have one. Set
backup.aws.cloudAccountNameto its name. - Create an IAM policy with the following JSON, replacing
YOUR_BUCKET_NAME, and setbackup.aws.policyNameto its name. The workload identity is granted this policy andcpln-connectoronly — no broad managed policy:
- Set
backup.aws.prefixto the folder path where backups will be stored, then setbackup.enabled: trueandbackup.provider: aws.
GCS
- Create a GCS bucket. Set
backup.gcp.bucketto its name. - Create a cloud account for your GCP project if you do not have one. Set
backup.gcp.cloudAccountNameto its name. - Grant the Storage Admin (
roles/storage.admin) role to the GCP service account associated with the cloud account. The template additionally binds the identity toroles/storage.objectAdminon exactly the bucket named inbackup.gcp.bucket. - Set
backup.gcp.prefixto the folder path where backups will be stored, then setbackup.enabled: trueandbackup.provider: gcp.
Restoring a Backup
What the backup is: a standard, gzipped RDB file. The upstream-documented way to read one is to start a standalone server of the same engine that wrote it (Valkey 8 cannot read an RDB written by Redis 7.4) with AOF disabled and the decompressed file in place as itsdump.rdb — see Restoring a backup in the Redis persistence documentation — for example on a local machine with Docker. From there you can inspect the data or replay it into the cluster’s current master with your own tooling (redis-cli --scan with DUMP/RESTORE, or a migration tool). Loading the file into this template’s instances in place has not been tested; if you need that path, rehearse it against a throwaway release before relying on it, rather than attempting it for the first time during an incident.
Upgrading from Version 2
Every release before 3.0.0 created its own GVC. Do nothelm upgrade a 2.x release onto 3.0.0. Once the chart stops declaring a GVC, Helm deletes the one the old release created — and deleting a GVC deletes every workload, volume set and identity inside it, while the command prints upgraded successfully.
Install 3.0.0 as a new release against an existing GVC, copy the keyspace across, then remove the old release:
Pick the target GVC
global.locations must be one of them.Install 3.0.0 as a new release and keep the old one running
global.gvc.name and rename global.gvc.locations to a top-level global.locations, then install under a new release name as shown under Installation.Move the data
SENTINEL get-master-addr-by-name mymaster) and copy the keyspace across — redis-cli --scan plus DUMP/RESTORE, or replication from the old master, whichever suits your dataset. Confirm the new cluster serves reads and writes before going on.Cut clients over
Remove the old release
cpln helm uninstall of the old release deletes the GVC it created and the old volume sets with it, so make sure the previous step is genuinely complete and nothing else depends on that GVC.firewall.internalAllowType: workload-list this release’s own workloads are now added to the list for you, so remove them from firewall.workloads if you had listed them.
Scaling and Availability
- Shape. The cluster has
redis.replicasPerLocationRedis instances in each of yourglobal.locationsand one Sentinel per location — six Redis instances and three Sentinels by default. One Redis instance is the master; every other instance, in every location, is a replica of it. Write capacity is that of a single instance; replicas serve reads only. - What a failover looks like. When the master stops responding, the Sentinels mark it down after 5 seconds and, if a majority of them agree, promote a replica — in any location — and repoint the rest at it. Writes pause for the duration; reads from replicas continue; the old master rejoins as a replica when it returns. Clients that discover the master through Sentinel follow it; clients pinned to one instance get
READONLYerrors. - What it survives. With three or more locations, losing the master instance or an entire location leaves a quorum of Sentinels standing and a replica in a surviving location is promoted. With two locations the surviving replicas keep the data but no automatic failover happens — the vote cannot reach a majority. See How Many Locations You Need.
- Replication is asynchronous, so a failover can lose the writes the promoted replica had not yet received, bounded by the replication lag at the moment of failure.
- Every
cpln helm upgrade, and every forced redeployment of the Redis tier, restarts all Redis instances in every location at once rather than one at a time. Treat it as a planned write outage and expect the master to move afterwards. - Resizing. Change
redis.replicasPerLocationto add or remove instances in every location, or editglobal.locationsto add or remove a location; both are acpln helm upgradeand both restart the tier. Grow a volume withredis.volumeset.autoscalingor a largerinitialCapacity. Data survives upgrades, butcpln helm uninstalldeletes both volume sets.
Troubleshooting
Deployment never becomes ready and cpln logs returns nothing
Deployment never becomes ready and cpln logs returns nothing
redis.passwordSecretName or sentinel.passwordSecretName set, the workload never reaches ready and cpln logs returns no lines at all.Cause: the named secret does not exist, so the container never starts. Nothing is logged because nothing ran.Fix: read the missing secret’s name from status.versions[].message:cpln workload force-redeployment RELEASE_NAME-redis --gvc GVC_NAME to skip the wait.Redis exits with a [redis] FATAL line about a location not in the GVC
Redis exits with a [redis] FATAL line about a location not in the GVC
[redis] FATAL: the first location in global.locations ('LOCATION_NAME') is not in GVC 'GVC_NAME'— every Sentinel is seeded to monitor replica 0 in that location, so no master would ever form.[redis] FATAL: Sentinel can never reach its failover quorum.— fewer declared locations exist in the GVC than the quorum needs.[redis] FATAL: locations declared in global.locations are not in GVC 'GVC_NAME'— some other declared location is missing, so losing any one location would end automatic failover.[redis] FATAL: this instance is running in location 'LOCATION_NAME', which is not in the configured location list— the workload is running somewhereglobal.locationsdoes not name.
global.locations and the GVC’s actual location list disagree. The platform accepts the mismatch silently; the instances refuse to bootstrap a cluster that could never fail over.Fix: compare spec.staticPlacement.locationLinks from cpln gvc get GVC_NAME -o yaml with your values, then either add the missing location to the GVC or remove it from global.locations and run cpln helm upgrade. On an already-initialised cluster the same conditions appear as [redis] WARNING: lines and the instances keep serving; fix the list the same way. A [redis] WARNING: could not read the location list of GVC line means the check was skipped this start and nothing was refused.Can't handle RDB format version 12 after changing engine
Can't handle RDB format version 12 after changing engine
cpln helm upgrade that set engine: valkey, every Redis instance crash-loops with Can't handle RDB format version 12 and Error reading the RDB base file appendonly.aof.N.base.rdb, AOF loading aborted.Cause: the data directories were written by Redis 7.4, whose on-disk format Valkey 8 cannot read. engine is an install-time choice.Fix: set engine back to redis and upgrade again; the data is recovered untouched. Move between engines with a dump and restore, or by replicating into a fresh install. See Redis or Valkey.Bad directive or wrong number of arguments at start
Bad directive or wrong number of arguments at start
Bad directive or wrong number of arguments.Cause: valkeyImage, redis.image or sentinel.image points at an -alpine tag. Both tiers assemble their configuration in a way those images do not support.Fix: use a Debian-based tag such as redis:7.4 or valkey/valkey:8.1.9.Render fails with the global.gvc values key was REMOVED
Render fails with the global.gvc values key was REMOVED
cpln helm install or cpln helm upgrade refuses to render and prints a message beginning redis-multi-location 3.0.0: saying the global.gvc values key was REMOVED.Cause: your values file is from a 2.x release.Fix: do not upgrade the 2.x release in place. Follow Upgrading from Version 2: delete global.gvc.name, rename global.gvc.locations to global.locations, and install as a new release.Render fails with requires at least 2 entries in global.locations
Render fails with requires at least 2 entries in global.locations
redis-multi-location requires at least 2 entries in global.locations although your values list several.Cause: the list was passed with --set global.locations[0].name=..., which replaces the whole list with that one entry.Fix: pass the location list from a values file with -f values.yaml as shown under Installation. For a single-location cluster, use the Redis template.Render fails with global.locations[].replicas is not read by redis-multi-location
Render fails with global.locations[].replicas is not read by redis-multi-location
global.locations[].replicas is not read by redis-multi-location — set redis.replicasPerLocation instead.Cause: a replicas field on a global.locations entry. That field belongs to other multi-location templates that share the same list shape.Fix: remove replicas from every location entry and set redis.replicasPerLocation, which applies to every location.The old password still works after rotating the secret
The old password still works after rotating the secret
redis.passwordSecretName or sentinel.passwordSecretName, but the old password is still accepted and the new one is not.Cause: both tiers read the secret at container start, and updating a secret does not restart anything.Fix: force a redeployment of each affected tier, Sentinel first and Redis second, as shown under Authentication. Expect the Redis redeployment to restart every instance at once and trigger a failover.READONLY You can't write against a read only replica
READONLY You can't write against a read only replica
READONLY You can't write against a read only replica. after a failover or an upgrade.Cause: the client is connected to a fixed instance, or to the load-balanced Redis address, and landed on a replica.Fix: configure the client for Sentinel at RELEASE_NAME-sentinel.GVC_NAME.cpln.local:26379 with master name mymaster, so it follows the master wherever it moves.A client is refused after switching to workload-list
A client is refused after switching to workload-list
firewall.internalAllowType: workload-list, a client workload can no longer connect while the cluster itself is healthy.Cause: the client’s workload link is not in firewall.workloads, or the change has not propagated yet.Fix: add //gvc/GVC_NAME/workload/WORKLOAD_NAME for each client to firewall.workloads and upgrade. This release’s own workloads are added for you and must not be listed. Firewall changes take a few minutes to take effect.Important Notes
- Never
helm upgradea 2.x release onto 3.0.0 — it deletes the GVC the old release created and everything in it. Install a new release and migrate; see Upgrading from Version 2. - Every entry in
global.locationsmust already be a location of the GVC, and there must be at least two. A fresh install that names a missing location refuses to start and says which one; an already-running cluster warns and keeps serving. - Choose
enginebefore the first install. Switching it on a running release crash-loops every instance until it is set back; migrate with a dump and restore instead. - Create the password secrets before installing. A
passwordSecretNamethat points at a secret that does not exist wedges the deployment silently; the name appears only instatus.versions[].message. - Rotating a password needs a forced redeployment, Sentinel first and Redis second. Updating the secret alone leaves the old password in force.
- Always discover the master through Sentinel (
mymasteratRELEASE_NAME-sentinel.GVC_NAME.cpln.local:26379). A client pinned to one instance getsREADONLYerrors after any failover or upgrade. - Two locations give no automatic failover. Use three or more to survive losing a location, and remember replication is asynchronous.
- Every
helm upgraderestarts all Redis instances in every location at once — plan it as a short write outage and expect a new master. - Pass
global.locationsfrom a values file, never with--set, which replaces the list with a single entry. - With
firewall.internalAllowType: workload-list, list only your clients — this release’s own workloads are added for you. - Public access needs the dedicated load balancer enabled on your GVC and the DNS records in place first; otherwise the domain is rejected or never serves.
- Bucket backups have no verified in-place restore. Treat them as portable RDB files; see Restoring a Backup.
- Data survives an upgrade but not an uninstall —
cpln helm uninstalldeletes both volume sets.