Skip to main content

Overview

Redpanda is a Kafka-compatible streaming platform written in C++. It implements the Kafka wire protocol natively, so any Kafka client, SDK, or tool works without modification. This template deploys a stateful Redpanda broker cluster with SASL authentication, Schema Registry, an optional HTTP Proxy, and an optional web console. SASL credentials are not template values. Each user comes from a dictionary secret you create before installing, so the credentials never pass through Helm and never land in the release. The console ships closed to the internet, because the build shipped here has no login of its own.
This template does not create a GVC. It deploys into an existing GVC that you already have, and every resource it creates lands in that GVC.

What Gets Created

The template creates no credential secret of its own: every SASL user is a secret you create.

Prerequisites

One dictionary secret per SASL user must exist before you install. Every entry in redpanda.auth.users names a dictionary secret holding the keys username and password. Secrets are org-level, so no GVC flag is involved.
1

Create the superuser credentials secret

The first entry in redpanda.auth.users is the cluster superuser. The console, the Schema Registry client and (when enabled) the HTTP Proxy all authenticate to the brokers as this user.
Set redpanda.auth.users[0].credentialsSecretName to this name.
2

Add one secret per additional user

Create another dictionary secret in the same shape for each extra SASL user, and add one credentialsSecretName entry per secret under redpanda.auth.users. An additional user starts with no ACLs: it can authenticate but cannot read or write anything until you grant it access (see Troubleshooting).
3

Read a secret back later

The -o yaml is required. A bare cpln secret reveal prints only a summary table, not the values:
Use printf, not echo, if you pipe a generated password in from elsewhere. echo appends a newline, which Redpanda cannot carry in a config value. The broker startup script detects that and fails with a message naming the secret and key, rather than starting a cluster that rejects every login. Passwords containing quotes, colons and braces are handled correctly.
Create the secrets before installing. A credentialsSecretName pointing at a secret that does not exist installs “successfully” and then wedges: every resource reports created, the workload never becomes ready, and cpln logs returns zero lines because no container ever starts. The only diagnostic is status.versions[].message:
It names the missing secret: The secret <name> no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed. The command is get-deployments — plain cpln workload get has no versions key. Once the secret exists the workload recovers on its own after several minutes; to skip the wait:
Nothing else is required for a default install. Exposing the brokers over the internet additionally needs a domain you control and a dedicated load balancer on the GVC — see External Kafka Access.

Installation

Install the template into an existing GVC, naming the superuser credentials secret you created:
To install using another method, follow the instructions for it:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Each block below matches the shipped values.yaml for the keys it shows.

Cluster Size and Resources

replicas must be 1, 3 or 5; an even count cannot form a Raft quorum and is rejected at install. The shipped CPU limit-to-reservation ratio is 3:1 — if you raise maxCpu, raise minCpu with it, because a stateful workload rejects a ratio above 4:1.

Storage

Each broker gets its own persistent volume. For high-throughput production workloads switch to high-throughput-ssd (minimum 200 GB). Topic data is the only durable state this template holds, and the scheduled snapshots are the only thing backing it up — see Backing Up.
To encrypt the volumes with your own AWS KMS key, uncomment customEncryption and set region to the Control Plane location of the key and keyId to the key’s full ARN. Custom encryption applies only when a volume is first created, and the key policy must grant Control Plane access before the volume is provisioned — see custom encryption in the volume set reference.

Firewall

The broker workload is reachable from the GVC only; external inbound and outbound are off unless you uncomment them. inboundAllowWorkload narrows internal access to specific workloads, and it is the control that limits who can reach the unauthenticated Admin API (see Admin API Access).

Listeners

Set pandaproxy.enabled: true to produce and consume over REST without a Kafka client. The external listener is covered under External Kafka Access.

Authentication

SASL is always on. Every user’s credentials are a prerequisite secret (see Prerequisites); the first entry is the cluster superuser.
SASL users are created on the first boot of broker 0 and then live in the cluster’s own metadata. Editing a credentials secret afterwards does not change the user’s password — see Rotating SASL Credentials.

ACLs

With allowEveryoneIfNoAclFound: false (the default), a client with no matching ACL is denied: a non-superuser with no ACLs can authenticate but sees no topics and cannot create one.

Cluster Identity and Extra Broker Configuration

cluster_id is generated on first boot when left empty; set it explicitly to preserve the cluster identity across reinstalls. Any key under extra_configurations is written into the broker configuration at startup.

Console

The console workload is on by default and closed to the internet. domain publishes it on a custom domain over TLS and also requires external_inboundAllowCIDR to be set. Read Redpanda Console before opening either.

Connecting

Redpanda is reachable from any workload in the same GVC: A specific broker replica is addressable directly:
Verify the cluster is healthy from inside a broker. rpk cluster health uses the Admin API, so it needs no password:
Connect with rpk from a workload in the same GVC, using the username and password from your credentials secret:
For Kafka clients:
The Schema Registry answers the Confluent-compatible API with HTTP basic auth, using the same credentials:

Redpanda Console

The console has no login in this build. Console authentication (OIDC and basic login) and role-based authorization are Redpanda Enterprise features. Without a license there is no login screen, every visitor shares the same access level, and that shared session carries the cluster superuser’s SASL credentials. Anyone who can reach the console can browse and publish messages, create and delete topics, and manage consumer groups and ACLs. External inbound is therefore closed by default. Reach the UI through a tunnel instead — it works with the firewall fully closed:
Uncommenting redpanda_console.firewall.external_inboundAllowCIDR publishes an unauthenticated Kafka admin UI. Do it only behind your own authenticating proxy, or narrowed to a CIDR range you control — never 0.0.0.0/0. Setting redpanda_console.domain also requires external inbound to be open. Allow a few minutes for a firewall change to take effect.

Admin API Access

The broker Admin API on port 9644 requires no credentials — for writes as well as reads — and under the default same-gvc firewall it is reachable by every workload in the GVC. A workload holding no credentials at all can create or delete SASL users, read the cluster topology, and list the SASL usernames through it. It is not publicly reachable: external inbound on the broker workload is closed, so the exposure is bounded by the GVC. But treat GVC membership as equivalent to cluster admin until you narrow it with redpanda.firewall.inboundAllowWorkload, listing exactly the workloads that need broker access. The brokers’ own health checks and the console depend on this listener, so it cannot simply be closed.

External Kafka Access

Redpanda brokers can be exposed over the internet with TLS via a public domain. Each broker advertises its own per-replica subdomain and Control Plane routes clients to the correct broker using SNI.
1

Bring a domain and a dedicated load balancer

You need a domain you control with DNS at your registrar, and a dedicated load balancer enabled on the GVC — external TCP routing requires it.
2

Add the DNS records before deploying

Disable proxying (for example Cloudflare’s orange cloud) — TCP traffic must pass through directly. GVC_ALIAS is the GVC alias shown under the GVC’s settings in the Control Plane console, and LOCATION is the GVC location a broker runs in (for example aws-us-east-1):Add one CNAME per broker replica, pointing at the GVC gateway rather than at a direct replica address. The _acme-challenge record lets Control Plane issue the certificate via DNS-01. See the Configure Domain guide for the full DNS procedure.
3

Enable the external listener

Uncomment the external block under redpanda.listeners.kafka and set publicAddress to your domain. The template then creates the domain resource, opens the extra container port and advertises the per-replica hostnames:
4

Connect from outside

Each broker advertises RELEASE_NAME-cluster-N-LOCATION.your-domain.com. Use them all as the bootstrap list:
For Kafka clients, use security.protocol=SASL_SSL, sasl.mechanism=SCRAM-SHA-256, and the same bootstrap list.

Operations

Backing Up

Backups are platform-managed volume snapshots of RELEASE_NAME-data, one volume per broker — no cloud account or bucket is involved. By default a snapshot is taken daily at midnight UTC (redpanda.volume.snapshots.schedule: 0 0 * * *), kept for 7 days (retentionDuration: 7d), and a final snapshot is taken when the volume set is deleted (createFinalSnapshot: true). Set schedule: "" to keep only the final-on-delete snapshot. List the snapshots of the release’s volume set:
Take an on-demand snapshot of one broker’s volume (the volume index is the broker ordinal):
Snapshots live in the platform storage layer alongside the volume, not off-site, and they are the only backup this template provides. If you need point-in-time recovery elsewhere, use Redpanda’s tiered storage or mirror the topics you care about to another cluster.

Restoring a Backup

The restore path has not been verified for this template. The platform can restore a volume to one of its snapshots — the replica using that volume restarts and comes back on a fresh volume carrying the snapshot’s data — but that restores one broker’s volume, not the cluster. In a 3- or 5-broker cluster the other brokers keep their current data, so a single-volume restore is not a cluster-level point-in-time recovery, and how a restored broker reconciles with its peers has not been tested here. Treat the snapshots as protection against losing a volume, not as a tested recovery procedure. If you need to rely on them, rehearse the restore on a disposable cluster first and verify the topics and offsets you expect are present before trusting it in production.

Rotating SASL Credentials

SASL users live in the cluster’s metadata after first boot. Editing a credentials secret does not change the user’s password — it only changes what the console and the internal Schema Registry and HTTP Proxy clients present at their next start. Rotate in this order:
1

Change the password in the cluster

2

Update the secret to match

Set the password key of the user’s dictionary secret to the new value, using the same printf precaution as at creation time. Until the next step runs, the console and the internal clients still present the old password and fail to authenticate.
3

Redeploy both workloads

A secret change is only picked up when a replica starts, so force a redeployment of the console and the brokers:

Upgrading from 1.0.x

Template version 1.0.x took each SASL user’s username and password as plain Helm values and shipped a working default for the superuser, and it published the console to 0.0.0.0/0. Both changed in 1.1.0. Each removed key is rejected at render with a message naming its replacement, so a failing upgrade leaves the running release untouched.
  • Create the secret with the username and password the cluster already uses. SASL users live in the cluster’s own metadata on disk, so the secret does not change the password of an existing cluster — it only changes what the internal clients present, and a mismatch means they can no longer authenticate. To change the password itself, follow Rotating SASL Credentials.
  • The console’s public URL now returns 403. Reach it with cpln port-forward (see Redpanda Console), or set redpanda_console.firewall.external_inboundAllowCIDR back explicitly — knowing the UI has no login at all.

Upgrading from 1.1.0

Template version 1.2.0 renames the resource limits in both blocks so it is no longer ambiguous which number is the ceiling. Rename these keys in your values; an upgrade that still carries the old names is refused at render with a message naming the replacement, and the running release is untouched. minCpu and minMemory are unchanged in both blocks.

Scaling and Availability

  • redpanda.replicas must be 1, 3 or 5. Changing it changes broker membership, so treat it as a deliberate cluster operation rather than a routine scale.
  • Topic data survives container restarts, reschedules and upgrades of the same release: each broker’s volume is reattached to the same ordinal.
  • A rolling upgrade is not serialized for a stateful workload on this platform, so brokers may restart together. Configure producers and consumers to retry, and expect the first helm upgrade after an install to restart both workloads even when nothing changed.
  • redpanda.multiZone: true spreads the brokers across availability zones within each location.
  • The console is stateless; redpanda_console.replicas can be raised freely.

Troubleshooting

Symptom: every resource reports created, RELEASE_NAME-cluster never reaches ready, and cpln logs returns zero lines.Cause: a credentialsSecretName names a secret that does not exist, so no container ever starts.Fix: read the one place the missing secret is named, then create it:
Look for The secret <name> no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed. in status.versions[].message. After creating the secret, cpln workload force-redeployment RELEASE_NAME-cluster --gvc GVC_NAME skips the wait for automatic recovery.
Symptom: the broker container exits at start with FATAL: key 'password' of the dictionary secret '<name>' (redpanda.auth.users[0]) contains a newline. This is usually a trailing newline added by echo.Cause: the secret value was piped in with echo, which appends a newline Redpanda cannot carry in a config value. The console reports the same condition as FATAL: ... contains a newline — usually a trailing newline added by echo.Fix: recreate the secret with printf '%s' instead of echo, then force a redeployment of the affected workload.
Symptom: on a cold install the console restarts a few times, logging Invalid credentials.Cause: the console started before broker 0 had created the SASL users, so its first connection attempt was rejected.Fix: none needed — it self-heals once the brokers are healthy. Investigate a credential mismatch only if the error persists after the broker workload is ready.
Symptom: helm upgrade or helm install fails at render with redpanda: redpanda.cpu was RENAMED to redpanda.maxCpu (or the same message for redpanda.memory, redpanda_console.cpu, redpanda_console.memory).Cause: your values still carry the 1.1.0 key names.Fix: rename the keys as listed under Upgrading from 1.1.0. Nothing was applied; the running release is unchanged.
Symptom: render fails with redpanda.auth.users[0].password was REMOVED in redpanda 1.1.0 or redpanda.auth.users[0].username was REMOVED in redpanda 1.1.0.Cause: your values still carry 1.0.x SASL credentials as plain values.Fix: move them into a dictionary secret and reference it with credentialsSecretName — see Upgrading from 1.0.x.
Symptom: render fails with redpanda.replicas must be 1, 3, or 5 — Raft consensus requires an odd number for quorum.Cause: an even replica count, or more than five.Fix: set redpanda.replicas to 1, 3 or 5.
Symptom: a client using an additional user’s credentials connects, but rpk topic list returns nothing and produce or consume fails with an authorization error.Cause: redpanda.acl.allowEveryoneIfNoAclFound is false, so a user with no ACLs is denied everything.Fix: grant the user ACLs as the superuser, for example from inside a broker:
See the rpk security acl create reference for the full set of operations and resources.

Important Notes

  • Every credentials secret named in redpanda.auth.users must exist before you install; without one the deployment wedges with no log output — see Prerequisites.
  • Anyone who can reach the console acts as the cluster superuser — there is no login. Keep external inbound closed unless the console sits behind your own authenticating proxy.
  • The broker Admin API on 9644 is unauthenticated for reads and writes and open to the whole GVC by default — narrow it with redpanda.firewall.inboundAllowWorkload.
  • Editing a credentials secret does not change a SASL user’s password; rotate with rpk security user update, update the secret, then force a redeployment — see Rotating SASL Credentials.
  • redpanda.replicas must be 1, 3 or 5; changing it changes cluster membership.
  • Brokers may restart together during an upgrade — configure producers and consumers to retry.
  • Volume snapshots are the only backup, and the restore path is unverified for this template — see Restoring a Backup.
  • Set redpanda.secrets.cluster_id explicitly if you need the cluster identity to survive an uninstall and reinstall.

External References

Redpanda Documentation

Official Redpanda documentation

Redpanda Console Documentation

Redpanda Console UI guide, including which features need an Enterprise license

rpk CLI Reference

rpk command reference for managing Redpanda clusters

Schema Registry

Managing schemas with Redpanda’s Confluent-compatible Schema Registry

Redpanda Template

View the source files, default values, and chart definition