> ## Documentation Index
> Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Redpanda

> Deploy a Kafka-compatible Redpanda streaming cluster on Control Plane. Covers SASL authentication, Schema Registry, Redpanda Console, external TLS access with per-replica SNI routing, and broker tuning.

## Overview

Redpanda is a Kafka-compatible streaming platform written in C++. It implements the Kafka wire protocol natively, so any Kafka client, SDK, or tool works without modification. This template deploys a stateful Redpanda broker cluster with SASL authentication, Schema Registry, an optional HTTP REST proxy, and an optional web console.

### What Gets Created

* **Stateful Redpanda Workload** — A multi-replica broker cluster using the Seastar async runtime. Each broker gets its own persistent volume.
* **Standard Redpanda Console Workload** *(optional, enabled by default)* — Web UI for browsing topics, inspecting messages, managing consumer groups, and viewing Schema Registry schemas.
* **Volume Set** — One persistent volume per broker replica for data storage.
* **Identity & Policy** — An identity bound to the workloads with `reveal` access to credential secrets.
* **Secrets** — A dictionary secret holding SASL user credentials injected at startup.

<Note>
  This template does not create a GVC. You must deploy it into an existing GVC.
</Note>

## Prerequisites

This template has no external prerequisites. To install, follow the instructions for your preferred method:

<CardGroup cols={2}>
  <Card title="UI" href="/template-catalog/install-manage/ui" icon="laptop">
    Browse, install, and manage templates visually
  </Card>

  <Card title="CLI" href="/template-catalog/install-manage/cli" icon="terminal">
    Manage templates from your terminal
  </Card>

  <Card title="Terraform" href="/template-catalog/install-manage/terraform" icon={<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128"><g fill-rule="evenodd"><path d="M77.941 44.5v36.836L46.324 62.918V26.082zm0 0" fill="#5c4ee5"/><path d="M81.41 81.336l31.633-18.418V26.082L81.41 44.5zm0 0" fill="#4040b2"/><path d="M11.242 42.36L42.86 60.776V23.941L11.242 5.523zm0 0M77.941 85.375L46.324 66.957v36.82l31.617 18.418zm0 0" fill="#5c4ee5"/></g></svg>}>
    Declare templates in your Terraform configurations
  </Card>

  <Card
    title="Pulumi"
    href="/template-catalog/install-manage/pulumi"
    icon={<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" id="Pulumi-Icon--Streamline-Svg-Logos" height="24" width="24">
    <desc>
        Pulumi Icon Streamline Icon: https://streamlinehq.com
    </desc>
    <path fill="#f26e7e" d="M4.683025 13.3318c0.869125 -0.5018 0.870575 -2.1264 0.003225 -3.62865s-2.27504 -2.313275 -3.1441725 -1.811475C0.672945 8.3935 0.6715 10.0181 1.53885 11.52035c0.86735 1.502275 2.27505 2.313275 3.144175 1.81145Zm0.0052 3.2167c0.86735 1.502275 0.865925 3.126875 -0.003225 3.628675 -0.86915 0.5018 -2.2768275 -0.309225 -3.144175 -1.81145 -0.8673525 -1.50225 -0.8659075 -3.126875 0.003225 -3.628675 0.8691325 -0.5018 2.276825 0.309225 3.144175 1.81145Zm5.922875 3.4243c0.86735 1.50225 0.8659 3.126775 -0.003225 3.62875 -0.869125 0.501775 -2.27685 -0.309325 -3.1442 -1.81155 -0.867325 -1.50225 -0.865875 -3.12685 0.00325 -3.628675 0.869125 -0.5018 2.276825 0.309225 3.144175 1.811475Zm-0.001925 -6.845275c0.86735 1.50225 0.8659 3.12685 -0.003225 3.628675 -0.869125 0.5018 -2.276825 -0.309225 -3.144175 -1.811475 -0.86735 -1.50225 -0.8659 -3.12685 0.003225 -3.62865 0.869125 -0.501825 2.276825 0.3092 3.144175 1.81145Z" stroke-width="0.25"></path>
    <path fill="#8a3391" d="M22.45775 11.524125c0.86725 -1.502225 0.865925 -3.12685 -0.003225 -3.62865 -0.869125 -0.501825 -2.276825 0.3092 -3.144175 1.811475 -0.86735 1.50225 -0.8659 3.126825 0.003225 3.62865 0.869125 0.501825 2.276825 -0.3092 3.144175 -1.811475Zm0.000175 3.2151c0.869075 0.5018 0.870625 2.1264 0.003225 3.62865 -0.86735 1.50225 -2.27505 2.313275 -3.144175 1.81145 -0.869125 -0.5018 -0.870575 -2.126425 -0.003225 -3.62865 0.86735 -1.50225 2.27505 -2.313275 3.144175 -1.81145ZM16.536225 18.157875c0.86915 0.501825 0.8706 2.126425 0.00325 3.628675 -0.86735 1.502125 -2.275075 2.313225 -3.1442 1.81145 -0.869125 -0.50175 -0.870575 -2.126425 -0.003225 -3.62865 0.867375 -1.502275 2.27505 -2.3133 3.144175 -1.811475Zm-0.003325 -6.843775c0.869125 0.5018 0.870575 2.126425 0.003225 3.628675s-2.27505 2.313275 -3.1442 1.811475c-0.869125 -0.501825 -0.870575 -2.126425 -0.003225 -3.628675 0.86735 -1.502275 2.27505 -2.313275 3.1442 -1.811475Z" stroke-width="0.25"></path>
    <path fill="#f7bf2a" d="M15.138225 2.06721c0 1.003615 -1.40625 1.817215 -3.14095 1.817215 -1.7347 0 -3.14095 -0.8136 -3.14095 -1.817215C8.856325 1.06359 10.262575 0.25 11.997275 0.25c1.7347 0 3.14095 0.81359 3.14095 1.81721ZM9.2166 5.482375c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172s1.40625 -1.817225 3.14095 -1.817225c1.7347 0 3.14095 0.8136 3.14095 1.817225Zm8.71005 1.8172c1.7347 0 3.14095 -0.813575 3.14095 -1.8172s-1.40625 -1.817225 -3.14095 -1.817225c-1.7347 0 -3.14095 0.8136 -3.14095 1.817225s1.40625 1.8172 3.14095 1.8172Zm-2.788425 1.605625c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172 0 -1.0036 1.40625 -1.8172 3.14095 -1.8172 1.7347 0 3.14095 0.8136 3.14095 1.8172Z" stroke-width="0.25"></path>
    </svg>}
  >
    Declare templates in your Pulumi programs
  </Card>
</CardGroup>

## Configuration

The default `values.yaml` for this template:

```yaml theme={null}
redpanda:
  name: cluster
  image: redpandadata/redpanda:v26.1.9
  replicas: 3
  multiZone: false
  env: []

  cpu: 1500m
  memory: 4Gi
  minCpu: 500m
  minMemory: 2Gi
  smp: 1           # Seastar reactor threads — must match the floor of your cpu limit
  reserveMemory: 1G # memory reserved for the OS; Redpanda uses (memory - reserveMemory)

  volume:
    initialCapacity: 10 # In GiB
    performanceClass: general-purpose-ssd # or high-throughput-ssd (min 200 GiB)
    fileSystemType: xfs # xfs / ext4
    # customEncryption:
    #   enabled: true
    #   region: aws-us-east-2
    #   keyId: arn:aws:kms:us-east-2:1234567890:key/your-key-id

  firewall:
    internal_inboundAllowType: "same-gvc" # Options: same-org / same-gvc
    # external_inboundAllowCIDR: 0.0.0.0/0
    # inboundAllowWorkload:
    #   - //gvc/my-gvc/workload/my-app

  listeners:
    kafka:
      internal:
        port: 9092
      # external:
      #   directReplicaRouting:
      #     containerPort: 9094
      #     publicAddress: redpanda.example.com

    adminApi:
      port: 9644

    schemaRegistry:
      port: 8081

    pandaproxy:
      enabled: false
      port: 8082

  auth:
    saslMechanism: SCRAM-SHA-256 # SCRAM-SHA-256 / SCRAM-SHA-512
    users:
      - username: admin
        password: "your-admin-password"
      # - username: your-app-user
      #   password: "your-app-password"
    superusers: []

  acl:
    allowEveryoneIfNoAclFound: false

  secrets:
    cluster_id: ""  # leave empty to auto-generate; set explicitly to preserve identity across reinstalls

  extra_configurations: {}
    # auto_create_topics_enabled: false
    # log_retention_ms: 604800000
    # log_segment_size: 134217728

redpanda_console:
  enabled: true
  name: console
  image: redpandadata/console:v3.7.4
  cpu: 200m
  memory: 256Mi
  minCpu: 50m
  minMemory: 64Mi
  replicas: 1
  # domain: console.your-domain.com
  firewall:
    external_inboundAllowCIDR: "0.0.0.0/0"
```

### Cluster Size and Resources

* `redpanda.replicas` — Number of broker replicas. A minimum of 3 is recommended for production to ensure Raft quorum.
* `redpanda.cpu` / `redpanda.memory` — Maximum CPU and memory per broker.
* `redpanda.minCpu` / `redpanda.minMemory` — Minimum guaranteed CPU and memory per broker.
* `redpanda.smp` — Number of Seastar reactor threads. Must match the floor of `cpu` (e.g., `cpu: 1500m` → `smp: 1`, `cpu: 3` → `smp: 3`). Without this, Seastar uses all node CPUs and incorrectly divides memory across them.
* `redpanda.reserveMemory` — Memory set aside for the OS. Redpanda uses `(memory - reserveMemory)` for its own heap. Default `1G` works for most configurations.
* `redpanda.multiZone` — Spread brokers across availability zones within the location.

### Storage

Each broker replica gets its own persistent volume. For production workloads with high throughput, use `high-throughput-ssd` (minimum 200 GiB).

* `redpanda.volume.initialCapacity` — Initial volume size in GiB.
* `redpanda.volume.performanceClass` — `general-purpose-ssd` or `high-throughput-ssd`.
* `redpanda.volume.fileSystemType` — `xfs` (default, recommended for Redpanda) or `ext4`.

**Volume encryption** via AWS KMS is supported:

```yaml theme={null}
redpanda:
  volume:
    customEncryption:
      enabled: true
      region: aws-us-east-2
      keyId: arn:aws:kms:us-east-2:1234567890:key/your-key-id
```

<Note>
  After deploying with custom encryption enabled, navigate to each created volume in the Control Plane console, click `spec`, and follow the **AWS Custom Encryption Instructions** to complete the setup.
</Note>

### Authentication

SASL is always enabled. All users are defined under `redpanda.auth.users`. The first user in the list is automatically granted superuser privileges. Additional superusers can be added under `redpanda.auth.superusers`.

* `redpanda.auth.saslMechanism` — `SCRAM-SHA-256` (default) or `SCRAM-SHA-512`.
* `redpanda.auth.users` — List of `username` / `password` pairs created at startup.
* `redpanda.auth.superusers` — Additional usernames to grant superuser privileges.

### ACLs

* `redpanda.acl.allowEveryoneIfNoAclFound` — When `false` (default), clients without an explicit ACL are denied. Set to `true` to allow unauthenticated access when no ACL exists for a resource.

### Listeners

| Listener        | Port   | Description                               |
| --------------- | ------ | ----------------------------------------- |
| Kafka           | `9092` | Kafka-compatible wire protocol (internal) |
| Admin API       | `9644` | Redpanda Admin API for cluster management |
| Schema Registry | `8081` | Confluent-compatible Schema Registry      |
| PandaProxy      | `8082` | HTTP REST proxy *(disabled by default)*   |

Enable PandaProxy to produce and consume messages over HTTP without a Kafka client:

```yaml theme={null}
redpanda:
  listeners:
    pandaproxy:
      enabled: true
      port: 8082
```

### Extra Broker Configuration

Pass any Redpanda broker property directly via `extra_configurations`. These are injected into `redpanda.yaml` at startup:

```yaml theme={null}
redpanda:
  extra_configurations:
    auto_create_topics_enabled: false
    log_retention_ms: 604800000    # 7 days
    log_segment_size: 134217728    # 128 MiB
    log_retention_bytes: -1        # unlimited
```

### Firewall

* `redpanda.firewall.internal_inboundAllowType` — Controls which workloads can reach the brokers:
  * `same-gvc` — All workloads in the same GVC (default).
  * `same-org` — All workloads in the org.
* `redpanda.firewall.inboundAllowWorkload` — Allow specific workloads by path.

## Connecting

Redpanda is accessible internally from any workload in the same GVC:

| Listener                  | Hostname                         | Port   |
| ------------------------- | -------------------------------- | ------ |
| Kafka                     | `{clusterName}.{gvc}.cpln.local` | `9092` |
| Admin API                 | `{clusterName}.{gvc}.cpln.local` | `9644` |
| Schema Registry           | `{clusterName}.{gvc}.cpln.local` | `8081` |
| PandaProxy *(if enabled)* | `{clusterName}.{gvc}.cpln.local` | `8082` |

To connect to a specific broker replica directly:

```
{clusterName}-0.{clusterName}.{gvc}.cpln.local:9092
{clusterName}-1.{clusterName}.{gvc}.cpln.local:9092
```

Connect using `rpk`:

```bash theme={null}
rpk topic list \
  -X brokers={clusterName}.{gvc}.cpln.local:9092 \
  -X sasl.mechanism=SCRAM-SHA-256 \
  -X user=admin \
  -X pass=your-admin-password
```

For Kafka clients, use the following connection properties:

```properties theme={null}
bootstrap.servers={clusterName}.{gvc}.cpln.local:9092
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="admin" \
  password="your-admin-password";
```

## Redpanda Console

The Redpanda Console is enabled by default and accessible via the Control Plane external endpoint for the `{release-name}-console` workload. It provides a web UI for browsing topics, inspecting messages, managing consumer groups, and viewing Schema Registry schemas.

To expose the console on a custom domain, set `redpanda_console.domain`:

```yaml theme={null}
redpanda_console:
  domain: console.your-domain.com
```

This creates a Control Plane domain resource that routes HTTPS traffic to the console workload. The same DNS prerequisites apply as for any Control Plane domain (ownership TXT record and CNAME to the GVC alias).

To restrict console access to specific IPs, update `redpanda_console.firewall.external_inboundAllowCIDR`:

```yaml theme={null}
redpanda_console:
  firewall:
    external_inboundAllowCIDR: "203.0.113.0/24"
```

## External Access

Redpanda brokers can be exposed over the internet via TLS using a public domain. Each broker advertises its own per-replica subdomain and Control Plane routes clients to the correct broker using SNI.

### Prerequisites

1. **A domain you control** with DNS managed by your registrar (e.g. Cloudflare).
2. **Dedicated Load Balancer** enabled on your GVC — required for external TCP routing. Enable under GVC settings in the Control Plane console. See [Configure Domain documentation](https://docs.controlplane.com/guides/configure-domain#dedicated-load-balancing).
3. **DNS records added before deploying.** Disable proxying (e.g. Cloudflare's orange cloud) — TCP traffic must pass through directly:

| Type  | Name                         | Value                                 |
| ----- | ---------------------------- | ------------------------------------- |
| TXT   | `_cpln.your-domain.com`      | your Control Plane org name or org ID |
| CNAME | `@`                          | `{gvcAlias}.cpln.app`                 |
| CNAME | `_acme-challenge`            | `_acme-challenge.cpln.app`            |
| CNAME | `{clusterName}-0-{location}` | `{gvcAlias}.cpln.app`                 |
| CNAME | `{clusterName}-1-{location}` | `{gvcAlias}.cpln.app`                 |
| CNAME | `{clusterName}-N-{location}` | `{gvcAlias}.cpln.app`                 |

Add one CNAME per broker replica. The `_acme-challenge` record is required for Control Plane to issue the TLS certificate via DNS-01. Your GVC alias is visible under GVC settings in the Control Plane console.

### Configuration

```yaml theme={null}
redpanda:
  listeners:
    kafka:
      external:
        directReplicaRouting:
          containerPort: 9094
          publicAddress: your-domain.com
```

### Connecting Externally

Each broker advertises its own subdomain in the format `{clusterName}-{ordinal}-{location}.{domain}`. Use all broker addresses as the bootstrap list:

```bash theme={null}
rpk topic list \
  -X brokers=cluster-0-aws-us-east-1.your-domain.com:9094,cluster-1-aws-us-east-1.your-domain.com:9094,cluster-2-aws-us-east-1.your-domain.com:9094 \
  -X tls.enabled=true \
  -X sasl.mechanism=SCRAM-SHA-256 \
  -X user=admin \
  -X pass=your-admin-password
```

For Kafka clients:

```properties theme={null}
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-256
bootstrap.servers=cluster-0-aws-us-east-1.your-domain.com:9094,cluster-1-aws-us-east-1.your-domain.com:9094,cluster-2-aws-us-east-1.your-domain.com:9094
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="admin" \
  password="your-admin-password";
```

## External References

<CardGroup cols={2}>
  <Card title="Redpanda Documentation" href="https://docs.redpanda.com/" icon="book">
    Official Redpanda documentation
  </Card>

  <Card title="Redpanda Console Documentation" href="https://docs.redpanda.com/current/console/" icon="display">
    Redpanda Console UI guide
  </Card>

  <Card title="rpk CLI Reference" href="https://docs.redpanda.com/current/reference/rpk/" icon="terminal">
    rpk command reference for managing Redpanda clusters
  </Card>

  <Card title="Schema Registry API" href="https://docs.redpanda.com/current/develop/http-proxy/" icon="code">
    Confluent-compatible Schema Registry and HTTP Proxy API reference
  </Card>
</CardGroup>