LiteLLM

Overview

LiteLLM is an OpenAI-compatible LLM gateway and proxy: a single endpoint in front of 100+ model providers, with virtual API keys, per-key and per-team spend tracking and budgets, rate limiting, and routing with fallbacks. This template deploys the stateless LiteLLM proxy tier (API and Admin UI on port 4000) backed by a bundled PostgreSQL for virtual keys, spend, teams, and budgets, and a bundled Sentinel-managed Redis for rate-limit counters and the response cache shared across proxy replicas. Clients call it exactly as they would call OpenAI — point any OpenAI SDK at the endpoint and swap in a LiteLLM key.

This template deploys into an existing GVC that you already have — it does not create or manage a GVC.

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-litellm`	LiteLLM proxy: OpenAI-compatible API and Admin UI on port `4000` (standard, `litellm.replicas` replicas)	—
Workload	`RELEASE_NAME-postgres`	Bundled PostgreSQL from the postgres template — virtual keys, spend, teams, and budgets (stateful, 1 replica)	—
Workload	`RELEASE_NAME-postgres-backup`	Cron job that dumps the database to object storage on a schedule	`postgres.backup.enabled: true`
Workload	`RELEASE_NAME-redis`	Bundled Redis from the redis template — shared rate-limit counters and response cache (stateful, `redis.redis.replicas` replicas)	`redis.enabled: true` (default)
Workload	`RELEASE_NAME-sentinel`	Redis Sentinel that monitors the Redis master as `mymaster` and promotes a replica on failure (stateful, `redis.sentinel.replicas` replicas)	`redis.enabled: true` (default)
Volume set	`RELEASE_NAME-pg-vs`	PostgreSQL data directory	—
Secret	`RELEASE_NAME-litellm-config`	Opaque secret the chart creates with the proxy’s `config.yaml` (model list and Redis cache block), mounted at `/app/config.yaml`	—
Secret	`RELEASE_NAME-redis-config`, `RELEASE_NAME-sentinel-config`	Redis and Sentinel configuration files created by the bundled redis template	`redis.enabled: true` (default)
Identity	`RELEASE_NAME-litellm-identity`	Bound to the proxy workload	—
Policy	`RELEASE_NAME-litellm-policy`	Grants that identity `reveal` on exactly the secrets it reads: the config secret, your keys and database credentials secrets, and your Redis password secret when auth is on	—
Identity	`RELEASE_NAME-pg-identity`	Bound to the bundled PostgreSQL (and its backup job)	—
Policy	`RELEASE_NAME-pg-policy`	Grants the PostgreSQL identity `reveal` on its credentials secret only	—
Identity	`RELEASE_NAME-redis-identity`, `RELEASE_NAME-sentinel-identity`	Bound to the Redis and Sentinel workloads	`redis.enabled: true` (default)
Policy	`RELEASE_NAME-redis-policy`, `RELEASE_NAME-sentinel-policy`	Grant those identities `reveal` on their configuration secrets	`redis.enabled: true` (default)

Your master key, salt key, and provider API keys, your database credentials secret and, when Redis auth is on, your Redis password secret are not created by the template — see Prerequisites.

Prerequisites

LiteLLM needs three kinds of credential that never pass through Helm values: the master key that gates the Admin API and UI and mints virtual keys, the salt key that encrypts provider keys stored in PostgreSQL, and one provider API key per entry in litellm.providerEnv. All of them live in a single dictionary secret you create before installing; the chart references it by name and grants the proxy identity reveal on exactly that secret. The bundled PostgreSQL’s credentials live in a second dictionary secret you also create before installing (a third holds the Redis password when Redis auth is on).

Create the dictionary secret

The keys LITELLM_MASTER_KEY and LITELLM_SALT_KEY are required and must start with sk-. Add one more entry for every environment-variable name listed in litellm.providerEnv (the default list is just OPENAI_API_KEY), holding that provider’s real API key:

cpln secret create-dictionary --name LITELLM_SECRET_NAME \
  --entry LITELLM_MASTER_KEY="sk-$(openssl rand -hex 24)" \
  --entry LITELLM_SALT_KEY="sk-$(openssl rand -hex 24)" \
  --entry OPENAI_API_KEY='YOUR-OPENAI-API-KEY'

Set secrets.name to this name. Secrets are org-level, so no GVC flag is involved.

Back the salt key up outside Control Plane

LITELLM_SALT_KEY is write-once: it encrypts every provider key stored in PostgreSQL, and changing it corrupts all of them. The master key can be rotated later; the salt key cannot once models or keys are stored. Keep a copy somewhere safe.

Create the database credentials secret

The bundled PostgreSQL initializes with it and the proxy connects with it. Set postgres.config.credentialsSecretName to its name (default my-litellm-db-credentials); secret names are org-wide, so give each LiteLLM release its own:

cpln secret create-dictionary --name my-litellm-db-credentials \
  --entry username=litellm \
  --entry password="$(openssl rand -hex 24)" \
  --entry database=litellm

All three values must be URL-safe (letters, digits, _): the proxy’s DATABASE_URL is assembled from them at container start. The snippet’s hex password is URL-safe. PostgreSQL reads the secret only when its data directory is first initialized — see Important Notes for how to change the password later.

Create the secrets before installing. Installing without one reports success, but the deployment then wedges silently — cpln logs returns nothing, and the only place the missing secret is named is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-litellm --gvc GVC_NAME -o yaml

The message reads

The secret LITELLM_SECRET_NAME no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed.

Creating the secret lets the deployment recover on its own; cpln workload force-redeployment RELEASE_NAME-litellm --gvc GVC_NAME skips the wait.

Everything else works with the defaults. Redis authentication needs a third dictionary secret holding the key password (see Bundled Redis), and scheduled database backups, which need a bucket and cloud account, are described under Backing Up.

Installation

Once both dictionary secrets exist, install from the marketplace registry, pointing secrets.name and postgres.config.credentialsSecretName at them:

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/litellm \
  --version 1.2.1 --gvc GVC_NAME \
  --set secrets.name=LITELLM_SECRET_NAME \
  --set postgres.config.credentialsSecretName=my-litellm-db-credentials

Installing a second LiteLLM release in the same org? Create a separate database credentials secret for it — secret names are org-wide. Other ways to install and manage the template:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

LiteLLM Proxy

litellm:
  image: ghcr.io/berriai/litellm-database:v1.93.0 # DB-bundled image — runs Prisma migrations on boot; pin a concrete tag, never main-stable/latest
  replicas: 1 # proven single-replica shape; set >=2 for the scaled/HA tier (keep redis on for shared rate limits)
  resources:
    minCpu: 250m
    maxCpu: 1000m
    minMemory: 1Gi
    maxMemory: 2Gi
  storeModelInDb: true # also manage models via the Admin UI (persisted in Postgres)
  modelList: # seeded into config.yaml; api_key values reference env-var NAMES, never literal keys
    - model_name: gpt-4o-mini
      litellm_params:
        model: openai/gpt-4o-mini
        api_key: os.environ/OPENAI_API_KEY
  providerEnv: # env-var NAMES exposed to the proxy from the prerequisite secret — each must be a key in it
    - OPENAI_API_KEY

Every api_key in modelList points at an environment variable (os.environ/NAME), and every such name must appear in both providerEnv and your prerequisite secret. Adding a provider is therefore three edits: a new modelList entry, its variable name in providerEnv, and the key itself in the secret.

Prerequisite Secret

secrets:
  name: my-litellm-secrets # dictionary secret with LITELLM_MASTER_KEY, LITELLM_SALT_KEY + every providerEnv key — MUST exist before install

Access

publicAccess:
  enabled: true # key-gated *.cpln.app endpoint (UI + API); false = internal-only
internalAccess:
  type: same-gvc # none | same-gvc | same-org | workload-list
  workloads: [] # used only with workload-list, e.g. //gvc/GVC_NAME/workload/WORKLOAD_NAME

`internalAccess.type`	Who can reach the proxy over the internal network
`none`	No workload
`same-gvc`	Every workload in the same GVC (default)
`same-org`	Every workload in the same org
`workload-list`	Only the workloads listed in `internalAccess.workloads`

The public endpoint is key-gated, not open: every request still needs a valid master or virtual key. A change to either access knob takes a couple of minutes to take effect after the upgrade reports success.

Bundled PostgreSQL

The database is the postgres template as a subchart, so every knob of that template is available under postgres.*. Its credentials come from the prerequisite database credentials secret you created (see Prerequisites).

postgres:
  image: postgres:18
  config:
    credentialsSecretName: my-litellm-db-credentials # PREREQUISITE dictionary secret (username, password, database) — must exist BEFORE install
  resources:
    minCpu: 250m
    maxCpu: 1000m
    minMemory: 512Mi
    maxMemory: 1Gi
  volumeset:
    capacity: 10 # GiB (minimum is 10)
  backup:
    enabled: false # scheduled pg_dumpall to object storage — see Backing Up
    schedule: "0 2 * * *" # daily at 02:00 UTC
    provider: aws # aws | gcp | minio

Bundled Redis

Redis is the redis template as a subchart: a Sentinel-managed master-replica set that the proxy reaches through Sentinel (master name mymaster), so a failover is transparent to it. It holds the rate-limit and budget counters and the response cache that all proxy replicas share.

redis:
  enabled: true # gates the dependency; false = in-memory per-replica rate limiting
  redis:
    image: redis:7.4
    replicas: 3
    auth:
      fromSecret:
        enabled: false # true = require AUTH (Redis, Sentinel and the proxy read the secret below)
        name: "" # prerequisite dictionary secret (key: password); used only when enabled
    resources:
      cpu: 200m
      memory: 256Mi
      minCpu: 80m
      minMemory: 128Mi
  sentinel:
    image: redis:7.4
    replicas: 3 # no `auth:` block — Sentinel auth (incl. a plaintext `auth.password`) is refused at render
    resources:
      cpu: 200m
      memory: 256Mi
      minCpu: 80m
      minMemory: 128Mi

Redis ships authless by default, with the same-GVC firewall as the boundary. To require AUTH, create a dictionary secret holding the key password before you install, then set redis.redis.auth.fromSecret.enabled: true and redis.redis.auth.fromSecret.name to its name. Redis, Sentinel and the proxy all read it:

cpln secret create-dictionary --name my-litellm-redis \
  --entry password="$(openssl rand -hex 24)"

Sentinel authentication must stay off — the proxy sends only the master password, and the chart refuses to render with any redis.sentinel.auth setting, including a plaintext redis.sentinel.auth.password. Redis keeps no password on disk, so changing it later follows Rotating the Redis Password.

The database credentials secret seeds PostgreSQL on first boot; editing it later does not change the database password. Uninstalling the release deletes the volume set; reinstall to reset.

Connecting

What	Address	Notes
Public API and Admin UI	`https://<canonical>.cpln.app`	Assigned automatically while `publicAccess.enabled` is `true`. Read it from `status.canonicalEndpoint` with `cpln workload get RELEASE_NAME-litellm --gvc GVC_NAME -o yaml`. The Admin UI is at `/ui`.
OpenAI-compatible base URL (internal)	`http://RELEASE_NAME-litellm.GVC_NAME.cpln.local:4000`	Reachable per `internalAccess.type`. Point other workloads’ OpenAI SDKs here (`/v1/chat/completions`, `/v1/models`, …).
Health	`/health/readiness`, `/health/liveliness`	The endpoints the readiness and liveness probes use, on the same host.
Master key	`LITELLM_MASTER_KEY` in your prerequisite secret	`cpln secret reveal LITELLM_SECRET_NAME -o yaml`. Send it as `Authorization: Bearer …`; it is also the Admin UI password.
PostgreSQL	`RELEASE_NAME-postgres.GVC_NAME.cpln.local:5432`	`username` / `password` / `database` from `cpln secret reveal SECRET_NAME -o yaml`, where `SECRET_NAME` is your `postgres.config.credentialsSecretName`.
Redis	`RELEASE_NAME-sentinel.GVC_NAME.cpln.local:26379` (Sentinel, master `mymaster`); `RELEASE_NAME-redis.GVC_NAME.cpln.local:6379`	Internal only. When auth is on, the password is the `password` key of the secret named by `redis.redis.auth.fromSecret.name` (`cpln secret reveal SECRET_NAME -o yaml`).

Verify the proxy answers with your master key — the response lists the models from modelList:

curl https://<canonical>.cpln.app/v1/models \
  -H "Authorization: Bearer YOUR-MASTER-KEY"

With publicAccess.enabled: false there is no public endpoint; reach the proxy from your machine through a tunnel instead, then use http://localhost:4000 (UI at http://localhost:4000/ui) in place of the canonical endpoint:

cpln port-forward RELEASE_NAME-litellm 4000:4000 --gvc GVC_NAME

First Run

The Admin UI is gated by the master key from the first request, so there is no setup wizard to race — but the master key should not be what your applications use.

Wait for the proxy to report ready

PostgreSQL comes up first, then the proxy runs its database migrations and starts serving. On a cold install the proxy can start before the database accepts connections, log a P1001 error, and restart once; it recovers on its own.

cpln workload get-deployments RELEASE_NAME-litellm --gvc GVC_NAME -o yaml

Open /ui on the canonical endpoint (or the port-forward URL). The username is admin and the password is your LITELLM_MASTER_KEY.

Mint a virtual key for each application

Create keys with their own budgets and rate limits under Virtual Keys in the UI, or with the API as shown in Managing Virtual Keys, and hand those out instead of the master key.

Point an OpenAI SDK at the proxy

Set the SDK’s base URL to the canonical endpoint (or the internal address for workloads in the same GVC) and its API key to a virtual key. The model names are the model_name values from modelList, plus any models added through the UI while litellm.storeModelInDb is true.

Operations

Managing Virtual Keys

Rather than handing out the master key, mint scoped virtual keys — each with its own budget, rate limits, and allowed models — and give those to your applications and teams. Spend is tracked per key in PostgreSQL.

curl -X POST https://<canonical>.cpln.app/key/generate \
  -H "Authorization: Bearer YOUR-MASTER-KEY" \
  -H "Content-Type: application/json" \
  -d '{"key_alias": "team-a", "models": ["gpt-4o-mini"], "max_budget": 10}'

The response includes a new sk-… key usable on /v1/chat/completions and /v1/models. Keys, teams, and budgets can also be managed visually in the Admin UI at /ui. Rate limits are enforced across all proxy replicas only while Redis is enabled; see Scaling and Availability.

Backing Up

Scheduled backups of the bundled PostgreSQL are off by default. Enabling postgres.backup.enabled: true adds one cron workload, RELEASE_NAME-postgres-backup, that runs pg_dumpall on the schedule and uploads a gzipped dump to your bucket under the configured prefix. Pick a postgres.backup.provider and complete the matching setup before installing or upgrading with it on.

AWS S3
Google Cloud Storage
MinIO / S3-compatible

postgres:
  backup:
    enabled: true
    provider: aws
    aws:
      bucket: my-postgres-bucket
      region: us-east-1
      cloudAccountName: my-s3-cloud-account
      policyName: my-postgres-backup-policy # bucket-scoped IAM policy
      prefix: postgres/backups

Create a bucket

Create an S3 bucket. Set postgres.backup.aws.bucket and postgres.backup.aws.region to match.

Set up a cloud account

If you do not have one, create a cloud account for the AWS account holding the bucket. Set postgres.backup.aws.cloudAccountName to its name.

Create a bucket-scoped IAM policy

Create an IAM policy with the JSON below (replace YOUR_BUCKET_NAME) and set postgres.backup.aws.policyName to its name:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket",
        "s3:GetObjectVersion",
        "s3:DeleteObjectVersion",
        "s3:GetBucketLocation",
        "s3:AbortMultipartUpload",
        "s3:ListBucketMultipartUploads",
        "s3:ListMultipartUploadParts"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_NAME",
        "arn:aws:s3:::YOUR_BUCKET_NAME/*"
      ]
    }
  ]
}

postgres:
  backup:
    enabled: true
    provider: gcp
    gcp:
      bucket: my-postgres-bucket
      cloudAccountName: my-gcs-cloud-account
      prefix: postgres/backups

Create a bucket

Create a GCS bucket. Set postgres.backup.gcp.bucket to its name.

Set up a cloud account

Create a cloud account for the GCP project holding the bucket and set postgres.backup.gcp.cloudAccountName to its name.

Grant storage access

Grant the cloud account’s service account the Storage Admin (roles/storage.admin) role on the project. The chart additionally binds the identity to roles/storage.objectAdmin on exactly the bucket in postgres.backup.gcp.bucket.

postgres:
  backup:
    enabled: true
    provider: minio
    minio:
      endpoint: http://my-minio-workload:9000 # S3 API address including the port
      bucket: my-postgres-bucket
      credentialsSecretName: my-postgres-minio-credentials # dictionary secret with accessKey + secretKey
      prefix: postgres/backups

Create a bucket

Create the bucket on your server. Set postgres.backup.minio.bucket to its name and postgres.backup.minio.endpoint to the S3 API address including the port — for the MinIO template in the same GVC, http://WORKLOAD_NAME.GVC_NAME.cpln.local:9000.

Create the credentials secret

Create a dictionary secret with the server’s keys and set postgres.backup.minio.credentialsSecretName to its name. No cloud account is involved.

cpln secret create-dictionary --name BACKUP_MINIO_SECRET_NAME \
  --entry accessKey=YOUR-ACCESS-KEY \
  --entry secretKey='YOUR-SECRET-KEY'

A zero-length backup object is a failed run, not a backup. If the dump cannot reach the database, the upload still writes a near-empty gzip under a normal timestamped filename. Check the object size before you rely on one.

The database dump covers virtual keys, spend, teams, budgets, and any models added through the UI. It does not cover your prerequisite secret — the salt key in particular is needed to read the provider keys stored in that dump, so keep it backed up alongside. Redis holds only counters and cache and is not backed up.

Restoring a Backup

Each backup is a gzipped plain-SQL pg_dumpall archive. Restoring means replaying that SQL into the bundled PostgreSQL. The one transport that carries a dump into the container intact is cpln workload exec with --stdin and a base64 wrapper on both ends — raw binary piped through exec is corrupted:

gunzip -c BACKUP_FILE.sql.gz | base64 | cpln workload exec RELEASE_NAME-postgres --gvc GVC_NAME \
  --container postgresql --stdin --quiet -- sh -c 'base64 -d | psql -U "$POSTGRES_USER" -d postgres'

Two constraints make this a planned operation: the archive contains CREATE ROLE and CREATE DATABASE statements, so a straight replay collides with the database the proxy already created and migrated on first boot; and no proxy replica may be connected while you drop and recreate that database before the replay. The restored data is only readable with the same LITELLM_SALT_KEY the dump was taken under.

This restore path has not been exercised against this template end to end. Rehearse it on a throwaway release before you depend on it. Restoring from a volume set snapshot is likewise unverified here.

Upgrading from 1.0.0

Version 1.1.0 adopted the postgres 3.4.1 template, which stopped taking database credentials as values. Three keys were removed:

Removed key (1.0.0)	Replacement (1.2.0 and later)
`postgres.config.username`	`username` key in the database credentials secret named by `postgres.config.credentialsSecretName`
`postgres.config.password`	`password` key in the same secret
`postgres.config.database`	`database` key in the same secret

Put the values your database was initialized with into the database credentials secret under Prerequisites, set postgres.config.credentialsSecretName to its name, and remove the three keys. A values file that still carries an old key fails the render with the postgres template’s message (config.username was REMOVED in postgres 3.4.0).

The first helm upgrade after an install can restart the bundled PostgreSQL and Redis even when their values are unchanged, so expect a brief interruption.

Upgrading from 1.1.0

Version 1.2.0 removed postgres.credentials: the database credentials are now a prerequisite secret, the same one the postgres template requires on its own. Carrying the old key forward fails the render. 1.1.0 created the database secret itself, so this upgrade deletes it. Before upgrading, create a secret under a new name holding the values your database was initialized with — not new ones, or the proxy loses its database connection. Read your current values from the old secret (named by postgres.config.credentialsSecretName in the installed release, my-litellm-db-credentials by default) before the upgrade deletes it:

cpln secret reveal my-litellm-db-credentials -o yaml
cpln secret create-dictionary --name RELEASE_NAME-postgres-credentials \
  --entry username=litellm --entry password='YOUR_CURRENT_PASSWORD' --entry database=litellm

(litellm / litellm were the 1.1.0 defaults; use your own values if you overrode them.) If you never overrode the password, it is literally change-me-litellm-pg — put that in the new secret, and rotate it afterwards as described under Important Notes. Then upgrade with postgres.config.credentialsSecretName set to the new name and postgres.credentials removed. The old RELEASE_NAME-litellm-db secret is deleted too; nothing replaces it. The upgrade restarts the bundled PostgreSQL too (it now reads the new secret), so LiteLLM briefly loses its database; it reconnects by itself. The new version of a workload can also wait for several minutes, with get-deployments reporting that it is not allowed to reveal the secret, while access to the new secret propagates; the previous version keeps serving meanwhile, and it clears by itself. If you ran 1.1.0 with Redis auth on, redis.redis.auth.password was removed too (carrying it forward fails the render), and the chart-created RELEASE_NAME-redis-auth-password secret is deleted by the upgrade. Before upgrading, copy your current Redis password into a secret under a new name, keeping the same value so the replicas keep replicating while they restart:

cpln secret reveal RELEASE_NAME-redis-auth-password -o yaml
cpln secret create-dictionary --name RELEASE_NAME-litellm-redis \
  --entry password='YOUR_CURRENT_REDIS_PASSWORD'

Then upgrade with redis.redis.auth.fromSecret.enabled: true, redis.redis.auth.fromSecret.name set to the new name, and redis.redis.auth.password removed. With Redis auth off in 1.1.0 (the default), just remove redis.redis.auth.password from your values. Version 1.2.1 also refuses any redis.sentinel.auth setting, so remove that block too if your values carry one.

Rotating the Redis Password

Only with Redis auth on. Redis keeps no password on disk, but a running Sentinel holds the one it started with, so a new password is applied live in three places before the proxy restarts. Each Redis node first accepts both passwords, which keeps replication and every connected client working throughout. rx and sx run a command on one Redis node or Sentinel (LOCATION is your GVC’s location; nodes and Sentinels are numbered from 0 up to redis.redis.replicas and redis.sentinel.replicas minus one):

rx() { cpln workload exec RELEASE_NAME-redis --gvc GVC_NAME --location LOCATION \
  --container redis --replica "RELEASE_NAME-redis-$1" -- "${@:2}"; }
sx() { cpln workload exec RELEASE_NAME-sentinel --gvc GVC_NAME --location LOCATION \
  --container sentinel --replica "RELEASE_NAME-sentinel-$1" -- "${@:2}"; }
# 1. On EVERY Redis node: accept the new password alongside the old one, and replicate with it
rx 0 redis-cli --no-auth-warning -a OLD_PASSWORD ACL SETUSER default ">NEW_PASSWORD"
rx 0 redis-cli --no-auth-warning -a OLD_PASSWORD CONFIG SET masterauth NEW_PASSWORD
# 2. On EVERY Sentinel
sx 0 redis-cli -p 26379 SENTINEL SET mymaster auth-pass NEW_PASSWORD
# 3. Put the new password in the secret and read it back
cpln secret edit SECRET_NAME
cpln secret reveal SECRET_NAME -o yaml
# 4. Restart the proxy, which keeps the old password until it restarts
cpln workload force-redeployment RELEASE_NAME-litellm --gvc GVC_NAME
# 5. Once get-deployments shows a higher version, alone and ready: retire the old password on EVERY Redis node
rx 0 redis-cli --no-auth-warning -a NEW_PASSWORD ACL SETUSER default "<OLD_PASSWORD"

The proxy is briefly unavailable while it restarts (with litellm.replicas of 2 or more it rolls one replica at a time). Redis and Sentinel need no restart: when they next start they read the new password from the secret.

Scaling and Availability

litellm.replicas sets how many proxy replicas run. The proxy is stateless — all state lives in PostgreSQL and Redis — so replicas need no peer discovery and a rolling restart cycles one at a time. Set it to 2 or more for an always-on tier:

litellm:
  replicas: 2
redis:
  enabled: true

Keep redis.enabled: true whenever litellm.replicas is above 1. Rate-limit and budget counters are shared only through Redis; without it each replica counts in memory, so the effective limit becomes the configured value multiplied by the replica count.

The bundled Redis is already highly available: three Redis replicas with three Sentinels, and the proxy connects through Sentinel so a master failover needs no reconfiguration. The bundled PostgreSQL is a single instance on a persistent volume set, which survives restarts, redeploys, and upgrades under the same release name. Scale-out proxy replicas run the boot-time database migration as a no-op.

Troubleshooting

Deployment never becomes ready and the logs are empty

Symptom: cpln helm install reported success, cpln logs returns nothing, and cpln workload get-deployments RELEASE_NAME-litellm --gvc GVC_NAME -o yaml shows

The secret LITELLM_SECRET_NAME no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed.

under status.versions[].message.Cause: The prerequisite dictionary secret named by secrets.name does not exist, or it is missing one of the keys the workload references (LITELLM_MASTER_KEY, LITELLM_SALT_KEY, or a name listed in litellm.providerEnv). The database credentials secret named by postgres.config.credentialsSecretName, or with Redis auth on the secret named by redis.redis.auth.fromSecret.name, may also be missing.Fix: Create the secret with every required key (see Prerequisites). The deployment recovers on its own once the secret exists; cpln workload force-redeployment RELEASE_NAME-litellm --gvc GVC_NAME skips the wait.

Readiness never passes after the secret exists

Symptom: The proxy container starts but /health/readiness keeps failing and the deployment does not reach ready.Cause: The readiness check is database-aware. The proxy cannot reach the bundled PostgreSQL, or the migration it runs on boot has not completed.Fix: Check that RELEASE_NAME-postgres is ready, then read the proxy’s logs:

cpln logs '{gvc="GVC_NAME", workload="RELEASE_NAME-litellm"} |= "ERROR"' --limit 50 --since 10m

A single P1001 (cannot reach database server) followed by one restart on a cold install is normal and recovers on its own.

Render fails because the Redis password secret name is required

Symptom: litellm: redis.redis.auth.fromSecret.name is required when redis.redis.auth.fromSecret.enabled is true.Cause: redis.redis.auth.fromSecret.enabled is true with an empty redis.redis.auth.fromSecret.name.Fix: Create the Redis password secret and set redis.redis.auth.fromSecret.name to its name (see Bundled Redis), or set enabled back to false.

Render fails because Sentinel auth must stay disabled

Symptom:

litellm: redis.sentinel.auth must stay disabled — LiteLLM sends only the Redis master password, not a Sentinel password

.Cause: Your values carry a redis.sentinel.auth setting — redis.sentinel.auth.password.enabled, a plaintext redis.sentinel.auth.password.value, or redis.sentinel.auth.fromSecret.enabled. The proxy authenticates to the Redis master only, so a Sentinel password would lock it out.Fix: Remove redis.sentinel.auth from your values. Use redis.redis.auth.fromSecret to protect the master; the same-GVC firewall is the boundary for Sentinel.

Upgrade from 1.0.0 fails asking you to create a dictionary secret

Symptom: The render fails with config.username was REMOVED in postgres 3.4.0 and the postgres template’s advice to create a prerequisite dictionary secret.Cause: Your values file still carries postgres.config.username, postgres.config.password, or postgres.config.database.Fix: Put their values into the database credentials secret and remove the keys, as described in Upgrading from 1.0.0.

Upgrade from 1.1.0 fails because postgres.credentials or redis.redis.auth.password was removed

Symptom: The render fails with postgres.credentials was removed in litellm 1.2.0 or redis.redis.auth.password was removed in litellm 1.2.0.Cause: Your values file still carries the 1.1.0 plaintext database credentials or Redis password.Fix: Create the prerequisite secrets holding your current values and remove the old keys, as described in Upgrading from 1.1.0.

Requests are rejected with 401 on the public endpoint

Symptom: API calls return 401, including on the public endpoint.Cause: The endpoint is key-gated. The request carries no Authorization: Bearer header, or the key is not the master key or a valid virtual key for the requested model.Fix: Send the master key or a virtual key minted for that model (see Managing Virtual Keys). Compare the master key you are sending with cpln secret reveal LITELLM_SECRET_NAME -o yaml.

Important Notes

Create every prerequisite secret before installing. secrets.name must be a dictionary secret with LITELLM_MASTER_KEY, LITELLM_SALT_KEY, and every litellm.providerEnv key; postgres.config.credentialsSecretName one with username, password and database; and, with Redis auth on, redis.redis.auth.fromSecret.name one with password. A missing one wedges the deployment silently.
Never rotate LITELLM_SALT_KEY. It encrypts the provider keys stored in PostgreSQL; changing it corrupts every stored key. The master key may be rotated — force a redeployment of RELEASE_NAME-litellm afterwards so the proxy picks up the new value.
Hand out virtual keys, not the master key. The public endpoint is key-gated, and virtual keys carry their own budgets, rate limits, and model lists.
Keep Redis enabled for litellm.replicas of 2 or more. Otherwise rate limits are counted per replica.
Redis is authless by default, bounded by the same-GVC firewall. To require a master password, create the Redis password secret and set redis.redis.auth.fromSecret.enabled: true with its name; changing it later follows Rotating the Redis Password. Sentinel auth must stay off.
Give each release its own database credentials secret. Secret names are org-wide.
Rotating the database password needs ALTER ROLE plus a forced redeployment of the proxy (not the database). PostgreSQL keeps the password it was initialized with, and a running workload keeps the values it started with. Run ALTER ROLE DB_USERNAME PASSWORD 'NEW_PASSWORD'; first, then cpln secret edit SECRET_NAME, then right away cpln workload force-redeployment RELEASE_NAME-litellm --gvc GVC_NAME, and wait until get-deployments shows a higher version number, alone and ready. Connections the proxy already has open keep working after ALTER ROLE; new ones fail until its redeploy finishes, so rotate in a quiet window.
PostgreSQL data survives proxy restarts and redeploys. To reset virtual keys and spend, uninstall the whole release — that deletes the volume set.

External References

LiteLLM Documentation

Official LiteLLM documentation

Proxy Deployment

LiteLLM proxy deployment reference

Virtual Keys

Create and manage virtual keys, budgets, and rate limits

Config Reference

Full proxy config.yaml settings reference

LiteLLM Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

What Gets Created

Prerequisites

Installation

UI

CLI

Terraform

Pulumi

Configuration

LiteLLM Proxy

Prerequisite Secret

Access

Bundled PostgreSQL

Bundled Redis

Connecting

First Run

Operations

Managing Virtual Keys

Backing Up

Restoring a Backup

Upgrading from 1.0.0

Upgrading from 1.1.0

Rotating the Redis Password

Scaling and Availability

Troubleshooting

Important Notes

External References

LiteLLM Documentation

Proxy Deployment

Virtual Keys

Config Reference

LiteLLM Template

​Overview

​What Gets Created

​Prerequisites

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​LiteLLM Proxy

​Prerequisite Secret

​Access

​Bundled PostgreSQL

​Bundled Redis

​Connecting

​First Run

​Operations

​Managing Virtual Keys

​Backing Up

​Restoring a Backup

​Upgrading from 1.0.0

​Upgrading from 1.1.0

​Rotating the Redis Password

​Scaling and Availability

​Troubleshooting

​Important Notes

​External References

LiteLLM Documentation

Proxy Deployment

Virtual Keys

Config Reference

LiteLLM Template

Overview

What Gets Created

Prerequisites

Installation

Configuration

LiteLLM Proxy

Prerequisite Secret

Access

Bundled PostgreSQL

Bundled Redis

Connecting

First Run

Operations

Managing Virtual Keys

Backing Up

Restoring a Backup

Upgrading from 1.0.0

Upgrading from 1.1.0

Rotating the Redis Password

Scaling and Availability

Troubleshooting

Important Notes

External References