> ## Documentation Index
> Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LiteLLM

> Deploy LiteLLM on Control Plane using the Template Catalog. An OpenAI-compatible LLM gateway and proxy — one endpoint in front of 100+ providers with virtual API keys, per-key and per-team spend tracking, budgets, rate limiting, and routing — backed by PostgreSQL and Redis. Covers the prerequisite secret, model list, Redis auth, scaling, and access.

## Overview

LiteLLM is an OpenAI-compatible LLM gateway and proxy: a single endpoint in front of 100+ model providers, with virtual API keys, per-key and per-team spend tracking and budgets, rate limiting, and routing with fallbacks. This template deploys the stateless LiteLLM proxy tier backed by PostgreSQL (virtual keys, spend, teams, budgets) and Redis (shared rate-limit counters and response cache). Clients call it exactly as they would call OpenAI — point any OpenAI SDK at the endpoint and swap in a LiteLLM key.

### Architecture

* **LiteLLM proxy** — A stateless `standard` workload serving the OpenAI-compatible API and Admin UI on port `4000`. Runs a single replica by default; set `litellm.replicas` to `2` or more for an always-on tier with zero-downtime rolling restarts. All state lives in PostgreSQL and Redis, so replicas are independent — no clustering.
* **PostgreSQL** ([postgres](/template-catalog/templates/postgres) subchart) — The durable store for virtual keys, spend, teams, and budgets. The DB-bundled proxy image runs its Prisma migrations automatically on boot.
* **Redis + Sentinel** ([redis](/template-catalog/templates/redis) subchart, optional via `redis.enabled`) — Shared rate-limit / budget counters and response cache across proxy replicas.

### What Gets Created

* **Standard LiteLLM Workload** — One or more stateless replicas serving the API and Admin UI on port `4000`.
* **Stateful PostgreSQL Workload** — Single-replica Postgres with a persistent volume set for keys and spend.
* **Redis Workloads** *(optional)* — A Redis master-replica set with Redis Sentinel for rate limits and cache.
* **Config Secret** — The model/provider list mounted into the proxy as `config.yaml`.
* **DB-credentials Secret** — Assembles the proxy's `DATABASE_URL`.
* **Identity & Policy** — An identity bound to the proxy with least-privilege `reveal` on exactly its own secrets and your prerequisite secret.

<Note>
  This template does not create a GVC. You must deploy it into an existing GVC.
</Note>

## Prerequisites

**You must create a dictionary secret before installing.** The proxy references it by name (`secrets.name`, default `my-litellm-secrets`) and will not become healthy until it exists — a missing secret wedges the deployment and makes it look broken. Create it first, in the same GVC, as a **dictionary** secret containing:

* `LITELLM_MASTER_KEY` — the admin key (format `sk-...`); gates the Admin API/UI and mints virtual keys.
* `LITELLM_SALT_KEY` — encrypts provider keys stored in PostgreSQL. **Write-once — never rotate it** (rotating corrupts every stored provider key).
* one key per name in `litellm.providerEnv` (default: `OPENAI_API_KEY`) — the actual provider API key.

Create it with the CLI (add one entry per provider key you use):

```bash theme={null}
cpln secret create-dictionary --name my-litellm-secrets \
  --entry LITELLM_MASTER_KEY=sk-$(openssl rand -hex 24) \
  --entry LITELLM_SALT_KEY=sk-$(openssl rand -hex 24) \
  --entry OPENAI_API_KEY=sk-...your-openai-key...
```

Install the template using your preferred method:

<CardGroup cols={2}>
  <Card title="UI" href="/template-catalog/install-manage/ui" icon="laptop">
    Browse, install, and manage templates visually
  </Card>

  <Card title="CLI" href="/template-catalog/install-manage/cli" icon="terminal">
    Manage templates from your terminal
  </Card>

  <Card title="Terraform" href="/template-catalog/install-manage/terraform" icon={<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128"><g fill-rule="evenodd"><path d="M77.941 44.5v36.836L46.324 62.918V26.082zm0 0" fill="#5c4ee5"/><path d="M81.41 81.336l31.633-18.418V26.082L81.41 44.5zm0 0" fill="#4040b2"/><path d="M11.242 42.36L42.86 60.776V23.941L11.242 5.523zm0 0M77.941 85.375L46.324 66.957v36.82l31.617 18.418zm0 0" fill="#5c4ee5"/></g></svg>}>
    Declare templates in your Terraform configurations
  </Card>

  <Card
    title="Pulumi"
    href="/template-catalog/install-manage/pulumi"
    icon={<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" id="Pulumi-Icon--Streamline-Svg-Logos" height="24" width="24">
    <desc>
        Pulumi Icon Streamline Icon: https://streamlinehq.com
    </desc>
    <path fill="#f26e7e" d="M4.683025 13.3318c0.869125 -0.5018 0.870575 -2.1264 0.003225 -3.62865s-2.27504 -2.313275 -3.1441725 -1.811475C0.672945 8.3935 0.6715 10.0181 1.53885 11.52035c0.86735 1.502275 2.27505 2.313275 3.144175 1.81145Zm0.0052 3.2167c0.86735 1.502275 0.865925 3.126875 -0.003225 3.628675 -0.86915 0.5018 -2.2768275 -0.309225 -3.144175 -1.81145 -0.8673525 -1.50225 -0.8659075 -3.126875 0.003225 -3.628675 0.8691325 -0.5018 2.276825 0.309225 3.144175 1.81145Zm5.922875 3.4243c0.86735 1.50225 0.8659 3.126775 -0.003225 3.62875 -0.869125 0.501775 -2.27685 -0.309325 -3.1442 -1.81155 -0.867325 -1.50225 -0.865875 -3.12685 0.00325 -3.628675 0.869125 -0.5018 2.276825 0.309225 3.144175 1.811475Zm-0.001925 -6.845275c0.86735 1.50225 0.8659 3.12685 -0.003225 3.628675 -0.869125 0.5018 -2.276825 -0.309225 -3.144175 -1.811475 -0.86735 -1.50225 -0.8659 -3.12685 0.003225 -3.62865 0.869125 -0.501825 2.276825 0.3092 3.144175 1.81145Z" stroke-width="0.25"></path>
    <path fill="#8a3391" d="M22.45775 11.524125c0.86725 -1.502225 0.865925 -3.12685 -0.003225 -3.62865 -0.869125 -0.501825 -2.276825 0.3092 -3.144175 1.811475 -0.86735 1.50225 -0.8659 3.126825 0.003225 3.62865 0.869125 0.501825 2.276825 -0.3092 3.144175 -1.811475Zm0.000175 3.2151c0.869075 0.5018 0.870625 2.1264 0.003225 3.62865 -0.86735 1.50225 -2.27505 2.313275 -3.144175 1.81145 -0.869125 -0.5018 -0.870575 -2.126425 -0.003225 -3.62865 0.86735 -1.50225 2.27505 -2.313275 3.144175 -1.81145ZM16.536225 18.157875c0.86915 0.501825 0.8706 2.126425 0.00325 3.628675 -0.86735 1.502125 -2.275075 2.313225 -3.1442 1.81145 -0.869125 -0.50175 -0.870575 -2.126425 -0.003225 -3.62865 0.867375 -1.502275 2.27505 -2.3133 3.144175 -1.811475Zm-0.003325 -6.843775c0.869125 0.5018 0.870575 2.126425 0.003225 3.628675s-2.27505 2.313275 -3.1442 1.811475c-0.869125 -0.501825 -0.870575 -2.126425 -0.003225 -3.628675 0.86735 -1.502275 2.27505 -2.313275 3.1442 -1.811475Z" stroke-width="0.25"></path>
    <path fill="#f7bf2a" d="M15.138225 2.06721c0 1.003615 -1.40625 1.817215 -3.14095 1.817215 -1.7347 0 -3.14095 -0.8136 -3.14095 -1.817215C8.856325 1.06359 10.262575 0.25 11.997275 0.25c1.7347 0 3.14095 0.81359 3.14095 1.81721ZM9.2166 5.482375c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172s1.40625 -1.817225 3.14095 -1.817225c1.7347 0 3.14095 0.8136 3.14095 1.817225Zm8.71005 1.8172c1.7347 0 3.14095 -0.813575 3.14095 -1.8172s-1.40625 -1.817225 -3.14095 -1.817225c-1.7347 0 -3.14095 0.8136 -3.14095 1.817225s1.40625 1.8172 3.14095 1.8172Zm-2.788425 1.605625c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172 0 -1.0036 1.40625 -1.8172 3.14095 -1.8172 1.7347 0 3.14095 0.8136 3.14095 1.8172Z" stroke-width="0.25"></path>
    </svg>}
  >
    Declare templates in your Pulumi programs
  </Card>
</CardGroup>

## Configuration

The default `values.yaml` for this template:

```yaml theme={null}
litellm:
  # DB-bundled image variant — runs Prisma `migrate deploy` on boot.
  image: ghcr.io/berriai/litellm-database:v1.93.0
  replicas: 1                 # 1 = proven single-replica; 2+ = always-on, zero-downtime restarts
  resources:
    minCpu: 250m
    maxCpu: 1000m
    minMemory: 1Gi
    maxMemory: 2Gi
  storeModelInDb: true        # also manage models via the Admin UI (persisted in Postgres)
  # Model/provider list seeded into config.yaml. api_key values reference env vars
  # supplied via the prerequisite secret (never inline real keys here).
  modelList:
    - model_name: gpt-4o-mini
      litellm_params:
        model: openai/gpt-4o-mini
        api_key: os.environ/OPENAI_API_KEY
  # Env-var NAMES (non-sensitive) exposed to the proxy from the prerequisite secret.
  # Every name here must exist as a key in the secrets.name dictionary secret.
  providerEnv:
    - OPENAI_API_KEY

# Prerequisite dictionary secret — MUST EXIST BEFORE INSTALL (see Prerequisites).
secrets:
  name: my-litellm-secrets

publicAccess:
  enabled: true               # key-gated *.cpln.app endpoint (UI + API); false = internal-only
internalAccess:
  type: same-gvc              # options: none, same-gvc, same-org, workload-list
  workloads: []               # used only with workload-list

# PostgreSQL — virtual keys / spend / budgets
postgres:
  image: postgres:18
  config:
    username: litellm
    password: change-me-litellm-pg     # change before installing
    database: litellm
  resources:
    minCpu: 250m
    maxCpu: 1000m
    minMemory: 512Mi
    maxMemory: 1Gi
  volumeset:
    capacity: 10              # initial capacity in GiB (minimum is 10)

# Redis (Sentinel HA) — rate limits / cache
redis:
  enabled: true               # false = in-memory per-replica rate limiting (not shared)
  redis:
    image: redis:7.4
    replicas: 3
    auth:
      password:
        enabled: false        # true = require AUTH; wires the master password into LiteLLM's cache config
        value: change-me-litellm-redis
  sentinel:
    image: redis:7.4
    replicas: 3
```

### LiteLLM proxy

* `litellm.image` — The DB-bundled LiteLLM proxy image (`litellm-database`), which runs Prisma migrations on boot. Pin a concrete tag.
* `litellm.replicas` — Number of stateless proxy replicas. `1` is the proven single-replica shape; `2` or more gives an always-on tier where rolling restarts cycle one replica at a time with no downtime. Keep Redis enabled at `2`+ so rate limits are shared across replicas.
* `litellm.resources` — CPU and memory per replica.
* `litellm.storeModelInDb` — When `true`, models added through the Admin UI/API are persisted in PostgreSQL alongside the static `modelList`.
* `litellm.modelList` — The model/provider list seeded into the mounted `config.yaml`. Each entry's `api_key` references an environment variable name (`os.environ/OPENAI_API_KEY`) — never a literal key.
* `litellm.providerEnv` — The environment-variable **names** exposed to the proxy from the prerequisite secret. Every name here must exist as a key in that dictionary secret.

### Prerequisite secret

* `secrets.name` — Name of the user-created dictionary secret holding `LITELLM_MASTER_KEY`, `LITELLM_SALT_KEY`, and every `providerEnv` key. It must exist **before** install — see [Prerequisites](#prerequisites).

### Access

* `publicAccess.enabled` — Serve the API and Admin UI on the canonical `*.cpln.app` HTTPS endpoint (default). The endpoint is **key-gated** — a valid master or virtual key is always required. Set to `false` for an internal-only instance.
* `internalAccess.type` — Internal firewall scope of the proxy workload:

| Type            | Description                                                            |
| --------------- | ---------------------------------------------------------------------- |
| `none`          | No internal access.                                                    |
| `same-gvc`      | Allow access from all workloads in the same GVC (default).             |
| `same-org`      | Allow access from all workloads in the same organization.              |
| `workload-list` | Allow access only from workloads listed in `internalAccess.workloads`. |

### PostgreSQL

* `postgres.config.password` — The bundled database password. **Change it before installing.**
* `postgres.volumeset.capacity` — Initial volume size in GiB (minimum 10).

PostgreSQL stores all virtual keys, spend, teams, and budgets.

### Redis

* `redis.enabled` — When `true` (default), deploys Redis + Sentinel for shared rate-limit counters and response cache. When `false`, no Redis is created and each proxy replica rate-limits in-memory (not shared across replicas).
* `redis.redis.auth.password.enabled` — When `false` (default), Redis runs authless with the same-GVC firewall as the boundary. Set to `true` to require AUTH; the master password is then wired into the proxy's cache config. Both paths are tested end to end, including Sentinel monitoring the auth-protected master.
* `redis.redis.replicas` / `redis.sentinel.replicas` — Redis and Sentinel replica counts.

## Connecting

| Target                  | Address                                                     | Credentials                                    |
| ----------------------- | ----------------------------------------------------------- | ---------------------------------------------- |
| Public API + Admin UI   | `https://<canonical>.cpln.app` (UI at `/ui`)                | `LITELLM_MASTER_KEY` (Bearer token / UI login) |
| Internal (same GVC)     | `http://{release}-litellm.{gvc}.cpln.local:4000`            | `LITELLM_MASTER_KEY`                           |
| OpenAI-compatible calls | `POST /chat/completions` with `Authorization: Bearer <key>` | master key or a minted virtual key             |

The canonical `*.cpln.app` hostname appears under `status.canonicalEndpoint` (`cpln workload get {release}-litellm -o yaml`). Point any OpenAI SDK at this base URL and pass a LiteLLM key.

## Managing Virtual Keys

Rather than handing out the master key, mint scoped virtual keys — each with its own budget and rate limits — and give those to your applications and teams. Spend is tracked per key in PostgreSQL.

```bash theme={null}
curl -X POST https://<canonical>.cpln.app/key/generate \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"key_alias": "team-a", "max_budget": 10}'
```

The response includes a new `sk-...` key usable on `/chat/completions` and `/v1/models`. Manage keys, teams, and budgets visually in the Admin UI at `/ui`.

## Important Notes

* **The prerequisite secret must exist before install.** `secrets.name` is a dictionary secret with `LITELLM_MASTER_KEY`, `LITELLM_SALT_KEY`, and every `providerEnv` key. A missing secret wedges the deployment.
* **`LITELLM_SALT_KEY` is write-once — never rotate it.** It encrypts stored provider keys; changing it corrupts every stored key. The master key may be rotated.
* **The endpoint is key-gated, not open.** Public access still requires a valid master or virtual key — hand out virtual keys with budgets and limits rather than the master key.
* **Keep Redis enabled for `replicas` ≥ 2.** Without it, each replica rate-limits in-memory, so the effective limit becomes N× the configured value.
* **Redis ships authless by default** (same-GVC firewall is the boundary). Set `redis.redis.auth.password.enabled: true` to require AUTH; the master password is then wired into the proxy's cache config.
* **PostgreSQL data survives reinstalls of the proxy** — to reset virtual keys and spend you must also reinstall the database (which deletes its volume set).
* **First install self-heals a brief DB-timing gap.** On a cold install the proxy can start before PostgreSQL accepts connections and log a `P1001` error with one restart; it recovers automatically once the database is ready (about 1.5–2 minutes to healthy). No action needed.

## External References

<CardGroup cols={2}>
  <Card title="LiteLLM Documentation" icon="book" href="https://docs.litellm.ai/">
    Official LiteLLM documentation
  </Card>

  <Card title="Proxy Deployment" icon="server" href="https://docs.litellm.ai/docs/proxy/deploy">
    LiteLLM proxy deployment reference
  </Card>

  <Card title="Virtual Keys" icon="key" href="https://docs.litellm.ai/docs/proxy/virtual_keys">
    Create and manage virtual keys, budgets, and rate limits
  </Card>

  <Card title="Config Reference" icon="sliders" href="https://docs.litellm.ai/docs/proxy/config_settings">
    Full proxy config.yaml settings reference
  </Card>

  <Card title="LiteLLM Template" icon="github" href="https://github.com/controlplane-com/templates/tree/main/litellm">
    View the source files, default values, and chart definition
  </Card>
</CardGroup>