Overview
Langfuse is an open-source LLM observability and evaluation platform: traces, prompt management, evaluations, and a playground. This template deploys the full self-hosted stack — the web app, a background worker, and the three datastores Langfuse requires — with trace data landing in an object store you own. The instance is closed by default. Self-service registration is off, and the owner account is provisioned at boot from a secret you create, so a public endpoint does not hand a stranger an account.What Gets Created
clickhouse/) and Langfuse (prefixes events/ and media/). The auth secret, the database, Redis and ClickHouse credentials secrets and, on GCS, the storage credentials secret are prerequisite secrets you create yourself: the chart references them by name and never creates, modifies, or deletes them.
Prerequisites
Three things must exist before you install: the auth secret, the datastore credentials secrets, and the object storage that traces are written to.Create the auth secret
nextAuthSecret, encryptionKey and salt, and keep a copy of encryptionKey outside Control Plane. Read the values back later with cpln secret reveal AUTH_SECRET_NAME -o yaml.Set langfuse.auth.secretName to this name.Create the database credentials secret
username, password and database. The bundled PostgreSQL initializes with it and Langfuse connects with it:postgres.config.credentialsSecretName to this name. Keep all three values URL-safe (letters, digits, -, _): Langfuse puts them in its connection URL. PostgreSQL reads the secret only when its data directory is first initialized; to change the password later, see Rotating Credentials.Create the Redis and ClickHouse credentials secrets
redis.credentialsSecretName and clickhouse.credentialsSecretName to those names. Keep the ClickHouse values URL-safe (letters, digits, -, _; the username must also start with a letter): Langfuse puts them in its migration URL.Set up object storage
- AWS S3
- Google Cloud Storage
- Create an S3 bucket. Set
objectStore.aws.bucketto its name andobjectStore.aws.regionto its region. - If you do not have one, create a Cloud Account for your AWS account and set
objectStore.aws.cloudAccountNameto its name. The workloads then reach the bucket keylessly. - Create an IAM policy with the JSON below (replace
YOUR_BUCKET_NAME) and setobjectStore.aws.policyNameto its name.
Installation
Install from the marketplace registry, naming your auth secret, your three datastore credentials secrets and your bucket:objectStore.aws.* values with objectStore.provider=gcp, objectStore.gcp.bucket and objectStore.gcp.credentialsSecretName.
UI
CLI
Terraform
Pulumi
Configuration
Object Store
objectStore.provider is rendered into the workloads; the other is ignored.
Langfuse
langfuse.publicUrl is the origin the browser uses: auth redirects, the pages NextAuth renders server-side, and whether the session cookie gets the Secure flag all derive from it. Leave it empty unless you serve Langfuse on a custom domain, and when you set it include the scheme — a value without one is rejected at render.
With langfuse.auth.disableSignup: true (the default) the owner account from your auth secret is the only way in: Langfuse blocks both email/password sign-up and first-time SSO user creation, so the template provisions that account at boot using Langfuse’s headless initialization. Set it to false and anyone who can reach the UI can create an account and their own organization.
Access
publicAccess.enabled: false the API stays reachable from other workloads in the GVC, but there is no browser path to the UI: cpln port-forward does not reach the Langfuse web tier, so keep public access on if you need to sign in.
PostgreSQL
postgres.*, including scheduled backups — see Backing Up.
Redis
ClickHouse
Connecting
cpln workload get-deployments RELEASE_NAME-langfuse-web --gvc GVC_NAME -o yaml reports it ready:
First Run
- Open the canonical endpoint in a browser and sign in with the
adminEmailandadminPasswordfrom your auth secret. The account is anOWNERof the organization named bylangfuse.auth.organizationName, which already exists. - Create a project, then generate a key pair under Settings → API Keys.
- Send a first trace with the public ingestion API, or point a Langfuse SDK at your endpoint with the same keys:
events/ prefix of your bucket, the worker takes it off the Redis queue and writes it into ClickHouse, and the trace appears in the UI a few seconds later.
- For the playground and LLM-as-a-judge evaluations, add provider API keys under Settings → LLM Connections. They are stored encrypted with your
encryptionKey.
Operations
Backing Up
PostgreSQL is the one datastore that must be backed up: it holds users, projects, API keys, prompts, datasets and evaluation configs. ClickHouse keeps its data parts in your bucket and is as durable as that bucket — its volume set holds local metadata only. Redis holds only the transient ingestion queue and cache. Two paths cover PostgreSQL. A volume set snapshot captures the full disk state:RELEASE_NAME-postgres-backup cron workload and uploads a gzipped pg_dumpall to a bucket of your choice. It is off by default:
gcp and minio blocks — follow the postgres template’s backup prerequisites; the keys there live under postgres.backup.* here.
Restoring a Backup
These paths have not been exercised against this template; verify them on a throwaway release before relying on them.- From a scheduled dump: the dump is a plain
pg_dumpalloutput restored withpsqlagainstRELEASE_NAME-postgres.GVC_NAME.cpln.local:5432, using theusernameandpasswordfrom your database credentials secret. The command sequence per provider is on the postgres template’s restore section; it must run from a client that can reach the bucket and the database, which means a workload inside the GVC. - From a volume set snapshot:
cpln volumeset snapshot restoreexists (checkcpln volumeset snapshot restore --help) but has not been run against this template. - ClickHouse: trace data is in your bucket under
clickhouse/. A fresh ClickHouse workload pointed at the same bucket is the recovery path, but it has not been verified; deleting the bucket loses all trace history regardless of snapshots.
Upgrading from 1.0.0 or 1.0.1
Version 1.1.0 removed six values keys and moved all auth key material into the prerequisite auth secret. A values file that still carries any of them fails at render, so the upgrade aborts before anything is applied and the live release is untouched — but it does not proceed until the keys are gone. There are no compatibility fallbacks.Back up PostgreSQL
RELEASE_NAME-pg-vs before changing anything (see Backing Up).Create the prerequisite secrets
adminEmail and adminPassword have no 1.0.x equivalent — they provision the owner account.Remove the six keys and name the secrets
langfuse.auth.secretName and, on GCS, objectStore.gcp.credentialsSecretName. Replace langfuse.firewall.inboundAllowCIDR with publicAccess.enabled and internalAccess.type.Upgrade and verify
RELEASE_NAME-langfuse-web and RELEASE_NAME-langfuse-worker reach ready with cpln workload get-deployments RELEASE_NAME-langfuse-web --gvc GVC_NAME -o yaml.Upgrading from 1.1.0
postgres.config.username, .password and .database were removed. Put the values your database was initialized with into a database credentials secret as under Prerequisites (keys username, password, database), under any name except RELEASE_NAME-pg-config (the secret 1.1.0 created, which this upgrade deletes). Set postgres.config.credentialsSecretName to it and remove the three keys. The Redis and ClickHouse passwords move to prerequisite secrets too: follow Upgrading from 1.2.x for them (1.1.0 keeps them in the same RELEASE_NAME-langfuse-config secret). The auth path is unchanged.
Upgrading from 1.2.0
Version 1.2.1 removes the AWS managed policyaws::ReadOnlyAccess from the identity this template links to your AWS cloud account. That policy granted read access to every bucket in your AWS account and contained no write actions, so it was never carrying the object store itself — but it was silently supplying any read action your bucket-scoped policy omitted. The identity now carries cpln-connector and your own policy only.
Before upgrading, update your IAM policy to the ten-action list in Prerequisites — 1.2.0 documented six actions. If it already matches, no action is needed. Nothing else changes.
Upgrading from 1.2.x
Version 1.3.0 removedpostgres.credentials, redis.auth.password and clickhouse.config.password: all three datastores now read prerequisite secrets, and carrying any old key forward fails the render.
1.2.x created the database secret itself, so this upgrade deletes it. Before upgrading, create a secret under a new name holding the values your database was initialized with — not new ones, or Langfuse loses its database connection:
postgres.credentials of the 1.2.x release, and also in the old secret named by postgres.config.credentialsSecretName in 1.2.x (cpln secret reveal SECRET_NAME -o yaml) until the upgrade deletes it.
1.2.x kept the Redis and ClickHouse passwords in RELEASE_NAME-langfuse-config (keys redisPassword, clickhousePassword), which this upgrade also deletes, so use other names. Create them holding your current passwords, so the old and new versions keep talking to each other during the rollout:
redis.auth.password and clickhouse.config.password of the 1.2.x release, and also in cpln secret reveal RELEASE_NAME-langfuse-config -o yaml until the upgrade deletes it.
Then upgrade with postgres.config.credentialsSecretName, redis.credentialsSecretName and clickhouse.credentialsSecretName set to those names, and postgres.credentials, redis.auth and clickhouse.config.password removed. The upgrade restarts the bundled PostgreSQL (it now reads the new secret), so Langfuse briefly loses its database and reconnects by itself. While access to the new secrets propagates, the new version of a workload can wait in get-deployments reporting that it is not allowed to reveal the secret; the previous version keeps serving meanwhile, and it clears by itself.
Upgrading from 1.3.0
Version 2.0.0 moves Langfuse 3 to Langfuse 4, and ClickHouse 25.10 to 26.8 (Langfuse 4 needs 25.12 or newer). It is an in-placehelm upgrade of the existing release: the data stays where it is, the web and worker workloads wait until ClickHouse reports the new version, and the web workload then applies the Langfuse 4 database migrations at start.
The upgrade is one-way. After the migrations, a 1.3.0 web container exits at start, and moving ClickHouse back to 25.10 on the same data is not supported. Going back means restoring PostgreSQL from a backup; this template has no ClickHouse backup. Langfuse documents a manual schema rewind for the time before the cutover.
Before you upgrade:
- A release older than 1.3.0 goes to 1.3.0 first, following the matching sections above.
-
Check that every Langfuse 3 background migration has finished, because Langfuse 4 drops the tables they copy from. The five
20260701_v4_step_*migrations are already listed on 1.3.0 but run only on Langfuse 4, so the query leaves them out. It must print no rows:If it prints rows, check the UI (Langfuse version tag, then Background Migrations). The worker runs background migrations only when it starts, and gives up if PostgreSQL is not reachable yet, so rows that never started are run by restarting it: - Back up PostgreSQL (see Backing Up) and check the backup object is not empty.
- Choose the write mode. It decides what keeps working from Langfuse 3:
events_only, the following stop working: Python SDK v2 and older and JS/TS SDK v3 and older (rejected at ingestion), the legacy read endpoints such as GET /api/public/traces and GET /api/public/observations (404), every POST /api/public/ingestion event except scores, trace-level LLM-as-a-judge evaluators, and the “Traces and observations (legacy)” export source.
historicBackfill: true copies the traces stored before the upgrade into the new tables, once, in the worker, newest first: older traces appear gradually, and the progress is under Background Migrations. While it runs the bucket holds up to about three times the current ClickHouse data. A scratch copy of the old observations stays in the bucket afterwards: the last step, which drops it, is off by default upstream, so Background Migrations keeps showing one unfinished row. After a backfill on top of dual, a trace can appear twice in the UI until ClickHouse merges the two copies in the background; the data is not damaged. With dual, Langfuse suggests leaving it off until the dual write is confirmed working, then turning it on in a later helm upgrade. With legacy it must be off.
When every producer and consumer has moved, cut over with a helm upgrade setting writeMode: events_only (and historicBackfill: true if it has not run). From then on new data lands only in the new tables, so this is the point of no return.
The traces list is now an Observations view filtered to Is Root Observation = true, one row per trace; clear the filter to see every observation.
Rotating Credentials
A running workload keeps the credential values it started with, so every rotation ends with a forced redeployment. Change one key and keep the others with export, edit, apply:- Database password: PostgreSQL reads the secret only when it is first initialized, so run
ALTER ROLEin the database first, then update the secret, then right away force a redeployment ofRELEASE_NAME-langfuse-webandRELEASE_NAME-langfuse-worker. Do not redeploy the database; it would only add downtime. Connections already open keep working afterALTER ROLE, new ones fail until the redeploy finishes, so rotate in a quiet window. - Redis and ClickHouse passwords: neither datastore keeps its password in its data (Redis takes it on the command line, ClickHouse defines its user from the secret at every start), so there is no
ALTERstep. Update the secret, then force a redeployment of the datastore and of everything that connects to it, together:
username works the same way.
Scaling and Availability
- The web tier autoscales on CPU between
langfuse.web.minReplicasandlangfuse.web.maxReplicas. Keep the minimum at 2 or more for rolling deploys without downtime. - The worker runs a fixed
langfuse.worker.replicas; raise it for higher ingestion throughput. - PostgreSQL, Redis and ClickHouse are single-instance on persistent volume sets. A node failure reschedules each and reattaches its volume, so the cost is minutes of downtime, not data loss.
- The first
helm upgradeafter an install can restart the bundled datastores even when nothing changed; plan it as a short ingestion interruption and check that traces land again afterwards rather than trusting the health endpoint, which stays green throughout. - Settings carried in the web container’s environment —
langfuse.auth.disableSignup,langfuse.publicUrl, any change to the auth secret — take effect only as the web replicas roll. Changing a value in the auth secret does not redeploy anything by itself; runcpln workload force-redeployment RELEASE_NAME-langfuse-web --gvc GVC_NAMEand the same forRELEASE_NAME-langfuse-worker.
Troubleshooting
Web and worker never become ready and cpln logs returns nothing
Web and worker never become ready and cpln logs returns nothing
cpln helm install succeeded, but RELEASE_NAME-langfuse-web and RELEASE_NAME-langfuse-worker stay not-ready with no container output. status.versions[].message reads The secret <name> no longer exists. Workload updates are paused until the secret is added or the reference to the secret removed.Cause: a secret named by langfuse.auth.secretName, postgres.config.credentialsSecretName, redis.credentialsSecretName, clickhouse.credentialsSecretName (or, on GCS, objectStore.gcp.credentialsSecretName) did not exist at install time. Check the datastore workloads the same way.Fix: create it as shown in Prerequisites. The deployment recovers on its own after a few minutes, or force it:Render fails with a key was removed in 1.3.0
Render fails with a key was removed in 1.3.0
helm upgrade aborts with postgres.credentials was removed in 1.3.0 ... (or the same for redis.auth.password or clickhouse.config.password).Cause: your values file still carries a 1.2.x datastore credential key.Fix: follow Upgrading from 1.2.x.Render fails with a key was removed in 1.1.0
Render fails with a key was removed in 1.1.0
helm upgrade aborts with langfuse.auth.nextAuthSecret was removed in 1.1.0 ... (or the same for encryptionKey, salt, objectStore.gcp.accessKeyId, objectStore.gcp.secretAccessKey, langfuse.firewall.inboundAllowCIDR).Cause: your values file still carries a 1.0.x key.Fix: follow Upgrading from 1.0.0 or 1.0.1.Render fails with publicUrl must include the scheme
Render fails with publicUrl must include the scheme
langfuse.publicUrl must include the scheme, e.g. https://langfuse.example.com.Cause: langfuse.publicUrl was set to a bare hostname.Fix: prefix it with https:// (or http://).Sign-up page says registration is disabled
Sign-up page says registration is disabled
POST /api/auth/signup returns HTTP 422 with {"message":"Sign up is disabled."}.Cause: langfuse.auth.disableSignup is true, the default. This is the intended closed instance.Fix: sign in with the adminEmail / adminPassword from your auth secret and invite users from inside Langfuse, or set langfuse.auth.disableSignup: false and upgrade if you want self-service registration.Changed adminPassword in the secret but the old password still works
Changed adminPassword in the secret but the old password still works
adminPassword (or adminEmail) in the auth secret and redeployed, but the login did not change.Cause: headless initialization only creates the owner account when it does not exist yet; it does not update an existing user.Fix: change the password from inside Langfuse under the user’s settings. The values in the secret matter only for a fresh database.Database connection errors in web and worker logs on a cold install
Database connection errors in web and worker logs on a cold install
RELEASE_NAME-langfuse-web and RELEASE_NAME-langfuse-worker show Can't reach database server or Applying database migrations failed while RELEASE_NAME-postgres is still starting.Cause: startup ordering. The web and worker tiers retry and apply migrations once PostgreSQL accepts connections.Fix: none needed if the workloads reach ready. If they keep failing, check RELEASE_NAME-postgres with cpln workload get-deployments RELEASE_NAME-postgres --gvc GVC_NAME -o yaml and its logs:ClickHouse starts on GCS but object storage operations fail with an auth error
ClickHouse starts on GCS but object storage operations fail with an auth error
RELEASE_NAME-langfuse-clickhouse becomes ready, but writes to the bucket fail with an access or signature error in its logs.Cause: on the gcp path ClickHouse reads the HMAC pair from the accessKeyId and secretAccessKey keys of the secret named by objectStore.gcp.credentialsSecretName. A wrong key name, a key for a different service account, or a service account without roles/storage.objectAdmin all surface here.Fix: check the secret’s keys with cpln secret reveal GCS_CREDENTIALS_SECRET_NAME -o yaml, confirm the HMAC key belongs to the service account that was granted access, then force a redeployment:Important Notes
- Create the auth, database, Redis and ClickHouse credentials secrets before installing; a missing one leaves the workloads waiting on a secret that does not exist, named only in
status.versions[].messageofcpln workload get-deployments RELEASE_NAME-langfuse-web --gvc GVC_NAME -o yaml. - Rotating any of these secrets needs a forced redeployment; the database password also needs
ALTER ROLEfirst (see Rotating Credentials). encryptionKeycan never be rotated: it encrypts every provider key stored under Settings → LLM Connections, and changing it makes all of them unreadable. Keep a copy outside Control Plane and never delete the secret.- Sign-up is closed by default and the owner account comes from your secret. Record
adminPassword— it is displayed nowhere, and with sign-up closed there is no other way to create the first account. - Give each install its own bucket. Two installs on one bucket share ClickHouse data parts under
clickhouse/. - Give each release its own database, Redis and ClickHouse secrets. Secret names are org-wide, so two releases pointed at one secret share a password, and rotating it for one breaks the other.
- Upgrading from 1.3.0 is one-way and changes what older SDKs and the legacy APIs can do: read Upgrading from 1.3.0 before you upgrade.
- Serving Langfuse on your own domain requires
langfuse.publicUrl, including the scheme. - With
publicAccess.enabled: falsethere is no browser path to the UI; only other workloads in the GVC can reach the API. - Prerequisite secrets are not owned by the release: the auth, datastore credentials and GCS credentials secrets survive
helm uninstalland must be deleted by hand if no longer wanted. - Uninstall deletes the volume sets, including PostgreSQL’s. Trace data already in your bucket is untouched, but users, projects and API keys are not — back up PostgreSQL first.