Skip to main content

Overview

AnythingLLM organizes documents into workspaces. Each workspace is a set of files plus a chat that answers from them with citations. Retrieval is built in (a CPU embedding model and an embedded LanceDB vector store), so no extra services are needed. It also serves a website chat widget, a developer API and agents that call MCP tools. This template runs it as one workload on a snapshotted volume; the admin account comes from a secret, so a sign-in is required from the first start.
This template does not create a GVC. It deploys into an existing GVC that you already have, and that GVC should have exactly one location: the workload is a single replica that owns its volume.

What Gets Created

The template creates no secrets or credentials. The public *.cpln.app URL is on by default and every request needs a sign-in.

Prerequisites

1

Create the keys secret

A dictionary secret holding the keys that sign sessions and encrypt stored data. None may be empty:
Set keys.secretName to this name. Never change or delete it afterwards.
2

Create the admin secret

A dictionary secret with the admin account created on first start. The username is 2-64 lowercase letters, digits, ., _, @ or -, starting with a letter; the password is 8 characters or more:
Set admin.secretName to this name.
3

Create the optional secrets

A chat model API key (llm.apiKeySecretName, required for openai and anthropic), an embedding API key (embedding.apiKeySecretName) and a bearer token per HTTP MCP server (bearerTokenSecretName) are each an opaque secret:
The Control Plane MCP server needs a service-account key, see Control Plane MCP Server.
Secret names are org-wide, so give each release its own secrets.
Every secret your release names must exist before you install. If one does not, the deployment waits silently: cpln logs returns nothing, and the only place the missing secret is named is status.versions[].message in cpln workload get-deployments RELEASE_NAME-anythingllm --gvc GVC_NAME -o yaml. Create the secret and the deployment recovers on its own within about 12 minutes, or run cpln workload force-redeployment RELEASE_NAME-anythingllm --gvc GVC_NAME to recover at once. Run the same command after changing any secret the release uses.

Installation

The required values are the names of the two secrets from Prerequisites:
To install with the CLI, UI, Terraform or Pulumi, follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Keys and Admin Account

Chat Model

With provider: "", choose the model in Settings → AI Providers → LLM after signing in; the choice is kept on the volume. Until a model is chosen, chats and the developer API’s vector search fail. For a model served in the same GVC, such as the ollama template, set baseUrl to http://WORKLOAD_NAME.GVC_NAME.cpln.local:PORT, always fully qualified. Use generic-openai for vLLM or any other OpenAI-compatible server, with baseUrl ending in /v1.

Embedding Model

The built-in embedder runs on the CPU inside the same container, needs no key and suits small document sets. For large uploads, point engine at ollama or generic-openai.

MCP Tool Servers

A stdio server runs inside the AnythingLLM container with its environment, which includes the provider keys and JWT_SECRET; add only servers you trust. An HTTP server is reached at its url, streamable or sse, with an optional bearer token. AnythingLLM has no screen for adding an MCP server; Settings → Agent Skills only turns servers on or off and removes them. With the default servers: {}, add servers by editing /app/server/storage/plugins/anythingllm_mcp_servers.json through cpln workload exec; the chart leaves that file alone. With any server listed here, or cplnMcp.keySecretName set, the chart rewrites the whole file on every start, so hand edits are lost. With any MCP server configured, a workspace in the default Automatic chat mode can route ordinary chats through the agent, and a small model that does not call tools reliably then answers from the tools instead of your documents. Set such a workspace to Chat or Query mode so answers come from its documents. Start an agent run with @agent in a workspace chat.

Control Plane MCP Server

Setting keySecretName adds an MCP server named cpln so agents can work with your org. AnythingLLM’s MCP client cannot sign in with OAuth, so it authenticates with a service-account key, and every agent run that uses the server acts as that service account. Give the account only what your agents need. This example lets it view one GVC; add a policy for each further resource kind, such as workload with view:
Set cplnMcp.keySecretName to CPLN_MCP_KEY_SECRET_NAME. The default url exposes read-only tools; change it to https://mcp.cpln.io/mcp only if agents must create, update or delete resources. The key is sent to url, so point it only at the Control Plane MCP server. At each start a log line beginning cpln-mcp: says whether the server accepted the key:

Access

Storage and Backup

Changing capacity does not resize an existing volume; see Growing the Volume.

Server

Connecting

Verify the server is up:
The reply is {"online":true}.

Embed Widget

In Settings → Chat Embed, create an embed for a workspace, list the website’s address under allowed domains, and paste the script tag it shows into your site. An embed with no allowed domains refuses every chat, and a page on a domain that is not listed is refused too.

Operations

Backing Up

Platform snapshots of the whole volume run on the backup schedule (daily at 03:00 UTC, kept 7 days by default) and need no bucket or cloud account. List them:
The volume and its snapshots hold provider keys entered in the UI in plain text, alongside every document, so treat snapshots as sensitive.

Restoring a Backup

Restoring replaces the volume with the snapshot; everything written after the snapshot is lost, and the workload restarts onto the restored volume.

Growing the Volume

Scaling and Availability

AnythingLLM runs as a single replica: its database, vector store and job scheduler belong to one process, and upstream has no multi-instance mode. A restart or upgrade makes it unavailable for a few minutes. Use a single-location GVC.

Troubleshooting

A secret the release names does not exist. Read status.versions[].message with cpln workload get-deployments RELEASE_NAME-anythingllm --gvc GVC_NAME -o yaml, create the secret, then run cpln workload force-redeployment RELEASE_NAME-anythingllm --gvc GVC_NAME to skip the wait.
The admin secret is invalid (username not 2-64 lowercase letters, digits, ., _, @ or - starting with a letter, or a password under 8 characters), or a key in the keys secret is empty. Fix the secret and run cpln workload force-redeployment RELEASE_NAME-anythingllm --gvc GVC_NAME. The release never serves without a sign-in.
No chat model is configured. Set llm.provider and llm.model, or choose a model in Settings → AI Providers → LLM.
MCP servers are configured and the workspace is on Automatic chat mode. Set the workspace to Chat or Query mode.
Read the cpln-mcp: log line (see Control Plane MCP Server). A line reporting HTTP 401 means the key in cplnMcp.keySecretName was refused; create a new service-account key, update the secret and run cpln workload force-redeployment RELEASE_NAME-anythingllm --gvc GVC_NAME.
The page’s domain is not under the embed’s allowed domains in Settings → Chat Embed (HTTP 401 on the request).
The built-in embedder and document parsing share the container’s CPU and memory. Use an ollama or generic-openai embedder for large document sets, or raise resources.

Important Notes

  • Never change or delete the keys secret: SIG_KEY and SIG_SALT encrypt stored data, and a new JWT_SECRET signs everyone out.
  • The admin secret is read on first start only; change the password in the UI afterwards.
  • Values set in llm and embedding win over the UI on every restart; with llm.provider: "" the chat model chosen in the UI stays.
  • Changing the embedding engine or model in the UI deletes every workspace’s vectors and document links. Changing embedding in the values deletes nothing, but documents already embedded stay unsearchable until they are added again.
  • Every agent run that uses the Control Plane MCP server acts as its service account: keep that account’s policy narrow.
  • Uninstalling deletes the volume after a final snapshot, and with it every workspace, document and account.

External References