Ollama

Overview

Ollama is an open-source server for running open-weight large language models on your own infrastructure. This template deploys a single-replica Ollama server together with an Open WebUI sidecar in the same workload: the Ollama HTTP API listens on port 11434 for other workloads in the GVC, and the chat UI on port 8080 is the surface the public endpoint serves. Pulled models and the UI’s state live on one persistent volume. defaultModel sets the model Open WebUI preselects; pull it once by hand after install (see Installation).

This template deploys into an existing GVC that you already have. It does not create, provision, or manage a GVC. GPU-enabled locations are available across several cloud providers; a GVC with more than one location runs one replica of this workload in each.

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-ollama`	Stateful, 1 replica, two containers: `ollama` (API on `11434`, GPU-backed) and `ollama-ui` (Open WebUI on `8080`)	—
Volume Set	`RELEASE_NAME-ollama-vs`	`ext4` volume mounted at `/root/.ollama` (models) and `/app/backend/data` (Open WebUI state); final snapshot on delete	—
Secret	`RELEASE_NAME-ollama-secret`	Opaque startup script mounted at `/startup/entrypoint.sh`; starts the Ollama server	—
Identity	`RELEASE_NAME-ollama-identity`	Identity the workload runs as	—
Policy	`RELEASE_NAME-ollama-policy`	`reveal` on exactly `RELEASE_NAME-ollama-secret`	—

Open WebUI reaches Ollama over the container’s loopback (http://localhost:11434) and lists defaultModel as its default selection, so the two containers need no network configuration between them.

Prerequisites

None for a default install. No secret, bucket, or cloud account is required; the only external dependency is outbound internet access so models can be pulled from the Ollama registry. Two org-level requirements are the usual first blockers:

GPU enablement — the default workload.containers.api.gpu requests one NVIDIA T4. GPU resources must be enabled for your org by Control Plane support before a GPU workload can schedule. To run on CPU only, pass --set workload.containers.api.gpu=null at install; the chart omits the GPU block when the value is empty.
Quota — the default install requests 6 CPU and 8Gi of memory for the Ollama container plus 500m and 1Gi for the UI. If your org is at the default quotas, request an increase first.

Installation

A default install needs no --set values:

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/ollama \
  --version 1.1.2 --gvc GVC_NAME

The install does not download a model. Once the workload is ready, pull the model named by defaultModel (llama3 by default) once by hand; it lands on the volume and survives restarts:

cpln workload exec RELEASE_NAME-ollama --gvc GVC_NAME --container ollama -- ollama pull llama3

Models are several GiB, so the pull takes minutes.

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Default Model

Any model from the Ollama library can be named here. It sets the model Open WebUI preselects; it is not downloaded automatically, so pull it once by hand after install (see Installation).

# Alternatives: llava, gemmma, mistral, etc.
defaultModel: llama3

Containers

Both containers run in the one workload. api is the Ollama server; gpu is rendered only when present, so clear it (--set workload.containers.api.gpu=null) to run on CPU alone. Large models need more CPU and memory than the defaults; size the api container for the models you intend to run.

workload:
  containers:
    ui:
      name: ollama-ui
      image: ghcr.io/open-webui/open-webui:v0.11.0 # pinned: `main` is a moving branch build
      port: 8080
      resources:
        cpu: 500m
        memory: 1Gi
    api:
      name: ollama
      image: ollama/ollama:0.32.15 # pinned: an untagged image resolves to `latest`
      port: 11434
      resources:
        cpu: 6
        memory: 8Gi
      gpu:
        nvidia:
          model: t4
          quantity: 1

Storage

One volume holds both the pulled models and the Open WebUI database. Models are large: a 10 GiB volume fills quickly once a second model is pulled, so either start larger or enable autoscaling.

volumeset:
  initialCapacity: 10
  autoscaling:
    enabled: false # Set to true to enable autoscaling
    maxCapacity: 100 # Maximum capacity in GiB when autoscaling is enabled
    minFreePercentage: 10 # Minimum free percentage to trigger scaling when autoscaling is enabled
    scalingFactor: 1.2 # Scaling factor to determine how much to scale up when autoscaling is triggered
  performanceClass: general-purpose-ssd
  snapshots:
    retentionDuration: 7d

Firewall

Inbound is open to the internet by default so the Open WebUI is reachable at the canonical endpoint. Outbound must allow the Ollama registry (0.0.0.0/0 in the default) or model pulls fail.

firewall:
  external: # Change to restrict access to the workload if needed
    inboundAllowCIDR:
        - 0.0.0.0/0
    outboundAllowCIDR:
        - 0.0.0.0/0

Internal Access

Controls which workloads can reach the Ollama API on 11434 over internal DNS. The default admits every workload in the same GVC; workload-list restricts it to the workloads you name.

internal_access:
  type: same-gvc # options: same-gvc, same-org, workload-list
  workloads:  # Note: can only be used if type is same-gvc or workload-list
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

Startup Script

The ollama container runs this script instead of the image’s own entrypoint. It is stored in RELEASE_NAME-ollama-secret and mounted at /startup/entrypoint.sh; override it only if you need different first-boot behaviour.

entrypoint:
  payload: |
    #!/bin/bash
    # Define the model directory
    MODEL_DIR="/root/.ollama/models/manifests/registry.ollama.ai/library/$DEFAULT_MODELS/"
    # Start ollama serve in the background
    /bin/ollama serve &
    # Check if the model directory exists
    if [ ! -d "$MODEL_DIR" ]; then
        echo "Model directory not found. Pulling the $DEFAULT_MODELS model..."
        # Pull the $DEFAULT_MODELS model using the Ollama API
        apt-get update && apt-get install curl -y
        curl http://localhost:11434/api/pull -d '{
            "name": "$DEFAULT_MODELS"
        }'
    else
        echo "Model directory exists. No action required."
    fi
    # Keep the script running
    while true; do sleep 86400; done

Connecting

What	Address	Notes
Open WebUI (browser)	`status.canonicalEndpoint` of `RELEASE_NAME-ollama`	Read it with `cpln workload get RELEASE_NAME-ollama --gvc GVC_NAME -o yaml`; reachable from the CIDRs in `firewall.external.inboundAllowCIDR`
Ollama API (other workloads)	`http://RELEASE_NAME-ollama.GVC_NAME.cpln.local:11434`	Only callers admitted by `internal_access.type`; no authentication of its own
Ollama API (your machine)	`cpln port-forward RELEASE_NAME-ollama 11434:11434 --gvc GVC_NAME`, then `http://localhost:11434`	Works with the external firewall closed
Credentials	None for the API; the Open WebUI account you register at first run	The first account registered becomes the administrator

Verify the server is up and see which models are on the volume:

cpln workload exec RELEASE_NAME-ollama --gvc GVC_NAME --container ollama -- ollama list

Call the API from another workload in the GVC (the endpoints are the upstream Ollama API):

# list local models
curl http://RELEASE_NAME-ollama.GVC_NAME.cpln.local:11434/api/tags

# one-shot completion
curl http://RELEASE_NAME-ollama.GVC_NAME.cpln.local:11434/api/generate \
  -d '{"model": "llama3", "prompt": "Why is the sky blue?", "stream": false}'

Other templates in this catalog that consume Ollama over that internal URL include Open WebUI (as a standalone UI pointed at an existing Ollama workload) and Qdrant (local embedding models for a RAG stack).

First Run

Open WebUI has no preset credentials: the first account to register becomes the administrator, and the UI is public as soon as the workload is ready. Open the canonical endpoint and register your admin account immediately after install. If the endpoint must not be reachable from the internet before that happens, install with firewall.external.inboundAllowCIDR narrowed to your own address range and widen it afterwards. Open WebUI’s own setup (users, sign-up policy, additional model pulls) is documented in the Open WebUI documentation.

Operations

Backing Up

Backups are platform volume snapshots of RELEASE_NAME-ollama-vs; no cloud account or bucket is involved. The chart configures no snapshot schedule — it sets a 7-day retentionDuration and takes a final snapshot when the volume set is deleted. Take a snapshot by hand before an upgrade or a large model change:

cpln volumeset snapshot create RELEASE_NAME-ollama-vs --gvc GVC_NAME --snapshot-name before-upgrade
cpln volumeset snapshot get RELEASE_NAME-ollama-vs --gvc GVC_NAME

The volume holds two kinds of data with very different value: pulled models, which can always be downloaded again, and Open WebUI’s database (/app/backend/data: users, chats, uploaded documents), which cannot.

Restoring a Backup

Restoring a volume-set snapshot into this template has not been verified. Treat it as an unverified path and test it on a throwaway release before relying on it. Models need no backup: on a fresh or replaced volume, pull each model you use again with ollama pull (see Troubleshooting).

Upgrading from 1.1.1 or Earlier

Template version 1.1.2 pins both images — ghcr.io/open-webui/open-webui:v0.11.0 and ollama/ollama:0.32.15. Earlier versions used the moving open-webui:main branch build and an untagged ollama/ollama that resolved to latest, so an install from before 1.1.2 is not reproducible and may be running a newer or older build than the pins. Upgrading changes the running image versions and leaves the volume set, and the models and UI data on it, in place. Take a snapshot first (see Backing Up) and upgrade with your usual install method.

Scaling and Availability

The workload is fixed at one replica. A GVC with several locations runs one replica per location, each with its own volume, so each location needs its own copy of every model pulled into it and holds its own Open WebUI state; users reach the location closest to them through the canonical endpoint. Add capacity to the single replica (workload.containers.api.resources, a larger or additional GPU) rather than adding replicas.

Troubleshooting

The workload never becomes ready after install

Symptom: cpln workload get-deployments RELEASE_NAME-ollama --gvc GVC_NAME -o yaml shows the deployment not ready, and status.versions[].message refers to GPU resources or a quota.Cause: The default install requests one NVIDIA T4 plus 6 CPU and 8Gi of memory. GPU access must be enabled for the org by Control Plane support, and the CPU and memory must fit the org quota.Fix: Request GPU enablement or a quota increase from Control Plane support. To run on CPU only in the meantime, reinstall with --set workload.containers.api.gpu=null; inference is much slower without a GPU.

The default model is not in the model list

Symptom: ollama list (or GET /api/tags) does not include defaultModel, and Open WebUI’s model picker is empty.Cause: The install does not download any model; defaultModel only sets the UI’s preselection.Fix: Pull it by hand inside the ollama container, then reload the UI:

cpln workload exec RELEASE_NAME-ollama --gvc GVC_NAME --container ollama -- ollama pull llama3

Any other library model can be pulled the same way; it lands on the volume and survives restarts. If the pull fails, check that firewall.external.outboundAllowCIDR allows the Ollama registry, that the volume has space, and that the model name exists in the library.

Pulls fail with no space left on device

Symptom: A pull aborts with no space left on device in the Ollama logs, or Open WebUI starts failing to write.Cause: The volume (10 GiB by default) is shared by every pulled model and the UI’s data.Fix: Remove models you no longer use with ollama rm MODEL inside the ollama container, and enable volumeset.autoscaling (or install with a larger volumeset.initialCapacity) so the volume grows as models accumulate.

Open WebUI shows no models or cannot reach Ollama

Symptom: The UI loads but reports that Ollama is unreachable or offers no models.Cause: The UI talks to the Ollama container over http://localhost:11434; it shows nothing while the Ollama container is still starting or has restarted.Fix: Confirm both containers are ready with cpln workload get-deployments RELEASE_NAME-ollama --gvc GVC_NAME -o yaml, then reload the UI. If the API answers (ollama list above) but lists no models, follow the previous entry.

Important Notes

GPU and quota are the usual first blockers. The default requests one NVIDIA T4, 6 CPU and 8Gi of memory; GPU access must be enabled by Control Plane support and the rest must fit your org quota.
Register the admin account right after install. The Open WebUI is public by default and the first account to register becomes the administrator — see First Run.
Do not route public traffic to 11434. The Ollama API has no authentication; reach it only over internal DNS (internal_access) or a port-forward.
Models live on the volume set, and uninstall deletes it. A final snapshot is taken on delete, but every model must be pulled again after a reinstall; size the volume for what you intend to run.
Pull the default model after install. defaultModel only sets the model Open WebUI preselects; run ollama pull once in the ollama container before chatting.
Outbound internet is required. Model pulls need firewall.external.outboundAllowCIDR to allow the Ollama registry.

External References

Ollama Documentation

Official Ollama repository and documentation

Ollama API Reference

Endpoints for generate, chat, pull, and model listing

Ollama Model Library

Browse available models for Ollama

Open WebUI Documentation

Setup, users, and model management for the bundled UI

Ollama Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

What Gets Created

Prerequisites

Installation

UI

CLI

Terraform

Pulumi

Configuration

Default Model

Containers

Storage

Firewall

Internal Access

Startup Script

Connecting

First Run

Operations

Backing Up

Restoring a Backup

Upgrading from 1.1.1 or Earlier

Scaling and Availability

Troubleshooting

Important Notes

External References

Ollama Documentation

Ollama API Reference

Ollama Model Library

Open WebUI Documentation

Ollama Template

​Overview

​What Gets Created

​Prerequisites

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​Default Model

​Containers

​Storage

​Firewall

​Internal Access

​Startup Script

​Connecting

​First Run

​Operations

​Backing Up

​Restoring a Backup

​Upgrading from 1.1.1 or Earlier

​Scaling and Availability

​Troubleshooting

​Important Notes

​External References

Ollama Documentation

Ollama API Reference

Ollama Model Library

Open WebUI Documentation

Ollama Template

Overview

What Gets Created

Prerequisites

Installation

Configuration

Default Model

Containers

Storage

Firewall

Internal Access

Startup Script

Connecting

First Run

Operations

Backing Up

Restoring a Backup

Upgrading from 1.1.1 or Earlier

Scaling and Availability

Troubleshooting

Important Notes

External References