Mimir

Overview

Grafana Mimir is a long-term Prometheus metrics store you own. This template deploys Mimir in monolithic mode, with every Mimir component in one process. Your own collectors (Prometheus, Grafana Alloy, OpenTelemetry) push metrics in through Prometheus remote_write, and anything that speaks PromQL, such as your own Grafana, queries them back. Metric blocks are stored durably in your object bucket. This is a self-hosted metrics store for your own metrics from your own sources. It is separate from Control Plane’s built-in observability, which keeps collecting and dashboarding your workloads’ metrics natively.

This template deploys into an existing GVC that you already have. It does not create, provision, or manage a GVC. Pass the GVC name with --gvc GVC_NAME at install time.

What Gets Created

Resource	Name	Purpose	Optional
Workload	`RELEASE_NAME-mimir`	Mimir (stateful; 1 replica by default, 3 or more for an HA cluster). Remote write and PromQL on port `8080`	—
Volume Set	`RELEASE_NAME-mimir-vs`	20 GiB at `/data` per replica for the ingester WAL and compactor workspace. Metric blocks live in your bucket, not here	—
Secret	`RELEASE_NAME-mimir-config`	The rendered Mimir configuration, mounted as a file	—
Identity	`RELEASE_NAME-mimir-identity`	Workload identity. Carries keyless bucket access for AWS and GCS	—
Policy	`RELEASE_NAME-mimir-policy`	Grants the identity `reveal` on the config secret, and on your credentials secret when `storage.type: minio`	—

Prerequisites

Mimir needs an existing bucket in one of three backends, plus access to it. Complete the steps for your backend before installing, and set storage.type to match (aws is the default).

AWS S3
Google Cloud Storage
S3-compatible (MinIO, R2, Wasabi)

AWS S3 uses a Control Plane cloud identity. No credentials are stored; the workload identity receives temporary credentials at runtime.

Create a bucket

Create an S3 bucket. Set storage.aws.bucket to its name and storage.aws.region to its region.

Set up a Cloud Account

If you do not have one, create a Cloud Account for your AWS account. Set storage.aws.cloudAccountName to its name.

Create a bucket-scoped IAM policy

In AWS IAM, create a policy with the JSON below, replacing YOUR_BUCKET with your bucket name. Set storage.aws.policyName to the policy name (the bare name, not the ARN).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET",
        "arn:aws:s3:::YOUR_BUCKET/*"
      ]
    }
  ]
}

Google Cloud Storage uses a Control Plane cloud identity. No credentials are stored.

Create a bucket

Create a GCS bucket. Set storage.gcp.bucket to its name.

Set up a Cloud Account

If you do not have one, create a Cloud Account for your GCP project. Set storage.gcp.cloudAccountName to its name.

No policy to author

The template grants the workload identity roles/storage.objectAdmin on exactly that bucket.

Any S3-compatible server uses static access keys, held in a secret you create.

Create a bucket and access keys

Create the bucket on your server, and an access key pair that can read and write it. Set storage.minio.bucket and storage.minio.region to match.

Create the credentials secret

Create a dictionary secret holding exactly the keys accessKey and secretKey:

cpln secret create-dictionary --name SECRET_NAME \
  --entry accessKey='YOUR-ACCESS-KEY' \
  --entry secretKey='YOUR-SECRET-KEY'

Set storage.minio.credentialsSecretName to this name.

Set the endpoint

Set storage.minio.endpoint to the S3 API address as host:port with no scheme (for example my-minio:9000), and storage.minio.insecure: true when the endpoint is plain HTTP. For a server running as a workload in the same GVC, use its fully qualified name, WORKLOAD_NAME.GVC_NAME.cpln.local:9000.

Create the secret before installing. A credentialsSecretName that points at a secret that does not exist installs “successfully” and then wedges: the workload never becomes ready, and cpln logs returns nothing because no container starts. The only place the missing secret is named is status.versions[].message:

cpln workload get-deployments RELEASE_NAME-mimir --gvc GVC_NAME -o yaml

Plain cpln workload get has no versions key. Once the secret exists the workload recovers on its own; to skip the wait, run cpln workload force-redeployment RELEASE_NAME-mimir --gvc GVC_NAME.

Installation

Install with the bucket, cloud account and IAM policy from the AWS prerequisites (the default storage.type):

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/mimir \
  --version 1.2.0 --gvc GVC_NAME \
  --set storage.aws.bucket=BUCKET_NAME \
  --set storage.aws.region=BUCKET_REGION \
  --set storage.aws.cloudAccountName=CLOUD_ACCOUNT_NAME \
  --set storage.aws.policyName=IAM_POLICY_NAME

For Google Cloud Storage, set storage.type=gcp, storage.gcp.bucket and storage.gcp.cloudAccountName instead. For an S3-compatible server, set storage.type=minio, storage.minio.endpoint, storage.minio.bucket and storage.minio.credentialsSecretName. You can also install the template using your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

Image and Resources

image: grafana/mimir:3.1.3

resources: # monolithic Mimir; memory governs ingestable series count
  maxCpu: 1000m
  maxMemory: 2Gi
  minCpu: 500m
  minMemory: 1Gi

replicas: 1 # 1 for a single instance; 3 or more forms an HA cluster with 3-way replication

replicas must be 1 or at least 3; the template refuses to render with 2. See Scaling and Availability.

Object Storage

storage:
  type: aws # options: aws, gcp, minio

  # AWS S3 — via Control Plane cloud identity.
  aws:
    bucket: my-mimir-bucket # bucket must already exist
    region: us-east-1
    cloudAccountName: my-s3-cloud-account # Control Plane AWS cloud account
    policyName: my-mimir-s3-policy # custom IAM policy granting bucket access (bare name)

  # Google Cloud Storage — via Control Plane cloud identity.
  gcp:
    bucket: my-mimir-bucket # bucket must already exist
    cloudAccountName: my-gcs-cloud-account # Control Plane GCP cloud account

  # Any S3-compatible server (MinIO, R2, Wasabi, …) — static keys + endpoint.
  minio:
    endpoint: my-minio:9000 # host:port, NO scheme — use the FQDN, e.g. my-minio.my-gvc.cpln.local:9000
    insecure: true # true when the endpoint is plain HTTP
    bucket: my-mimir-bucket # bucket must already exist
    region: us-east-1
    # REQUIRED PREREQUISITE SECRET when type is `minio` — a `dictionary` secret
    # holding exactly `accessKey` and `secretKey`. They reach Mimir as
    # AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY rather than being written into
    # mimir.yaml, because a cpln:// reference inside a config file is never resolved.
    credentialsSecretName: my-mimir-minio-credentials

Only the block matching storage.type is read. One bucket serves the whole deployment; the template namespaces its contents internally.

Tenancy

multitenancy:
  enabled: false # when true, every client request must send X-Scope-OrgID: <tenant>

Tenants are implicit: writing with a new tenant ID creates it, with no provisioning step. The header identifies a tenant; it does not authenticate anyone. Changing multitenancy.enabled or retention.period updates only the Mimir configuration secret, which a running replica does not re-read. After the cpln helm upgrade, run cpln workload force-redeployment RELEASE_NAME-mimir --gvc GVC_NAME.

Retention

retention:
  period: "0" # blocks retention, e.g. 30d, 13w, 1y — "0" keeps data forever

Volume Set

volumeset:
  capacity: 20 # GiB — ingester WAL/TSDB and compactor workspace at /data

Internal Access

internalAccess:
  type: same-gvc # options: none, same-gvc, same-org, workload-list
  workloads: [] # used with workload-list, e.g. //gvc/GVC/workload/NAME

Mimir has no built-in authentication, so the template has no public access option. internalAccess.type decides which workloads can reach port 8080.

Connecting

What	Address	Notes
Remote write (from your collectors)	`http://RELEASE_NAME-mimir.GVC_NAME.cpln.local:8080/api/v1/push`	Prometheus `remote_write` endpoint
PromQL query (Grafana data source)	`http://RELEASE_NAME-mimir.GVC_NAME.cpln.local:8080/prometheus`	Prometheus-compatible query API
Tenant header	`X-Scope-OrgID: TENANT_ID`	Required on every request when `multitenancy.enabled: true`
Credentials	None	Mimir has no built-in authentication. Reachability is controlled by `internalAccess`

There is no public endpoint. Collectors and Grafana must run as workloads that internalAccess admits (by default, any workload in the same GVC). Point Prometheus at the remote write endpoint:

remote_write:
  - url: http://RELEASE_NAME-mimir.GVC_NAME.cpln.local:8080/api/v1/push
    # headers:
    #   X-Scope-OrgID: team-a # only when multitenancy.enabled

In Grafana, add a Prometheus data source with the PromQL query URL above. When multitenancy is on, add a custom HTTP header X-Scope-OrgID with the tenant ID. To check Mimir from your machine, open a tunnel and request its readiness endpoint:

cpln port-forward RELEASE_NAME-mimir 8080:8080 --gvc GVC_NAME
curl http://127.0.0.1:8080/ready

The image is distroless and has no shell, so cpln workload exec cannot be used for debugging. Use the tunnel above or a client workload in the GVC instead.

Operations

Backing Up

Your metric blocks live in your object bucket, so protecting the bucket (for example with your provider’s versioning or replication) protects the data. The volume set holds only the ingester WAL and compactor scratch space. Reinstalling the template against the same bucket resumes with your existing data.

Upgrading from 1.0.0

Version 1.1.0 renamed the resource limit keys so it is clear which number is the limit:

Old key	New key
`resources.cpu`	`resources.maxCpu`
`resources.memory`	`resources.maxMemory`

Rename both in your values before upgrading. An upgrade that still carries the old names is refused at render, before anything is changed. If you are going straight to 1.2.0 with an S3-compatible backend, also follow Upgrading from 1.1.0.

Upgrading from 1.1.0

Version 1.2.0 moved the S3-compatible (storage.type: minio) access keys out of values and into a prerequisite secret. AWS and GCS installs are unaffected.

Create the credentials secret described under Prerequisites, with your current access key pair as accessKey and secretKey.
Delete storage.minio.accessKey and storage.minio.accessSecret from your values. An upgrade that still carries them is refused at render.
Upgrade with your existing storage settings plus the new secret name:

cpln helm upgrade RELEASE_NAME oci://ghcr.io/controlplane-com/templates/mimir \
  --version 1.2.0 --gvc GVC_NAME \
  --set storage.type=minio \
  --set storage.minio.endpoint=MINIO_HOST:9000 \
  --set storage.minio.bucket=BUCKET_NAME \
  --set storage.minio.credentialsSecretName=SECRET_NAME

Scaling and Availability

replicas: 1 (the default) runs a single instance. A restart briefly interrupts ingest and queries.
replicas: 3 or more forms an HA cluster with 3-way replicated ingest, so pushes and queries continue through a replica loss or a rolling restart. 2 is rejected because it gives no failure tolerance.
When scaling an existing install from 1 to 3 replicas, metrics written shortly before the scale-up can be intermittently missing from query results for up to about 12 hours. No data is lost, new writes are unaffected, and it resolves on its own. Scale up at a quiet time if that matters.
Raise resources.maxMemory to ingest more active series; memory is what bounds the series count.

Troubleshooting

Workload never becomes ready and logs are empty

Cause: with storage.type: minio, the secret named by storage.minio.credentialsSecretName does not exist, so no container starts.Fix: read status.versions[].message with cpln workload get-deployments RELEASE_NAME-mimir --gvc GVC_NAME -o yaml; it names the missing secret. Create it as shown under Prerequisites, then wait or run cpln workload force-redeployment RELEASE_NAME-mimir --gvc GVC_NAME.

Upgrade refused because resources.cpu was renamed

Error: mimir: resources.cpu was RENAMED to resources.maxCpu. (or the same for resources.memory)Cause: your values still use the pre-1.1.0 key names.Fix: rename them as described in Upgrading from 1.0.0.

Upgrade refused because the MinIO access keys were removed

Error: mimir: minio.accessKey and minio.accessSecret were REMOVEDCause: your values still carry the pre-1.2.0 S3-compatible keys.Fix: move them into a credentials secret as described in Upgrading from 1.1.0.

Install refused with replicas set to 2

Error: mimir: replicas must be 1 or >= 3 — a 2-replica cluster has no failure tolerance under 3-way replicationFix: set replicas to 1 or to 3 or more.

Pushes and queries rejected after enabling multitenancy

Cause: with multitenancy.enabled: true, every request without an X-Scope-OrgID header is rejected.Fix: add the header to every collector’s remote_write config and to the Grafana data source.

Access Denied warnings right after a fresh boot

Cause: on AWS and GCS, the workload identity’s cloud credentials are still being issued during the first seconds of a fresh boot.Fix: none needed. Mimir retries and proceeds on its own. If the warnings persist, check the cloud account, the IAM policy name and the bucket name.

Important Notes

Mimir has no built-in authentication and this template never exposes a public endpoint. To serve clients outside Control Plane, put your own authenticating proxy in front of it.
X-Scope-OrgID separates tenants but is not a security boundary: any client that can reach Mimir can send any tenant ID.
Changing retention.period applies to existing blocks too, because the compactor enforces it.
Your data lives in your bucket. Deleting data means emptying the bucket; uninstalling the template does not.
After uninstalling, re-check the bucket before emptying it: a terminating replica can re-write a small cluster-seed file under blocks/__mimir_cluster/ shortly after teardown.

External References

Grafana Mimir Documentation

Official Grafana Mimir documentation

Mimir HTTP API

Ingest, query, and status endpoints reference

Prometheus remote_write

Remote-write tuning and best practices

Grafana Mimir (GitHub)

Source code and releases

Mimir Template

View the source files, default values, and chart definition

Template Catalog

Templates

Overview

What Gets Created

Prerequisites

Installation

UI

CLI

Terraform

Pulumi

Configuration

Image and Resources

Object Storage

Tenancy

Retention

Volume Set

Internal Access

Connecting

Operations

Backing Up

Upgrading from 1.0.0

Upgrading from 1.1.0

Scaling and Availability

Troubleshooting

Important Notes

External References

Grafana Mimir Documentation

Mimir HTTP API

Prometheus remote_write

Grafana Mimir (GitHub)

Mimir Template

​Overview

​What Gets Created

​Prerequisites

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​Image and Resources

​Object Storage

​Tenancy

​Retention

​Volume Set

​Internal Access

​Connecting

​Operations

​Backing Up

​Upgrading from 1.0.0

​Upgrading from 1.1.0

​Scaling and Availability

​Troubleshooting

​Important Notes

​External References

Grafana Mimir Documentation

Mimir HTTP API

Prometheus remote_write

Grafana Mimir (GitHub)

Mimir Template

Overview

What Gets Created

Prerequisites

Installation

Configuration

Image and Resources

Object Storage

Tenancy

Retention

Volume Set

Internal Access

Connecting

Operations

Backing Up

Upgrading from 1.0.0

Upgrading from 1.1.0

Scaling and Availability

Troubleshooting

Important Notes

External References