Polaris

​

Overview

​

What Gets Created

​

Prerequisites

cpln secret create-dictionary --name ROOT_CREDENTIALS_SECRET_NAME \
  --entry CLIENT_ID=root \
  --entry CLIENT_SECRET="$(openssl rand -hex 24)"

printf '%s' "$(openssl rand -hex 32)" | cpln secret create-opaque --name SIGNING_KEY_SECRET_NAME --encoding plain -f -

cpln workload get-deployments RELEASE_NAME-polaris --gvc GVC_NAME -o yaml

cpln workload force-redeployment RELEASE_NAME-polaris --gvc GVC_NAME

• Object-storage credentials for the bucket that holds your Iceberg tables. Polaris runs without them and catalogs can be added later — see Building a Lakehouse.
• A bucket and Cloud Account for metastore backups — see Backing Up.

​

Installation

cpln helm install RELEASE_NAME oci://ghcr.io/controlplane-com/templates/polaris \
  --version 1.2.1 --gvc GVC_NAME \
  --set rootCredentials.secretName=ROOT_CREDENTIALS_SECRET_NAME \
  --set tokenSigningKey.secretName=SIGNING_KEY_SECRET_NAME \
  --set postgres.credentials.password='YOUR-STRONG-PASSWORD'

​

Configuration

​

Server

image: apache/polaris:1.7.0

# Stateless replicas behind one metastore. All replicas share the token signing
# key below, so a token minted by one is accepted by every other.
replicas: 1

resources:
  minCpu: 500m
  maxCpu: 1000m
  minMemory: 1Gi
  maxMemory: 2Gi # JVM heap is a percentage OF THIS — see jvm.maxRAMPercentage

• replicas — A fixed number of interchangeable Polaris replicas (minimum 1; there is no request-based autoscaling). See Scaling and Availability.
• resources — Per replica. minMemory may not exceed maxMemory, and maxCpu may not exceed four times minCpu; the chart refuses to render otherwise and names the value to fix.

​

JVM

jvm:
  maxRAMPercentage: 70

​

Realm

realm: POLARIS

​

Credentials

rootCredentials:
  secretName: my-polaris-root-credentials

tokenSigningKey:
  secretName: my-polaris-signing-key

• rootCredentials.secretName — The prerequisite dictionary secret holding CLIENT_ID and CLIENT_SECRET. Required.
• tokenSigningKey.secretName — The prerequisite opaque secret whose payload is the shared token signing key. Required.

​

Object Storage

storage:
  credentialsSecretName: "" # e.g. my-seaweedfs-s3-credentials
  region: us-east-1 # AWS_REGION; S3-compatible servers ignore the value but the SDK requires one

• storage.credentialsSecretName — Name of a dictionary secret holding AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, used for Polaris’s own reads and writes of Iceberg metadata. Empty by default, which is a valid install: the server runs and answers, but catalogs on object storage cannot read or write. Per-provider setup is in Building a Lakehouse.
• storage.region — The bucket’s region. S3-compatible servers ignore it, but the AWS SDK requires a value.

​

Bootstrap

bootstrap:
  image: apache/polaris-admin-tool:1.7.0
  resources:
    minCpu: 100m
    maxCpu: 400m
    minMemory: 128Mi
    maxMemory: 1Gi

​

Access

publicAccess:
  enabled: false # true = Iceberg REST + management API on the auto *.cpln.app HTTPS endpoint

internalAccess: # who may reach Polaris from inside Control Plane
  type: same-gvc # options: none, same-gvc, same-org, workload-list ('none' hides it from in-GVC engines)
  workloads: [] # used with workload-list

​

Single-Instance Metastore

postgres:
  enabled: true
  credentials:
    username: polaris
    password: change-me-polaris-db # change before installing
    database: polaris
  config:
    credentialsSecretName: my-polaris-db-credentials
  resources:
    minCpu: 200m
    maxCpu: 500m
    minMemory: 256Mi
    maxMemory: 512Mi
  volumeset:
    capacity: 10 # initial capacity in GiB (minimum is 10)

  backup: # optional database backups (Postgres 17+) — see README storage setup
    enabled: false
    image: ghcr.io/controlplane-com/backup-images/postgres-backup:18.1.0
    schedule: "0 2 * * *"
    resources:
      cpu: 100m
      memory: 128Mi
    provider: aws # options: aws, gcp, minio
    aws:
      bucket: my-polaris-backup-bucket
      region: us-east-1
      cloudAccountName: my-s3-cloud-account
      policyName: my-polaris-backup-policy
      prefix: postgres/backups
    gcp:
      bucket: my-polaris-backup-bucket
      cloudAccountName: my-gcs-cloud-account
      prefix: postgres/backups
    minio:
      endpoint: http://my-minio-workload:9000
      bucket: my-polaris-backup-bucket
      credentialsSecretName: my-polaris-minio-credentials
      prefix: postgres/backups

• postgres.credentials.* — The metastore’s username, password and database name. Change the password before installing. The chart writes these three values into the credentials secret in both metastore modes, even though they sit under the postgres block, and Polaris connects to the database named here.
• postgres.config.credentialsSecretName — Name of the dictionary secret this chart creates in single-instance mode. Secret names are organization-wide, so give each Polaris release its own: a second release left on the default name is refused at install and creates nothing, and the first release is unaffected.
• postgres.backup.* — Optional scheduled backups. See Backing Up.

​

Highly Available Metastore

postgresHA:
  enabled: false
  config:
    credentialsSecretName: my-polaris-db-credentials
  replicas: 3
  volumeset:
    capacity: 10 # initial capacity in GiB per replica (minimum is 10)

  backup: # optional database backups — see README storage setup
    enabled: false
    mode: logical # logical or wal-g
    resources:
      cpu: 100m
      memory: 128Mi
    logical:
      image: ghcr.io/controlplane-com/backup-images/postgres-backup:17.1.0
      schedule: "0 2 * * *"
    walg:
      intervalSeconds: 21600
    provider: aws # options: aws, gcp, minio
    aws:
      bucket: my-polaris-backup-bucket
      region: us-east-1
      cloudAccountName: my-s3-cloud-account
      policyName: my-polaris-backup-policy
      prefix: postgres/backups
    gcp:
      bucket: my-polaris-backup-bucket
      cloudAccountName: my-gcs-cloud-account
      prefix: postgres/backups
    minio:
      endpoint: http://my-minio-workload:9000
      bucket: my-polaris-backup-bucket
      credentialsSecretName: my-polaris-minio-credentials
      prefix: postgres/backups

• postgresHA.config.credentialsSecretName — Name of the dictionary secret this chart creates in HA mode, built from postgres.credentials.* above. Give each release its own name, as in single-instance mode.
• postgresHA.replicas / postgresHA.volumeset.capacity — PostgreSQL replica count and volume size per replica. The remaining cluster settings take the defaults of the PostgreSQL Highly Available template; its HAProxy endpoint must stay enabled, because it is Polaris’s database address.
• postgresHA.backup.* — Optional backups as scheduled logical dumps or continuous WAL-G archiving. See Backing Up.

​

Connecting

cpln port-forward RELEASE_NAME-polaris 8181:8181 --gvc GVC_NAME

curl -X POST http://127.0.0.1:8181/api/catalog/v1/oauth/tokens \
  --user "$CLIENT_ID:$CLIENT_SECRET" \
  -d grant_type=client_credentials -d scope=PRINCIPAL_ROLE:ALL

​

Building a Lakehouse

cpln secret create-dictionary --name S3_CREDENTIALS_SECRET_NAME \
  --entry AWS_ACCESS_KEY_ID=MINIO_USERNAME \
  --entry AWS_SECRET_ACCESS_KEY='MINIO_PASSWORD'

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": ["s3:ListBucket", "s3:GetBucketLocation"], "Resource": "arn:aws:s3:::YOUR_BUCKET" },
    { "Effect": "Allow", "Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"], "Resource": "arn:aws:s3:::YOUR_BUCKET/*" }
  ]
}

cpln secret create-dictionary --name S3_CREDENTIALS_SECRET_NAME \
  --entry AWS_ACCESS_KEY_ID=ACCESS_KEY_ID \
  --entry AWS_SECRET_ACCESS_KEY='SECRET_ACCESS_KEY'

curl -X POST BASE_URL/api/management/v1/catalogs \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{
  "catalog": {
    "name": "lakehouse",
    "type": "INTERNAL",
    "properties": { "default-base-location": "s3://YOUR_BUCKET/warehouse" },
    "storageConfigInfo": {
      "storageType": "S3",
      "allowedLocations": ["s3://YOUR_BUCKET/warehouse"],
      "endpoint": "http://SEAWEEDFS_WORKLOAD_NAME.GVC_NAME.cpln.local:8333",
      "endpointInternal": "http://SEAWEEDFS_WORKLOAD_NAME.GVC_NAME.cpln.local:8333",
      "pathStyleAccess": true,
      "stsUnavailable": true,
      "region": "us-east-1"
    }
  }
}'

# Let the root principal administer the new catalog
curl -X PUT BASE_URL/api/management/v1/principal-roles/service_admin/catalog-roles/lakehouse \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"catalogRole":{"name":"catalog_admin"}}'

catalogs:
  - name: iceberg
    properties: |
      connector.name=iceberg
      iceberg.catalog.type=rest
      iceberg.rest-catalog.uri=http://POLARIS_WORKLOAD_NAME.GVC_NAME.cpln.local:8181/api/catalog
      iceberg.rest-catalog.security=OAUTH2
      iceberg.rest-catalog.oauth2.credential=${ENV:POLARIS_CLIENT_ID}:${ENV:POLARIS_CLIENT_SECRET}
      iceberg.rest-catalog.oauth2.scope=PRINCIPAL_ROLE:ALL
      iceberg.rest-catalog.warehouse=lakehouse
      iceberg.rest-catalog.vended-credentials-enabled=false
      fs.native-s3.enabled=true
      s3.endpoint=http://SEAWEEDFS_WORKLOAD_NAME.GVC_NAME.cpln.local:8333
      s3.region=us-east-1
      s3.path-style-access=true
      s3.aws-access-key=${ENV:S3_ACCESS_KEY}
      s3.aws-secret-key=${ENV:S3_SECRET_KEY}
    secrets:
      - env: POLARIS_CLIENT_ID
        secretName: ROOT_CREDENTIALS_SECRET_NAME
        secretKey: CLIENT_ID
      - env: POLARIS_CLIENT_SECRET
        secretName: ROOT_CREDENTIALS_SECRET_NAME
        secretKey: CLIENT_SECRET
      - env: S3_ACCESS_KEY
        secretName: S3_CREDENTIALS_SECRET_NAME
        secretKey: AWS_ACCESS_KEY_ID
      - env: S3_SECRET_KEY
        secretName: S3_CREDENTIALS_SECRET_NAME
        secretKey: AWS_SECRET_ACCESS_KEY

CREATE SCHEMA iceberg.demo;
CREATE TABLE iceberg.demo.t (a integer, b varchar);
INSERT INTO iceberg.demo.t VALUES (1,'alpha'), (2,'beta'), (3,'gamma');
SELECT count(*) FROM iceberg.demo.t;

​

Other Iceberg Clients

​

Operations

​

Backing Up

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "s3:ListBucket",
      "s3:GetBucketLocation",
      "s3:GetObject",
      "s3:PutObject",
      "s3:DeleteObject",
      "s3:AbortMultipartUpload"
    ],
    "Resource": [
      "arn:aws:s3:::YOUR_BUCKET",
      "arn:aws:s3:::YOUR_BUCKET/*"
    ]
  }]
}

cpln secret create-dictionary --name MINIO_CREDENTIALS_SECRET_NAME \
  --entry accessKey=MINIO_ACCESS_KEY \
  --entry secretKey='MINIO_SECRET_KEY'

cpln workload get-deployments RELEASE_NAME-postgres-ha-backup --gvc GVC_NAME -o yaml

​

Restoring a Backup

1. Keep realm identical to the value the backup was taken under, or the restored catalogs stay invisible.
2. Restore into a metastore that does not already hold a bootstrapped realm. A new release bootstraps its realm on first start, so restoring over it collides with the objects the bootstrap created. After a restore, force a redeployment of RELEASE_NAME-polaris so every replica reconnects.

​

Upgrading from 1.0.x

​

Upgrading from 1.1.0

​

Upgrading from 1.2.0

​

Scaling and Availability

• The server scales horizontally. Replicas share only the metastore and the token signing key, so a token minted by one replica is accepted by every other and a catalog created through one is visible on all of them. At the default replicas: 1, an upgrade replaces the only replica and requests fail while it restarts; with replicas: 2 or more, a rolling upgrade keeps serving.
• Mind the metastore’s connection limit. Each Polaris replica opens up to 20 database connections, plus one for the bootstrap workload, against PostgreSQL’s default limit of 100.
• The bootstrap workload always runs one replica and does nothing after its first run. Do not scale it.
• The metastore is the other half of availability. With the default single-instance postgres, a database restart takes Polaris down until PostgreSQL is back. postgresHA adds automatic failover behind a stable endpoint, at the cost of a larger footprint.
• The first upgrade after an install can restart the bundled database even when no database value changed. Expect a short outage on a single-instance metastore.
• Uninstall deletes the metastore volume sets and the chart-created credentials secret, taking every catalog definition with them. The Iceberg files in your bucket survive, but nothing indexes them any more. Your two prerequisite secrets are not deleted.

​

Troubleshooting

​

Important Notes

• Create both prerequisite secrets before installing, or the workloads wedge silently waiting on a secret that does not exist.
• Change postgres.credentials.password before installing. The default is a published placeholder, and it is the metastore password in both modes.
• realm is permanent. Renaming it hides every existing catalog behind a new, empty realm.
• Root credentials are write-once. Rotate by creating a new principal through the management API.
• Rotating the token signing key invalidates every outstanding token. Clients must request a new one.
• No credential vending. Polaris and each query engine hold their own object-storage credentials; keep iceberg.rest-catalog.vended-credentials-enabled=false in Trino.
• Polaris does not migrate its own database schema. Treat a future Polaris version change as an explicit schema step, not something startup handles.
• Uninstall deletes the metastore and every catalog definition with it. Enable backups or use postgresHA for anything you care about.

​

External References