OpenSearch

​

Overview

​

What Gets Created

• Stateful Workload — An OpenSearch cluster with a configurable number of replicas (must be odd). Installs S3 or GCS repository plugins at startup when backup is enabled.
• Volume Set — Persistent storage per replica with optional autoscaling.
• Secret — An opaque startup script secret that generates the OpenSearch node configuration and starts the process.
• Identity & Policy — An identity bound to the OpenSearch workload with reveal access to the startup script secret. When backup is enabled, the identity also grants cloud storage access to the cluster.
• OpenSearch Dashboards Workload (optional) — A web-based visualization UI for logs and data. Enabled when dashboard.enabled: true. Not exposed externally — access via cpln port-forward.
• Demo Logs Workload (optional) — A Python log generator with a Fluent Bit sidecar that ships sample logs to OpenSearch, plus a one-time setup job that creates index templates and dashboard patterns. Creates an additional Fluent Bit config secret with its own identity and policy. Enabled when demoLogs.enabled: true.
• Backup Setup Workload (optional) — A one-time job that registers the snapshot repository and creates an automated snapshot policy via the OpenSearch API. Enabled when backup.enabled: true.

​

Prerequisites

​

AWS S3

1. Create an S3 bucket. Set backup.aws.bucket and backup.aws.region in your values file.
2. If you do not have a Control Plane Cloud Account set up, follow the Create a Cloud Account guide. Set backup.aws.cloudAccountName to the name of your Cloud Account.
3. Create an IAM policy with the following JSON, replacing YOUR_BUCKET_NAME:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}

4. Set backup.aws.policyName to the name of the policy created in step 3.

​

GCS

1. Create a GCS bucket. Set backup.gcp.bucket in your values file.
2. If you do not have a Control Plane Cloud Account set up, follow the Create a Cloud Account guide. Set backup.gcp.cloudAccountName to the name of your Cloud Account.
3. Add the Storage Admin role to the GCP service account associated with the Cloud Account.

​

Installation

​

Configuration

image: opensearchproject/opensearch:3.4.0

replicas: 3 # Must be odd

clusterName: my-opensearch-cluster

resources:
  minCpu: 500m
  minMemory: 2Gi
  maxCpu: 1
  maxMemory: 4Gi

volumeset:
  capacity: 10 # initial capacity in GiB (minimum is 10)
  autoscaling:
    enabled: false
    maxCapacity: 100 # Maximum capacity in GiB
    minFreePercentage: 10 # Trigger scaling when free space drops below this percentage
    scalingFactor: 1.2 # Multiply current capacity by this factor when scaling up

internal_access:
  type: same-gvc # options: same-gvc, same-org, workload-list
  workloads: # Note: can only be used if type is same-gvc or workload-list
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

# Dashboard Configuration (recommended)
dashboard:
  enabled: true
  image: opensearchproject/opensearch-dashboards:3.4.0
  resources:
    cpu: 100m
    memory: 512Mi

# Demo Logs Configuration (optional)
demoLogs:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

# Backup Configuration (optional)
backup:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

  provider: aws # Options: aws or gcp

  schedule: "0 2 * * *" # Daily at 2am UTC

  retention:
    maxAge: 30d     # Delete snapshots older than 30 days
    maxCount: 30    # Keep maximum 30 snapshots

  aws:
    bucket: my-s3-bucket
    region: us-east-1
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account
    policyName: my-backup-policy

  gcp:
    bucket: my-gcs-bucket
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account

​

Cluster

• replicas — Number of OpenSearch nodes. Must be an odd number (3, 5, 7) to ensure quorum.
• clusterName — Name used for internal cluster coordination.

​

Resources

• resources.minCpu / resources.minMemory — Minimum CPU and memory guaranteed per node.
• resources.maxCpu / resources.maxMemory — Maximum CPU and memory per node.

​

Storage

• volumeset.capacity — Initial volume size per node in GiB (minimum 10).
• volumeset.autoscaling.enabled — Automatically expand each volume as it fills. When enabled:
  • maxCapacity — Maximum volume size in GiB.
  • minFreePercentage — Trigger a scale-up when free space drops below this percentage.
  • scalingFactor — Multiply the current capacity by this factor when scaling up.

​

Internal Access

• internal_access.type — Controls which workloads can connect to OpenSearch on port 9200:

​

OpenSearch Dashboards

• dashboard.enabled — When true, deploys an OpenSearch Dashboards workload for log visualization and search. Recommended for most deployments.
• dashboard.resources.cpu / dashboard.resources.memory — CPU and memory for the Dashboards workload.

cpln port-forward WORKLOAD_NAME --location LOCATION --org ORG_NAME 5601:5601

​

Demo Logs

• demoLogs.enabled — When true, deploys a sample log generator with a Fluent Bit sidecar that ships logs to OpenSearch, along with a setup job that creates the index template and dashboard pattern. Useful for testing the pipeline end-to-end.
• demoLogs.remove_setup_workload — Set to true after the setup job completes (~1–2 minutes) to remove the one-time setup workload and reduce resource usage. The log pipeline continues running.

​

Backup

• backup.provider — aws or gcp.
• backup.schedule — Cron schedule for automated snapshots (UTC).
• backup.retention.maxAge — Delete snapshots older than this duration.
• backup.retention.maxCount — Maximum number of snapshots to retain.

​

Restoring a Snapshot

curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

curl -X POST "http://WORKLOAD_NAME:9200/_all/_close"

curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

curl -X POST "http://WORKLOAD_NAME:9200/_all/_open"

curl "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/_all?pretty"

​

Connecting to OpenSearch

http://RELEASE_NAME-opensearch.GVC_NAME.cpln.local:9200

​

External References