> ## Documentation Index
> Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenSearch

> Deploy OpenSearch on Control Plane using the Template Catalog. Covers configuration, volumes, scaling, and distributed search clustering with optional Dashboards and S3/GCS backups.

## Overview

OpenSearch is an open-source distributed search and analytics engine. This template deploys a production-ready OpenSearch cluster with automated node configuration, an optional Dashboards visualization UI, an optional demo log pipeline, and optional scheduled snapshot backups to AWS S3 or GCS.

### What Gets Created

* **Stateful Workload** — An OpenSearch cluster with a configurable number of replicas (must be odd). Installs S3 or GCS repository plugins at startup when backup is enabled.
* **Volume Set** — Persistent storage per replica with optional autoscaling.
* **Secret** — An opaque startup script secret that generates the OpenSearch node configuration and starts the process.
* **Identity & Policy** — An identity bound to the OpenSearch workload with `reveal` access to the startup script secret. When backup is enabled, the identity also grants cloud storage access to the cluster.
* **OpenSearch Dashboards Workload** (optional) — A web-based visualization UI for logs and data. Enabled when `dashboard.enabled: true`. Not exposed externally — access via `cpln port-forward`.
* **Demo Logs Workload** (optional) — A Python log generator with a Fluent Bit sidecar that ships sample logs to OpenSearch, plus a one-time setup job that creates index templates and dashboard patterns. Creates an additional Fluent Bit config secret with its own identity and policy. Enabled when `demoLogs.enabled: true`.
* **Backup Setup Workload** (optional) — A one-time job that registers the snapshot repository and creates an automated snapshot policy via the OpenSearch API. Enabled when `backup.enabled: true`.

<Note>
  This template does not create a GVC. You must deploy it into an existing GVC.
</Note>

## Prerequisites

Prerequisites are only required if you plan to enable automated backups (`backup.enabled: true`). Skip this section if backups are not needed.

### AWS S3

1. Create an S3 bucket. Set `backup.aws.bucket` and `backup.aws.region` in your values file.

2. If you do not have a Control Plane Cloud Account set up, follow the [Create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account) guide. Set `backup.aws.cloudAccountName` to the name of your Cloud Account.

3. Create an IAM policy with the following JSON, replacing `YOUR_BUCKET_NAME`:

```json theme={null}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:ListBucket",
                "s3:GetObjectVersion",
                "s3:DeleteObjectVersion"
            ],
            "Resource": [
                "arn:aws:s3:::YOUR_BUCKET_NAME/*"
            ]
        }
    ]
}
```

4. Set `backup.aws.policyName` to the name of the policy created in step 3.

### GCS

1. Create a GCS bucket. Set `backup.gcp.bucket` in your values file.

2. If you do not have a Control Plane Cloud Account set up, follow the [Create a Cloud Account](https://docs.controlplane.com/guides/create-cloud-account) guide. Set `backup.gcp.cloudAccountName` to the name of your Cloud Account.

3. Add the **Storage Admin** role to the GCP service account associated with the Cloud Account.

## Installation

To install, follow the instructions for your preferred method:

<CardGroup cols={2}>
  <Card title="UI" href="/template-catalog/install-manage/ui" icon="laptop">
    Browse, install, and manage templates visually
  </Card>

  <Card title="CLI" href="/template-catalog/install-manage/cli" icon="terminal">
    Manage templates from your terminal
  </Card>

  <Card title="Terraform" href="/template-catalog/install-manage/terraform" icon={<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128"><g fill-rule="evenodd"><path d="M77.941 44.5v36.836L46.324 62.918V26.082zm0 0" fill="#5c4ee5"/><path d="M81.41 81.336l31.633-18.418V26.082L81.41 44.5zm0 0" fill="#4040b2"/><path d="M11.242 42.36L42.86 60.776V23.941L11.242 5.523zm0 0M77.941 85.375L46.324 66.957v36.82l31.617 18.418zm0 0" fill="#5c4ee5"/></g></svg>}>
    Declare templates in your Terraform configurations
  </Card>

  <Card
    title="Pulumi"
    href="/template-catalog/install-manage/pulumi"
    icon={<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" id="Pulumi-Icon--Streamline-Svg-Logos" height="24" width="24">
    <desc>
        Pulumi Icon Streamline Icon: https://streamlinehq.com
    </desc>
    <path fill="#f26e7e" d="M4.683025 13.3318c0.869125 -0.5018 0.870575 -2.1264 0.003225 -3.62865s-2.27504 -2.313275 -3.1441725 -1.811475C0.672945 8.3935 0.6715 10.0181 1.53885 11.52035c0.86735 1.502275 2.27505 2.313275 3.144175 1.81145Zm0.0052 3.2167c0.86735 1.502275 0.865925 3.126875 -0.003225 3.628675 -0.86915 0.5018 -2.2768275 -0.309225 -3.144175 -1.81145 -0.8673525 -1.50225 -0.8659075 -3.126875 0.003225 -3.628675 0.8691325 -0.5018 2.276825 0.309225 3.144175 1.81145Zm5.922875 3.4243c0.86735 1.50225 0.8659 3.126775 -0.003225 3.62875 -0.869125 0.501775 -2.27685 -0.309325 -3.1442 -1.81155 -0.867325 -1.50225 -0.865875 -3.12685 0.00325 -3.628675 0.869125 -0.5018 2.276825 0.309225 3.144175 1.811475Zm-0.001925 -6.845275c0.86735 1.50225 0.8659 3.12685 -0.003225 3.628675 -0.869125 0.5018 -2.276825 -0.309225 -3.144175 -1.811475 -0.86735 -1.50225 -0.8659 -3.12685 0.003225 -3.62865 0.869125 -0.501825 2.276825 0.3092 3.144175 1.81145Z" stroke-width="0.25"></path>
    <path fill="#8a3391" d="M22.45775 11.524125c0.86725 -1.502225 0.865925 -3.12685 -0.003225 -3.62865 -0.869125 -0.501825 -2.276825 0.3092 -3.144175 1.811475 -0.86735 1.50225 -0.8659 3.126825 0.003225 3.62865 0.869125 0.501825 2.276825 -0.3092 3.144175 -1.811475Zm0.000175 3.2151c0.869075 0.5018 0.870625 2.1264 0.003225 3.62865 -0.86735 1.50225 -2.27505 2.313275 -3.144175 1.81145 -0.869125 -0.5018 -0.870575 -2.126425 -0.003225 -3.62865 0.86735 -1.50225 2.27505 -2.313275 3.144175 -1.81145ZM16.536225 18.157875c0.86915 0.501825 0.8706 2.126425 0.00325 3.628675 -0.86735 1.502125 -2.275075 2.313225 -3.1442 1.81145 -0.869125 -0.50175 -0.870575 -2.126425 -0.003225 -3.62865 0.867375 -1.502275 2.27505 -2.3133 3.144175 -1.811475Zm-0.003325 -6.843775c0.869125 0.5018 0.870575 2.126425 0.003225 3.628675s-2.27505 2.313275 -3.1442 1.811475c-0.869125 -0.501825 -0.870575 -2.126425 -0.003225 -3.628675 0.86735 -1.502275 2.27505 -2.313275 3.1442 -1.811475Z" stroke-width="0.25"></path>
    <path fill="#f7bf2a" d="M15.138225 2.06721c0 1.003615 -1.40625 1.817215 -3.14095 1.817215 -1.7347 0 -3.14095 -0.8136 -3.14095 -1.817215C8.856325 1.06359 10.262575 0.25 11.997275 0.25c1.7347 0 3.14095 0.81359 3.14095 1.81721ZM9.2166 5.482375c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172s1.40625 -1.817225 3.14095 -1.817225c1.7347 0 3.14095 0.8136 3.14095 1.817225Zm8.71005 1.8172c1.7347 0 3.14095 -0.813575 3.14095 -1.8172s-1.40625 -1.817225 -3.14095 -1.817225c-1.7347 0 -3.14095 0.8136 -3.14095 1.817225s1.40625 1.8172 3.14095 1.8172Zm-2.788425 1.605625c0 1.003625 -1.40625 1.8172 -3.14095 1.8172 -1.7347 0 -3.14095 -0.813575 -3.14095 -1.8172 0 -1.0036 1.40625 -1.8172 3.14095 -1.8172 1.7347 0 3.14095 0.8136 3.14095 1.8172Z" stroke-width="0.25"></path>
    </svg>}
  >
    Declare templates in your Pulumi programs
  </Card>
</CardGroup>

## Configuration

The default `values.yaml` for this template:

```yaml theme={null}
image: opensearchproject/opensearch:3.4.0

replicas: 3 # Must be odd

clusterName: my-opensearch-cluster

resources:
  minCpu: 500m
  minMemory: 2Gi
  maxCpu: 1
  maxMemory: 4Gi

volumeset:
  capacity: 10 # initial capacity in GiB (minimum is 10)
  autoscaling:
    enabled: false
    maxCapacity: 100 # Maximum capacity in GiB
    minFreePercentage: 10 # Trigger scaling when free space drops below this percentage
    scalingFactor: 1.2 # Multiply current capacity by this factor when scaling up

internal_access:
  type: same-gvc # options: same-gvc, same-org, workload-list
  workloads: # Note: can only be used if type is same-gvc or workload-list
    #- //gvc/GVC_NAME/workload/WORKLOAD_NAME

# Dashboard Configuration (recommended)
dashboard:
  enabled: true
  image: opensearchproject/opensearch-dashboards:3.4.0
  resources:
    cpu: 100m
    memory: 512Mi

# Demo Logs Configuration (optional)
demoLogs:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

# Backup Configuration (optional)
backup:
  enabled: false
  remove_setup_workload: false # Set to true after setup completes to reduce resource usage

  provider: aws # Options: aws or gcp

  schedule: "0 2 * * *" # Daily at 2am UTC

  retention:
    maxAge: 30d     # Delete snapshots older than 30 days
    maxCount: 30    # Keep maximum 30 snapshots

  aws:
    bucket: my-s3-bucket
    region: us-east-1
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account
    policyName: my-backup-policy

  gcp:
    bucket: my-gcs-bucket
    prefix: opensearch-snapshots
    cloudAccountName: my-cloud-account
```

### Cluster

* `replicas` — Number of OpenSearch nodes. Must be an odd number (3, 5, 7) to ensure quorum.
* `clusterName` — Name used for internal cluster coordination.

<Note>
  Use a minimum of 3 replicas for high availability. For production workloads, plan resources based on log volume: 1 CPU / 4Gi handles 10–50 GB/day; scale to 2 CPU / 8Gi for 50–100 GB/day.
</Note>

### Resources

* `resources.minCpu` / `resources.minMemory` — Minimum CPU and memory guaranteed per node.
* `resources.maxCpu` / `resources.maxMemory` — Maximum CPU and memory per node.

### Storage

* `volumeset.capacity` — Initial volume size per node in GiB (minimum 10).
* `volumeset.autoscaling.enabled` — Automatically expand each volume as it fills. When enabled:
  * `maxCapacity` — Maximum volume size in GiB.
  * `minFreePercentage` — Trigger a scale-up when free space drops below this percentage.
  * `scalingFactor` — Multiply the current capacity by this factor when scaling up.

### Internal Access

* `internal_access.type` — Controls which workloads can connect to OpenSearch on port `9200`:

| Type            | Description                                                     |
| --------------- | --------------------------------------------------------------- |
| `same-gvc`      | Allow access from all workloads in the same GVC                 |
| `same-org`      | Allow access from all workloads in the same organization        |
| `workload-list` | Allow access only from specific workloads listed in `workloads` |

<Note>
  External access to OpenSearch is blocked by default. The Dashboards UI is also not exposed externally — use `cpln port-forward` to access it.
</Note>

### OpenSearch Dashboards

* `dashboard.enabled` — When `true`, deploys an OpenSearch Dashboards workload for log visualization and search. Recommended for most deployments.
* `dashboard.resources.cpu` / `dashboard.resources.memory` — CPU and memory for the Dashboards workload.

To access the Dashboards UI, use port-forwarding:

```bash theme={null}
cpln port-forward WORKLOAD_NAME --location LOCATION --org ORG_NAME 5601:5601
```

Then open `http://localhost:5601` in your browser.

### Demo Logs

* `demoLogs.enabled` — When `true`, deploys a sample log generator with a Fluent Bit sidecar that ships logs to OpenSearch, along with a setup job that creates the index template and dashboard pattern. Useful for testing the pipeline end-to-end.
* `demoLogs.remove_setup_workload` — Set to `true` after the setup job completes (\~1–2 minutes) to remove the one-time setup workload and reduce resource usage. The log pipeline continues running.

### Backup

Set `backup.enabled: true` to enable automated OpenSearch snapshot backups.

The backup setup workload registers the snapshot repository and creates the snapshot policy via the OpenSearch API. Once it completes successfully, set `backup.remove_setup_workload: true` and run `cpln helm upgrade` to remove it — snapshots will continue on schedule.

* `backup.provider` — `aws` or `gcp`.
* `backup.schedule` — Cron schedule for automated snapshots (UTC).
* `backup.retention.maxAge` — Delete snapshots older than this duration.
* `backup.retention.maxCount` — Maximum number of snapshots to retain.

Set `backup.provider` to `aws` or `gcp` and fill in the corresponding provider section.

### Restoring a Snapshot

Restore snapshots from any workload that can reach the cluster on port `9200`.

**Restore to an empty cluster:**

```bash theme={null}
curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'
```

**Restore to an existing cluster (close indices first):**

```bash theme={null}
curl -X POST "http://WORKLOAD_NAME:9200/_all/_close"

curl -X POST "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/SNAPSHOT_NAME/_restore" \
  -H 'Content-Type: application/json' \
  -d '{
    "indices": "*",
    "ignore_unavailable": true,
    "include_global_state": false
  }'

curl -X POST "http://WORKLOAD_NAME:9200/_all/_open"
```

**List available snapshots:**

```bash theme={null}
curl "http://WORKLOAD_NAME:9200/_snapshot/backup-repo/_all?pretty"
```

### Connecting to OpenSearch

Once deployed, connect to the cluster from within the same GVC using:

```text theme={null}
http://RELEASE_NAME-opensearch.GVC_NAME.cpln.local:9200
```

## External References

<CardGroup cols={2}>
  <Card title="OpenSearch Documentation" icon="book" href="https://opensearch.org/docs/latest/">
    Official OpenSearch documentation
  </Card>

  <Card title="OpenSearch Dashboards" icon="chart-line" href="https://opensearch.org/docs/latest/dashboards/">
    OpenSearch Dashboards documentation
  </Card>

  <Card title="Snapshot Management" icon="database" href="https://opensearch.org/docs/latest/tuning-your-cluster/availability-and-recovery/snapshots/snapshot-management/">
    OpenSearch snapshot and backup guide
  </Card>

  <Card title="OpenSearch Template" icon="github" href="https://github.com/controlplane-com/templates/tree/main/opensearch">
    View the source files, default values, and chart definition
  </Card>
</CardGroup>