> ## Documentation Index
> Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt
> Use this file to discover all available pages before exploring further.
# External Secret Syncer
> Deploy External Secret Syncer on Control Plane using the Template Catalog. Covers configuration, scaling, and syncing secrets from AWS, Vault, GCP Secret Manager, 1Password, Doppler, and Infisical.
## Overview
The External Secret Syncer (ESS) continuously syncs secrets and parameters from external providers into Control Plane secrets. This template deploys ESS as a workload that polls your configured providers on a set interval and creates or updates Control Plane secrets to match.
### How It Works
ESS runs as a workload on Control Plane. Your provider configuration and secrets list are stored in a Control Plane secret and mounted into the workload as `sync.yaml`. On startup, ESS schedules a polling loop for each configured secret. At each interval, it fetches the latest value from the external provider and creates or updates the corresponding Control Plane secret via the API.
ESS tags every secret it manages with `syncer.cpln.io/source` (set to the workload path). This prevents two ESS instances from accidentally overwriting each other's secrets. ESS only creates and updates secrets — it never deletes them, so removing a secret from `sync.yaml` leaves the existing Control Plane secret in place.
ESS watches its config file and automatically restarts when changes are detected (every \~5 seconds). No workload restart is needed after updating the config secret.
### Supported Providers
* [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html)
* [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/)
* [HashiCorp Vault](https://www.hashicorp.com/en/products/vault)
* [GCP Secret Manager](https://cloud.google.com/security/products/secret-manager)
* [1Password](https://1password.com/)
* [1Password Connect](https://developer.1password.com/docs/connect/)
* [Doppler](https://www.doppler.com/)
* [Infisical](https://infisical.com/)
### What Gets Created
* **Standard ESS Workload** — An ESS container with a readiness probe on `/about`.
* **Identity & Policy** — An identity bound to the workload with `manage` permissions on all secrets, allowing ESS to create and update Control Plane secrets.
* **Secret** — An opaque secret containing the sync configuration (`sync.yaml`) with providers and secret mappings.
This template does not create a GVC. You must deploy it into an existing GVC.
## Prerequisites
1. A secret or parameter stored in one of the [supported providers](#supported-providers).
2. Credentials with read access to the desired secret (API token, IAM keys, etc.). Alternatively, you can use a cloud access [identity](/reference/identity) instead of supplying keys directly.
### Installation
To install, follow the instructions for your preferred method:
Browse, install, and manage templates visually
Manage templates from your terminal
}>
Declare templates in your Terraform configurations
Pulumi Icon Streamline Icon: https://streamlinehq.com
}
>
Declare templates in your Pulumi programs
## Configuration
The default `values.yaml` for this template:
```yaml theme={null}
image: ghcr.io/controlplane-com/cpln-build/external-secret-syncer:v1.3.4
resources:
cpu: 200m
memory: 256Mi
port: 3004
allowedIp:
- 1.2.3.4 # Replace with your IP
essConfig:
providers:
- name: my-vault
vault:
address: https://my-vault.com:8200
token:
syncInterval: 1m
- name: my-aws-ssm
awsParameterStore:
region: us-east-1
accessKeyId: # alternatively configure identity to natively use AWS permissions
secretAccessKey: # alternatively configure identity to natively use AWS permissions
# - name: my-aws-secrets-manager
# awsSecretsManager:
# region: us-east-1
# accessKeyId:
# secretAccessKey:
# - name: my-1password
# onePassword:
# serviceAccountToken:
# integrationName: my-ess
# integrationVersion: 1.0.0
# - name: my-1password-connect
# onePasswordConnect:
# serverURL: https://my-connect-server.example.com
# token:
# - name: my-doppler
# doppler:
# accessToken:
# - name: my-gcp
# gcpSecretManager:
# projectId: 123456789876
# credentials:
# clientEmail:
# privateKey:
# - name: my-infisical
# infisical:
# clientId: # from an Infisical machine identity
# clientSecret:
# projectId:
secrets:
- name: auth
provider: my-vault
syncInterval: 20s
dictionary:
PORT:
path: /v1/secret/data/app
parse: data.port
default: 5432
PASSWORD:
path: /v1/secret/data/app
parse: data.password
USERNAME:
default: "no username"
path: /v1/secret/data/app
parse: data.username
- name: ssm
provider: my-aws
syncInterval: 20s
opaque: /example/app
# - name: secrets-manager
# provider: my-aws-secrets-manager
# dictionary:
# PASSWORD:
# path: /example/app
# parse: password
# - name: doppler-secret
# provider: my-doppler
# opaque: /project/config/SECRET_NAME
# - name: doppler-project
# provider: my-doppler
# dictionaryFromProject:
# path: project/config # syncs all secrets from a Doppler project+config
# - name: gcp
# provider: my-gcp
# opaque: database-password
# - name: gcp-project
# provider: my-gcp
# dictionaryFromProject: true # combines every accessible secret from the GCP project into one dictionary
# - name: gcp-discover
# provider: my-gcp
# discoverAllSecrets: true # GCP only — creates one Control Plane secret per project secret;
# # each secret's "cpln-type" label (opaque|dictionary) sets the type
# - name: infisical-secret
# provider: my-infisical
# opaque: dev/DATABASE_URL # format: "/"
# - name: json-config
# provider: my-aws-secrets-manager
# dictionaryFromJson: /example/app/config # fetches a JSON object and flattens it into a dictionary
```
### Top-Level Fields
* `image` — The ESS container image. Do not change unless upgrading.
* `resources.cpu` / `resources.memory` — Resource limits for the workload container.
* `port` — Port for the ESS HTTP admin API (default: `3004`). Used for health checks and manual sync triggers.
* `allowedIp` — List of CIDRs allowed to reach the ESS admin API externally. Replace the placeholder with your IP, or use `0.0.0.0/0` to allow all.
* `essConfig` — The full sync configuration — providers and secrets (see below).
### Providers
Each entry in `essConfig.providers` defines a connection to an external secret store. Every provider must have a unique `name`. An optional `syncInterval` sets the default polling interval for all secrets using that provider.
| Provider | Required Fields |
| ------------------- | --------------------------------------------------------------------- |
| HashiCorp Vault | `vault.address`, `vault.token` |
| AWS Parameter Store | `awsParameterStore.region` |
| AWS Secrets Manager | `awsSecretsManager.region` |
| GCP Secret Manager | `gcpSecretManager.projectId` |
| 1Password | `onePassword.serviceAccountToken` |
| 1Password Connect | `onePasswordConnect.serverURL`, `onePasswordConnect.token` |
| Doppler | `doppler.accessToken` |
| Infisical | `infisical.clientId`, `infisical.clientSecret`, `infisical.projectId` |
AWS providers optionally accept `accessKeyId` and `secretAccessKey`. If omitted, ESS falls back to credentials provided through the workload's cloud access [identity](/reference/identity).
GCP Secret Manager optionally accepts `credentials.clientEmail` and `credentials.privateKey`. If omitted, ESS uses Application Default Credentials.
**Provider examples:**
```yaml theme={null}
# HashiCorp Vault
- name: my-vault
vault:
address: https://my-vault.com:8200
token:
syncInterval: 1m
# AWS Parameter Store
- name: my-aws-ssm
awsParameterStore:
region: us-east-1
accessKeyId: # optional if using an IAM-linked identity
secretAccessKey: # optional if using an IAM-linked identity
# AWS Secrets Manager
- name: my-aws-secrets-manager
awsSecretsManager:
region: us-east-1
accessKeyId:
secretAccessKey:
# GCP Secret Manager
- name: my-gcp
gcpSecretManager:
projectId: 123456789876
credentials: # optional — omit to use Application Default Credentials
clientEmail:
privateKey:
# 1Password
- name: my-1password
onePassword:
serviceAccountToken:
integrationName: my-ess # optional
integrationVersion: 1.0.0 # optional
# 1Password Connect
- name: my-1password-connect
onePasswordConnect:
serverURL: https://my-connect-server.example.com
token:
# Doppler
- name: my-doppler
doppler:
accessToken: # use a Doppler service token (dp.st....)
# Infisical
- name: my-infisical
infisical:
clientId: example-client-id # create an Infisical machine identity for the ESS
clientSecret: example-client-secret
projectId: example-project-id
```
### Secrets
Each entry in `essConfig.secrets` maps an external secret to a Control Plane secret. Each secret must specify a `name`, a `provider`, and exactly one sync type.
#### `opaque` — Single value
Creates a Control Plane `opaque` secret from a single fetched value.
Shorthand (path only):
```yaml theme={null}
- name: my-secret
provider: my-vault
opaque: /v1/secret/data/myapp
```
With options:
```yaml theme={null}
- name: my-secret
provider: my-vault
opaque:
path: /v1/secret/data/myapp # path to fetch
parse: data.password # optional — extract a key from a JSON/YAML response
default: fallback-value # optional — used if fetch fails
encoding: base64 # optional — base64-decode the fetched value before storing
```
Vault KV engine secrets are nested under a `data` key. When using `parse`, start with `data` to access the secret content (e.g., `data.password`).
If you use the shorthand form with no `default`, a fetch failure causes the sync to fail with no fallback.
#### `dictionary` — Multiple values
Creates a Control Plane `dictionary` secret. Each key is fetched independently and supports `path`, `parse`, `default`, and `encoding`.
```yaml theme={null}
- name: auth
provider: my-vault
dictionary:
PORT:
path: /v1/secret/data/app
parse: data.port
default: 5432
PASSWORD:
path: /v1/secret/data/app
parse: data.password
USERNAME:
path: /v1/secret/data/app
parse: data.username
default: "no username"
```
A failure on one key does not block the others.
#### `dictionaryFromProject` — Sync an entire project
Syncs all secrets from a provider project in one operation, stored as a Control Plane `dictionary` secret. Only valid with a Doppler or GCP Secret Manager provider. The expected shape differs per provider.
**Doppler** — specify a `project/config` path:
```yaml theme={null}
- name: my-doppler-config
provider: my-doppler
dictionaryFromProject:
path: my-project/dev # format: "project/config" — exactly two segments
```
**GCP Secret Manager** — set to `true` to pull every accessible secret from the project configured on the provider. Each fetched secret's latest version becomes one key in the resulting dictionary. Secrets with no accessible latest version (no versions, disabled, or destroyed) are skipped.
```yaml theme={null}
- name: my-gcp-config
provider: my-gcp
dictionaryFromProject: true
```
Doppler requires the `{ path: ... }` object form; GCP Secret Manager requires the `true` form. Mixing them — or using either with another provider — causes ESS to exit at startup.
#### `dictionaryFromJson` — Flatten a JSON object
Fetches a single value that contains a JSON object and flattens it into a Control Plane `dictionary` secret. Set it to the path of the value to fetch. Valid with any provider.
```yaml theme={null}
- name: json-config
provider: my-aws-secrets-manager
dictionaryFromJson: /example/app/config # path to a value holding a JSON object
```
Each leaf in the JSON object becomes one dictionary key:
* **Nested objects** are flattened using dot notation (`{ "db": { "host": "x" } }` → key `db.host`).
* **Arrays** are JSON-stringified into the value (`{ "tags": ["a", "b"] }` → key `tags` with value `["a","b"]`).
* **`null`** is stored as the string `"null"`; numbers and booleans are stored as their string form.
For example, the JSON object `{"db":{"host":"db.internal","port":5432},"tags":["a","b"]}` produces a dictionary with keys `db.host` (`db.internal`), `db.port` (`5432`), and `tags` (`["a","b"]`).
If the fetched value is not a valid JSON object — a raw string, number, array, or malformed JSON — ESS stores the raw value under a single `__raw` key and logs a warning, so the secret stays usable as a dictionary type.
#### `discoverAllSecrets` — Mirror an entire GCP project (one secret each)
Discovers every accessible secret in the GCP project configured on the provider and creates a **separate** Control Plane secret for each one. This differs from `dictionaryFromProject: true`, which combines every secret into a single dictionary. GCP Secret Manager only, and must be set to `true`.
```yaml theme={null}
- name: gcp-discover # identifier only — created secrets are named after the GCP secrets
provider: my-gcp
discoverAllSecrets: true
```
The type of each created secret is controlled by a `cpln-type` **label** on the GCP secret:
* `cpln-type: dictionary` — the value is parsed as JSON and flattened into a Control Plane `dictionary` secret (same flattening rules as `dictionaryFromJson`, including the `__raw` fallback for non-JSON values).
* `cpln-type: opaque` (or no label) — stored as a Control Plane `opaque` secret.
Each Control Plane secret is named after its GCP secret, normalized to a valid Control Plane name (lowercased, with unsupported characters such as `_` replaced by `-` — e.g. `MY_API_KEY` → `my-api-key`). If two names normalize to the same value, the last one wins and a warning is logged. Created secrets carry a `syncer.cpln.io/discoveredBy` tag set to this entry's `name`. Secrets with no accessible latest version (no versions, disabled, or destroyed) are skipped.
`discoverAllSecrets` is only valid with a GCP Secret Manager provider. ESS does not delete a discovered Control Plane secret when its source GCP secret is later removed.
### Sync Interval
Intervals use the format `hms`. All parts are optional but at least one is required. Examples: `10s`, `5m`, `1h`, `1h30m`, `1h30m10s`.
Priority (highest wins):
1. Secret-level `syncInterval`
2. Provider-level `syncInterval`
3. Global default (`300s`)
### Doppler Path Formats
| Sync type | Path format | Example |
| ---------------------------- | ---------------------------- | -------------------------------- |
| `opaque` or `dictionary` key | `project/config/SECRET_NAME` | `my-app/production/DATABASE_URL` |
| `dictionaryFromProject` | `project/config` | `my-app/production` |
### Infisical Path Formats
The Infisical project is set on the provider (`infisical.projectId`). Secret paths are scoped to an environment within that project.
| Sync type | Path format | Example |
| ---------------------------- | -------------------------- | ------------------ |
| `opaque` or `dictionary` key | `/` | `dev/DATABASE_URL` |
## Synced Secret Output
A secret created by ESS will look like:
```yaml theme={null}
kind: secret
name: hello
description: hello
tags:
syncer.cpln.io/lastError: '' # populated if ESS encounters an error
syncer.cpln.io/source: //gvc//workload/
type: dictionary
data:
PORT: '1234'
PASSWORD: 'no pass' # default used if the key was not found
```
The `syncer.cpln.io/lastError` tag is empty on success. If ESS encounters an error syncing a secret, the tag is populated with the error message.
## Important Notes
* **Conflict protection** — If a Control Plane secret already exists and is managed by a different ESS instance, the sync for that secret will fail. Two ESS instances cannot manage the same secret.
* **Secret type changes** — Changing a secret from `opaque` to `dictionary` (or vice versa) causes ESS to delete the existing secret and recreate it. There is a brief window where the secret does not exist.
* **No automatic deletion** — ESS only creates and updates secrets; it never deletes them. Removing a secret from `sync.yaml`, or deleting a source secret upstream, leaves the existing Control Plane secret in place — delete unwanted secrets manually. ESS-managed secrets carry the `syncer.cpln.io/source` tag (and discovered secrets additionally carry `syncer.cpln.io/discoveredBy`) to help identify them.
* **Doppler `parse`** — The `parse` field only works when the Doppler secret's value is JSON or YAML. Using `parse` on a plain string secret throws an error.
* **Hot reload** — ESS watches its config file and automatically restarts when changes are detected (every \~5 seconds). No workload restart is needed after updating the config secret.
## External References
AWS Systems Manager Parameter Store documentation
AWS Secrets Manager documentation
HashiCorp Vault secret management
Google Cloud Secret Manager documentation
1Password secret management
Doppler secrets platform
Infisical secret management platform
Source code for the External Secret Syncer image
View the source files, default values, and chart definition