Skip to main content

​
Overview

etcd is a distributed, reliable key-value store designed for the most critical data of a distributed system. It provides consistent coordination, service discovery, and configuration management across distributed systems, making it a common foundation for cluster health and orchestration. This template deploys an etcd cluster as a stateful workload with configurable replica count, persistent storage, and auto-compaction configured so the backend does not grow without bound.

​
What Gets Created

  • Stateful Workload — An etcd cluster with a configurable number of replicas (default: 3).
  • Volume Set — Persistent storage for etcd data.
  • Secret — An opaque startup script secret that handles cluster initialization, peer URL configuration, and replica setup.
  • Identity & Policy — An identity bound to the workload with reveal access to the startup script secret.
This template does not create a GVC. You must deploy it into an existing GVC.

​
Installation

This template has no external prerequisites. To install, follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

​
Configuration

The default values.yaml for this template:

​
Replicas

  • replicas — Number of etcd instances in the cluster (minimum 3).
The replica count must always be an odd number. etcd uses the Raft consensus algorithm, which requires a majority quorum — an odd number of members ensures a quorum can always be reached and avoids split-brain scenarios.

​
Resources

  • resources.cpu / resources.memory — CPU and memory allocated to each etcd instance (default: 1 CPU, 2Gi). These can be lowered for lighter workloads. See the etcd hardware recommendations for guidance.

​
Storage

  • volumeset.capacity — Persistent volume size in GiB for etcd data (minimum 10).

​
Compaction and Backend Growth

etcd never discards a superseded revision on its own. Every write creates a new revision and the old one is kept until a compaction removes it, so the backend grows with time alone whenever a client writes on a timer — Patroni, for example, renews its leader lease roughly every 10 seconds, which produces new revisions whether or not any application data changes. Measured on an otherwise idle cluster: about 151,000 revisions and 19 MB per day, which reaches etcd’s default 2 GiB backend quota in roughly 110 days. When the quota is reached, etcd raises a cluster-wide NOSPACE alarm and every member goes read-only until an operator intervenes. Auto-compaction is what prevents that, and this template configures it by default.
  • tuning.autoCompactionMode — How autoCompactionRetention is read. periodic treats it as a time window; revision treats it as a number of revisions to keep. There is no “off” — an unrecognized mode is rejected at render time.
  • tuning.autoCompactionRetention — How much history is kept before old revisions are discarded. The default 1h keeps the backend flat. Raising it to 24h or more buys a longer history window at the cost of a larger backend.
  • tuning.quotaBackendBytes — Backend size ceiling in bytes. 0 (the default) means the flag is not passed at all and etcd applies its own 2 GiB limit. Leave it at 0 unless the keyspace genuinely outgrows that; 8 GiB (8589934592) is etcd’s own suggested maximum, above which it warns at startup.
In periodic mode a retention value without a unit means hours. etcd reads a bare 30 as thirty hours, not thirty minutes. This template rejects an unsuffixed value at render time rather than letting it become a silently wrong window — write 30m, 1h or 24h.
quotaBackendBytes takes a plain byte count, not a size suffix. etcd itself refuses --quota-backend-bytes 2Gi at boot and the cluster would crash-loop, so a 2Gi-style value is rejected at render time instead. Write 2147483648.
Compaction is deliberately not switchable off: a retention of 0, an unrecognized mode and a negative quota (which etcd reads as “no quota at all”) are each rejected at render time. Each one produces a chart that installs cleanly and fails weeks later.
Compaction frees pages for reuse inside the backend file; it does not shrink the file. The reported dbSize therefore plateaus rather than dropping, while dbSizeInUse falls back to the size of the live keyspace. That is the expected behavior and is sufficient to stay under the quota, because reclaimed pages are reused for new writes. Only etcdctl defrag returns space to the filesystem, and this template does not automate it — defragmentation blocks the member it runs on.Testing measured the mechanism directly on a shortened retention window: the compacted revision advanced from 1 to 4001, dbSizeInUse collapsed from 5.76 MB to 20,480 bytes while dbSize stayed at about 5.8 MB, and reads of pre-compaction revisions then failed with required revision has been compacted. The multi-day dbSize plateau is derived from that mechanism rather than observed over days.
etcdctl endpoint status reports QUOTA | 0 B when quotaBackendBytes is 0, because that column reflects the flag rather than the effective limit. The effective quota in that case is still etcd’s 2 GiB default, which etcd logs at startup as enabled backend quota with default value.

​
Multi-Zone

  • multiZone — When true, distributes replicas equally across available zones for higher availability.
Not all locations support multi-zone deployments. Confirm that your target location supports multi-zone before enabling this option.

​
Internal Access

The internal_access section controls which workloads can reach the etcd cluster:

​
If the Backend Quota Is Already Full

Auto-compaction is configured from template version 1.4.2 onward. Earlier versions passed no compaction flags at all, so a cluster installed from one of them retains every revision it has ever written and grows until it hits the quota.
Upgrading turns compaction on, but it cannot rescue a cluster that has already filled its backend. Compaction stops further growth; it never shrinks an existing backend file. Once etcd has raised a NOSPACE alarm, writes stay rejected until an operator compacts, defragments each member and disarms the alarm — an upgrade does none of that.
The symptom usually surfaces in the client rather than in etcd. A Patroni replica that cannot renew its lease exits cleanly, so it restart-loops with exitCode: 0 and reason: Completed and a climbing restart count — which reads as healthy and gets misdiagnosed as a database fault. Check etcd first. Both inspection commands are read-only and safe to run on a live cluster:
A DB SIZE close to 2.1 GB on every member, plus NOSPACE in the alarm list, confirms it. A healthy cluster prints nothing at all for alarm list. Recovery from there — compacting to a revision, defragmenting each member, then disarming the alarm — is an operator procedure this template deliberately does not perform, because each step is disruptive and the order matters. Follow etcd’s maintenance guide and plan it as a maintenance window.

​
External References

etcd Documentation

Official etcd documentation

Maintenance and Compaction

Compaction, defragmentation, and clearing a NOSPACE alarm

Hardware Recommendations

etcd hardware and resource sizing guidelines

etcd Template

View the source files, default values, and chart definition