Skip to main content

Overview

The OpenTelemetry Collector receives, processes, and exports telemetry over OTLP. This template deploys a stateless collector that feeds Control Plane’s native tracing, and can additionally ingest OTLP metrics and push them to any Prometheus-remote-write-compatible store — such as this catalog’s Prometheus template — with optional authenticated public ingestion (bearer token or mTLS).

Architecture

  • Collector — A standard, stateless workload. replicas copies run behind one endpoint; 2 or more forms an HA ingestion pool that keeps accepting telemetry through rolling restarts and replica loss.
  • Traces — Applications send OTLP to the collector; it normalizes spans, derives span metrics, and forwards traces to the Control Plane tracing backend.
  • Metrics (optional) — An OTLP-to-prometheus_remote_write pipeline pushes ingested metrics to the store you configure.
  • Public ingestion (optional) — Bearer-token auth over the canonical HTTPS endpoint, or mutual TLS over a direct TCP load balancer. Public access always requires auth plus an explicit CIDR allowlist.

What Gets Created

  • Standard Collector Workload — Runs the collector in either a simple (knob-driven) or advanced (full custom config) mode.
  • Secret — An opaque secret holding the rendered collector configuration, mounted as a file.
  • Identity & Policy — An identity bound to the workload with reveal access to the config secret and — only when auth is enabled — the auth secret you name.
  • Direct Load Balancer (optional) — TCP passthrough on ports 4317/4318, created only for public mTLS ingestion.
This template does not create a GVC. You will need to deploy it into an existing GVC and enable tracing at the GVC level after installing, specifying the target workload and port. This will trigger a restart of all workloads in the GVC.

Prerequisites

None for a default install. The two auth modes have prerequisites:
  • Bearer auth (auth.method: bearer) — an opaque secret holding the token, created before install (a missing secret leaves the deployment waiting on it). Set auth.bearer.secretName to its name:
  • mTLS auth (auth.method: mtls) — a dictionary secret created before install with exactly the keys cert (server certificate), key (server private key), and ca (the CA that signed your client certificates). Set auth.mtls.secretName to its name:
Install the template using your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

The default otelCollector.mode changed from advanced to simple in template version 1.1.0. If you customized advanced.config while relying on the old default, set mode: advanced explicitly when upgrading.
The default values.yaml for this template:

Mode and Replicas

  • otelCollector.mode — Set to simple (the default) for a structured configuration generated from the knobs below, or advanced to provide a fully customized configuration. In advanced mode, the entire otelCollector.advanced.config is passed to the collector verbatim.
  • otelCollector.replicas — The collector is stateless, so replicas need no coordination. Set 2 or more for an HA ingestion pool behind the same endpoint; ingestion keeps succeeding through rolling restarts and replica loss.

Simple Mode

  • simple.processors.transform.traceStatements — A list of replace_pattern statements that normalize dynamic URL segments in http.url span attributes. Replace PLACEHOLDER with the path segment you want to collapse, e.g. user/profile.
  • simple.spanmetrics.histogram.buckets — Span-duration histogram bucket boundaries. Tune these to align with your SLO thresholds.
  • simple.spanmetrics.histogram.unit — Output unit of the recorded histogram. Choose ms (milliseconds) or s (seconds).
Histogram buckets must be duration strings ("250ms", "1s"). Bare numbers are parsed as nanoseconds by the collector, silently collapsing every span into the +Inf bucket. If you carried numeric buckets over from a 1.0.x install, convert them (e.g. 250"250ms") when upgrading.

Metrics Ingestion

With metrics.enabled: true, OTLP metrics posted to the collector (internally, or publicly with auth) are pushed to the store at metrics.remoteWrite.endpoint. Any Prometheus-remote-write-compatible URL works — for example, this catalog’s Prometheus template receives remote write at http://RELEASE_NAME-prometheus.GVC_NAME.cpln.local:9095/api/v1/write, and a Thanos query tier on top gives you a global view of what lands there. One collector pushes to one store; run multiple installs for multiple targets.

Ingestion Auth

Auth applies to a dedicated otlp/ingest receiver on 4318 (HTTP) / 4319 (gRPC). The plain gRPC :4317 receiver stays unauthenticated for the GVC tracing integration and is never exposed publicly.
  • bearer — Senders present Authorization: Bearer <token>; requests without the token (or with the wrong one) are rejected with 401. Public bearer ingestion runs over the canonical HTTPS endpoint and is HTTP-only — OTLP/gRPC does not pass the canonical endpoint; use mTLS for public gRPC.
  • mtls — The collector serves your certificate and requires a client certificate signed by your CA; the TLS handshake itself rejects unauthenticated clients. Public mTLS ingestion runs over a direct TCP load balancer on 4318 (HTTP) and 4317 (gRPC); the canonical https:// endpoint intentionally stops accepting traffic in this mode.

Access

Public ingestion always requires auth: enabling publicAccess with auth.method: none or an empty allowedCidrs fails at install — opening to the world requires an explicit ["0.0.0.0/0"]. internalAccess.type controls which workloads inside the org can reach the collector internally (none, same-gvc, or same-org).

Advanced Mode

The advanced.config block accepts a full OpenTelemetry Collector configuration and is fully authoritative. The shipped default config provides: The metrics.* knobs are refused in advanced mode (the install fails rather than silently ignoring them). To ingest metrics, add the pipeline to your config yourself:
The auth.* and publicAccess.* knobs still wire the secret mounts, reveal policy, firewall, and load balancer in advanced mode, but your config must bind the authed receiver to 0.0.0.0:4318 (HTTP) / 0.0.0.0:4319 (gRPC) — certificate and token files are mounted at /etc/otel-collector/tls/{server.crt,server.key,ca.crt} and /etc/otel-collector/auth/token. Keep health_check on 0.0.0.0:13133 or readiness probes are skipped.

Setting Up Tracing

1. Normalize URL patterns (recommended) If your application has dynamic URL segments, use the transform processor to normalize them into fixed patterns. This keeps span cardinality manageable and makes your traces queryable. In either Simple or Advanced config, replace each occurrence of PLACEHOLDER with each path segment you want to collapse:
Add one statement per pattern you want to normalize. 2. Install the OpenTelemetry SDK in your application Your application must emit traces using the OpenTelemetry SDK. Install the SDK and configure it to export via OTLP/gRPC to:
3. Enable tracing at the GVC level Navigate to the GVC where the OpenTelemetry Collector is installed and enable tracing. Set the target workload on port 4317. This triggers a restart of all workloads in the GVC so they begin emitting traces to the collector. 4. Query span metrics in Grafana Span metrics are exposed via Prometheus on port 8889 of the collector workload. Use the spanmetrics connector’s output to build dashboards and alerts. The metric name suffix automatically changes based on the unit defined in your configuration:
If you changed the unit to s in the histogram, the metric name changes to:
Span-metric series first appear roughly 60–90 seconds after the first span — the spanmetrics connector flushes on a 60-second interval, and :8889 serves an empty response until then.

Connecting

The canonical endpoint (and the direct load balancer endpoint in mTLS mode) is in status.canonicalEndpoint of cpln workload get RELEASE_NAME -o yaml. The bearer token is whatever you stored in your prerequisite secret. Use the full .cpln.local hostname for in-GVC senders.

Important Notes

  • The default mode changed from advanced to simple in template version 1.1.0 — if you customized advanced.config while relying on the old default, set mode: advanced explicitly when upgrading.
  • Histogram buckets must be duration strings ("250ms", "1s") — bare numbers are parsed as nanoseconds, which silently breaks the histogram.
  • Enable tracing at the GVC level after install (target the collector workload, port 4317); this restarts all workloads in the GVC.
  • Public ingestion requires auth: publicAccess.enabled with auth.method: none or an empty allowedCidrs fails at install — opening to the world requires an explicit ["0.0.0.0/0"].
  • Create the auth secret before installing — a missing secret leaves the deployment waiting on it.
  • mTLS uses the direct load balancer (raw TCP), not the canonical endpoint; in mTLS mode the canonical https:// endpoint intentionally stops accepting traffic.
  • In mTLS mode, in-GVC senders must use the plain internal gRPC port :4317 — the TLS-terminating ingest ports (4318/4319) are reachable only by external clients via the direct load balancer.
  • One collector pushes to one remote-write store; run multiple installs for multiple targets.

External References

OTel Collector Documentation

Official OpenTelemetry Collector documentation

OTel SDK Setup

Language-specific SDK guides for instrumenting your application

Spanmetrics Connector

Reference for the spanmetrics connector configuration

Prometheus Remote Write Exporter

Reference for the prometheusremotewrite exporter

OTLP Specification

The OpenTelemetry protocol specification

OTel Collector Template

View the source files, default values, and chart definition