Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt

Use this file to discover all available pages before exploring further.

Overview

This guide demonstrates how to export metrics from multiple organizations using the Prometheus /federate endpoint. This setup allows you to centralize metrics collection by federating Control Plane metrics into a single external Prometheus instance or service that can collect data using a Prometheus scrape job.

Prerequisites

  • Superuser access to the Control Plane Console for each organization.
  • A Prometheus instance configured to scrape metrics or a third-party service that can collect metrics using a Prometheus scrape job.

Steps

In the Source Organization

1

Create a Service Account

2

Generate a Key

  • Click Keys and create a new key for the prometheus-federate service account and store it securely.
3

Create a Policy

Grant the service account created above the readMetrics permission by creating a policy using one of the following methods:
  1. Using the Console, create a policy named prometheus-federate that grants the readMetrics permission to the service account.
    • Click Policies in the left menu.
    • Click New and fill in the details:
      • Name: prometheus-federate
      • Description: prometheus-federate
    • Click Target.
      • Target Kind: org
    • Click Binding.
    • Click Add Binding:
      • On the Permissions tab, check readMetrics
      • On the Service Accounts tab, check prometheus-federate
    • Click Ok.
    • Click Create.
  2. Using YAML
    • Apply the YAML manifest below using the cpln apply command. Substitute ORG_NAME with your org name.
    YAML
    kind: policy
    name: prometheus-federate
    description: prometheus-federate
    tags: {}
    bindings:
       - permissions:
          - readMetrics
          principalLinks:
          - /org/ORG_NAME/serviceaccount/prometheus-federate
    target: all
    targetKind: org
    

In the Prometheus Configuration

  1. Configure Scrape Job:
    • Add the following scrape configuration to your Prometheus instance:
    scrape_configs:
      - job_name: 'federate'
        scrape_interval: 1m
        honor_labels: true
        scheme: https
        metrics_path: '/metrics/org/${org}/api/v1/federate'
        params:
          'match[]':
            - '{__name__=~".+"}'  # Adjust this matcher as needed
        authorization:
          type: Bearer
          credentials: "${CPLN_SERVICE_ACCOUNT_TOKEN}"  # Token value from env variable
        static_configs:
          - targets:
              - 'metrics.cpln.io'
    
    • Replace ${org} with the actual organization name.
    • Ensure ${CPLN_SERVICE_ACCOUNT_TOKEN} is set as an environment variable containing the token from the prometheus-federate service account.

Verification

  • After configuring the scrape job, verify that Prometheus is successfully collecting metrics from the source organization by checking the Prometheus UI.
  • You should be able to see the same metrics in the Prometheus UI as you would in the Control Plane managed Grafana instance.

Additional Notes

  • Adjust the match[] parameter in the scrape configuration to filter the metrics as needed.
  • Repeat the steps for additional organizations by creating separate service accounts and policies.
  • Egress charges apply to metrics scraped from Control Plane.

Cleanup

  • Delete the scrape job in the Prometheus instance.
  • Delete the policy and service account in the source organization.