CPLN Task Runner

Overview

Control Plane Task Runner is a self-hosted task queue and scheduler service similar to Google Cloud Tasks. It provides HTTP-based task enqueuing with automatic retry, delayed and scheduled execution, per-client rate limiting, and multi-queue support with priority levels.

What Gets Created

API Workload — HTTP endpoint for enqueuing tasks, managing clients, and health checks. Scales between 1 and 3 replicas by default.
Worker Workload — Background processor that picks tasks off the queue and executes them. Scales between 1 and 5 replicas by default.
Redis with Sentinel — A Redis instance with Redis Sentinel for high-availability task persistence and coordination. Sentinel monitors the Redis primary and handles automatic failover.
Secret — An opaque secret containing Redis and Sentinel passwords and the admin API key (when createSecret is true).
Identity & Policy — An identity bound to the workloads with reveal access to the task runner secrets.

Architecture

The API workload receives task submissions over HTTP and pushes them into Redis. The Worker workload polls Redis and executes tasks by making outbound HTTP requests to the configured target URLs. Both workloads connect to Redis via Sentinel for failover resilience.

This template does not create a GVC. You must deploy it into an existing GVC.

Installation

This template has no external prerequisites. To install, follow the instructions for your preferred method:

UI

Browse, install, and manage templates visually

CLI

Manage templates from your terminal

Terraform

Declare templates in your Terraform configurations

Pulumi

Declare templates in your Pulumi programs

Configuration

The default values.yaml for this template:

# Same image for both API and Worker workloads
image: controlplanecorporation/cpln-task-runner:0.4

# API configuration
api:
  enabled: true
  replicas:
    min: 1
    max: 3
  port: 8080
  public:
    enabled: true
    pathPrefix: "" # Path prefix for public endpoint (empty for root)
  resources:
    cpu: 500m
    memory: 512Mi
  env:
    logLevel: info # debug, info, warn, error
    adminApiKey: "" # Admin API key for protected endpoints (leave empty to disable)
    otelEndpoint: "" # OpenTelemetry endpoint (leave empty to disable tracing)
    connectRetries: 30
    retryIntervalSec: 2

# Worker configuration
worker:
  enabled: true
  replicas:
    min: 1
    max: 5
  port: 8082 # Used for health checks
  resources:
    cpu: 1
    memory: 1Gi
  env:
    logLevel: info
    concurrency: 10 # Concurrent workers per replica
    taskTimeoutSec: 1800 # Task timeout (default: 30 minutes)
    maxRetry: 5 # Maximum retry attempts
    allowPrivateUrls: false # Allow tasks to target private/internal URLs
    cbFailureThreshold: 5 # Circuit breaker failure threshold
    cbTimeoutSec: 30 # Circuit breaker timeout in seconds
    connectRetries: 30
    retryIntervalSec: 2
    otelEndpoint: ""

# Creates a secret for Redis + Sentinel passwords and admin API key
createSecret: true
secretName: task-runner-secrets # Must match names below if createSecret is true

# Redis configuration
redis:
  redisPassword: mypassword # Change before deploying to production
  sentinelPassword: mypassword # Change before deploying to production
  admin:
    fromSecret:
      name: task-runner-secrets
      apiKeyKey: admin-api-key
  redis:
    auth:
      fromSecret:
        enabled: true
        name: task-runner-secrets
        passwordKey: redis-password
    persistence:
      enabled: true
  sentinel:
    auth:
      fromSecret:
        enabled: true
        name: task-runner-secrets
        passwordKey: redis-sentinel-password
    persistence:
      enabled: true

API Workload

api.enabled — Enable or disable the API workload.
api.replicas — Min/max replica count for autoscaling (default: 1–3).
api.port — Container port (default 8080).
api.public.enabled — Expose the API to the public internet.
api.public.pathPrefix — Optional path prefix for the public endpoint. Leave empty to serve from the root.
api.resources — CPU and memory allocated to each API replica.
api.env.logLevel — Log verbosity: debug, info, warn, or error.
api.env.adminApiKey — API key for admin-protected endpoints. Leave empty to disable admin authentication.
api.env.connectRetries / api.env.retryIntervalSec — Redis connection retry behavior on startup.

Worker Workload

worker.enabled — Enable or disable the Worker workload.
worker.replicas — Min/max replica count for autoscaling (default: 1–5).
worker.resources — CPU and memory allocated to each Worker replica.
worker.env.concurrency — Number of tasks a single Worker replica can execute concurrently.
worker.env.taskTimeoutSec — Maximum duration in seconds before a task execution is considered failed (default 1800).
worker.env.maxRetry — Maximum number of retry attempts for a failed task.
worker.env.allowPrivateUrls — When true, allows tasks to target internal/private URLs.
worker.env.cbFailureThreshold / worker.env.cbTimeoutSec — Circuit breaker settings to stop hammering failing endpoints.

Secrets and Redis

When createSecret is true, the template automatically creates a secret named by secretName containing the Redis password, Sentinel password, and admin API key. The redis.*.fromSecret fields are pre-wired to reference this secret. When using an existing secret (createSecret: false), update the redis.*.fromSecret fields to point to your secret name and the correct keys.

Change the default redis.redisPassword and redis.sentinelPassword values before deploying to production.

Enqueuing Tasks

Once deployed, submit tasks to the API workload’s public endpoint:

curl -X POST https://your-api-endpoint/v1/enqueue \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "my-service",
    "queue": "default",
    "task": {
      "url": "https://api.example.com/webhook",
      "method": "POST",
      "headers": {"Content-Type": "application/json"},
      "body": "{\"event\": \"user.created\"}"
    }
  }'

Admin Endpoints

When api.env.adminApiKey is set, admin endpoints require the X-Admin-Key header:

# List clients
curl https://your-api-endpoint/admin/clients \
  -H "X-Admin-Key: your-admin-key"

# Create or update a client
curl -X POST https://your-api-endpoint/admin/clients/set \
  -H "X-Admin-Key: your-admin-key" \
  -H "Content-Type: application/json" \
  -d '{
    "client_id": "new-service",
    "tier": "premium",
    "enabled": true
  }'

Rate Limiting

Rate limits are applied per client and configured via the admin API. Available tiers:

Tier	Requests/min	Max Concurrent
`free`	10	1
`basic`	100	5
`premium`	1,000	20
`enterprise`	5,000	50

OpenTelemetry

To enable distributed tracing, set otelEndpoint in both api.env and worker.env. In your GVC configuration, ensure the Tracing Provider is set to Control Plane, then use the default HTTP collector endpoint:

tracing.controlplane:4318

Template Catalog

Templates

Overview

What Gets Created

Architecture

Installation

UI

CLI

Terraform

Pulumi

Configuration

API Workload

Worker Workload

Secrets and Redis

Enqueuing Tasks

Admin Endpoints

Rate Limiting

OpenTelemetry

External References

OpenTelemetry Documentation

CPLN Task Runner Template

​Overview

​What Gets Created

​Architecture

​Installation

UI

CLI

Terraform

Pulumi

​Configuration

​API Workload

​Worker Workload

​Secrets and Redis

​Enqueuing Tasks

​Admin Endpoints

​Rate Limiting

​OpenTelemetry

​External References

OpenTelemetry Documentation

CPLN Task Runner Template

Overview

What Gets Created

Architecture

Installation

Configuration

API Workload

Worker Workload

Secrets and Redis

Enqueuing Tasks

Admin Endpoints

Rate Limiting

OpenTelemetry

External References