Workloads

Overview

Refer to the Workload concepts page.

Create a Workload

Refer to the Create a Workload guide for additional details.

CLI

Refer to the CLI documentation for workloads.

Access Report

Displays the permissions granted to principals for the workload.

Autoscaling

Overview

Workload auto-scaling is configured by setting a strategy, a target value, and in some cases as metric percentile. Together these values determine when the workload will scale up & down.

As the system scales up, traffic will not be sent to the new replicas until they pass the readiness probe, if configured. If there is no probe configured or if it is a basic TCP port check, the requests will hit the new replicas before they are ready to respond. This could cause a delay or errors for end-user traffic.

Scaling Strategies

The scaling strategy is set using autoscaling.metric.

• Concurrent Requests Quantity (concurrency)
  • The average number of requests executing at a given point in time across all the replicas. (requests * requestDuration)/(timePeriod * replicas).
  • Example: A workload with 5 replicas received 1000 requests with an average response time of 50ms (.05 seconds) over a 1 second period. The concurrent requests metric for that period is (1000 * .05)/(1 * 5) = 10.
• Requests Per Second (rps)
  • The raw number of requests received by a workload each second divided by the number of replicas. Requests are counted even if they have not yet been completed.
• Percentage of CPU Utilization (cpu)
  • The percentage of CPU consumed by system and user processes in the container(s) as specified in the container cpu field.
• Request Latency (latency)
  • The request response time (at a configurable percentile) in milliseconds, averaged across all replicas.

Autoscaling Standard Workloads

For standard workloads, Control Plane runs two asynchronous control loops:

1. The Scaling Decision Loop
2. The Metric Calculation Loop

The Scaling Decision Loop

A workload's scale is evaluated every 15 seconds, using the value most recently calculated by the metric calculation loop. Each time an evaluation is made the chosen metric is averaged across all available replicas and compared against the scale target. When scaling up, Control Plane does not enforce a stabilization window; the number of pods will increase as soon as the scaling algorithm dictates. When scaling down, a stabilization window of 5 minutes is used; the highest number of pods recommended by the scaling algorithm within the past 5 minutes will be applied to the running workload.

The Metric Calculation Loop

Requests per Second

Every 20 seconds, Control Plane calculates the average number of requests per second over the past 60 seconds.

Latency

Every 20 seconds, Control Plane calculates the average latency over the past 60 seconds at the specified percentile (p50, p75, p99).

CPU

Every 15 seconds, Control Plane calculates the average CPU usage over the past 15 seconds.

Autoscaling Serverless Workloads

The current capacity is evaluated every 2 seconds and compared against the scale target. It averages requests completed over the previous 60 seconds to avoid rapid changes. If ever a scaling decision is made which results in a scale increase above 200% then it suspends scale down decisions and averages over 6 seconds for 60 seconds. This is to allow for rapid scaling when a burst of traffic is detected.

Special Considerations for the latency Scaling Strategy

Because request latency is represented as a distribution, when using the latency scaling strategy, you must choose a metric percentile by setting the autoscaling.metricPercentile property to one of the following values:

• p50
• p75
• p99

Options

• Minimum Scale (autoscaling.minScale)
  • The minimum allowed number of replicas. A workload can scale down to 0 when there is no traffic and scale-up immediately to fulfill new requests. (Must be between 0 and Maximum Scale inclusive).
• Maximum Scale (autoscaling.maxScale)
  • The maximum allowed number of replicas.
• Scale to Zero Delay (autoscaling.scaleToZeroDelay)
  • The amount of time (in seconds) when there are no requests received before a workload is scaled down to 0. (Must be between 30 and 3600 inclusive).
• Maximum Concurrency (autoscaling.maxConcurrency)
  • The maximum allowed number of requests to be actively running against a single replica. If there are no replicas available that are processing less than the configured maximum number of concurrent requests, the system will queue the request and wait for a replica to be available. It will not trigger a scale decision. The purpose of this setting is to prevent a single replica from taking more traffic than it is designed to process.
  • If, for example, Max Concurrency = 100, Scaling Strategy = ‘Concurrent Requests’, and Target = 100, the results would not be desirable for most end-user traffic. When the system decides to scale up, it will queue the requests until an existing request completes or the new replica becomes available.
  • Must be between 0 and 1000 inclusive.
• Metric Percentile (autoscaling.metricPercentile)
  • The nth percentile is a value below which n percent of the values in a distribution lie.
  • This may only be set while using the latency scaling strategy. The default value is p50.
  • e.g. If the 50th percentile of a latency distribution is 200ms, 50% of requests took less than 200ms.
  • Control plane supports p50, p75, and p99 metric percentiles.

Capacity AI

Workloads can leverage intelligent allocation of its container's resources (CPU and Memory) by using Capacity AI.

Capacity AI uses an analysis of historical usage to adjust these resources up to a configured maximum.

This can significantly reduce cost, but may cause temporary performance issues with sudden spikes in usage.

If capacity AI is disabled, the amount of resources configured will be fully allocated.

Capacity AI must be disabled if the autoscaling strategy is set to CPU Utilization.

Minimum Capacity AI

When resources are not being used, Capacity AI will downscale CPU usage to a minimum of 25 millicores. The minimum will increase depending on the memory size being recommended by Capacity AI using a 1:3 ratio of CPU millicores to memory MiB.

Command Override

The container entrypoint can be overridden by entering a custom command value.

Command Line Arguments

Custom command line arguments can be sent to the container during deployment.

These arguments will be appended to the image ENTRYPOINT.

The argument list is ordered and will be passed to the container in the same order.

Connect

A specific replica of a workload can be connected to (similar to exec) from either the console or the CLI. This can be used for troubleshooting any issues with the replica.

To connect using the console, click the Connect link from a workload. Select the location, container, replica, and command. Click Connect to execute the command. By default, the bash shell will be executed.

To connect using the CLI, review the workload connect subcommand.

Containers

Workloads must have at least one container configured with the following:

• Image
• Port that the container exposes
• Reserved/Max Resources (CPU and Memory)
• Environment Variables
• Command Line Arguments
• Volumes
• Readiness / Liveness Probes

Deactivate

To deactivate / shut-down a workload, perform the following steps:

1. From the selected workload, click Options.
2. Set the Min Scale and Max Scale settings to 0.
3. Click Save

The workload will scale to zero and will not serve any traffic. To reactivate the workload, restore the Min Scale and Max Scale settings to their needed values.

Debug

Debug values can be included within the response headers of a workload's endpoint request.

The values will only be returned when:

1. debug is active and the header x-cpln-debug: true is in the request.
2. The global or canonical endpoint is being requested.

Using the console, debug can be activated by:

• Clicking Options.
• Clicking the Debug switch to on.
• Clicking Save.

After the workload redeploys, the response from the workload's endpoint will contain the following headers if the header x-cpln-debug: true is in the request:

• x-cpln-location: Location of the responding replica.
• x-cpln-replica: Name of the responding replica.

Sample Request Headers:

copy
GET https://doc-test-v39red0.cpln.app/ HTTP/1.1
Host: doc-test-v39red0.cpln.app
Connection: keep-alive
x-cpln-debug: true

Sample Response Headers:

copy
HTTP/1.1 200 OK
content-length: 2993
content-type: text/plain
date: Fri, 10 Sep 2021 21:34:27 GMT
x-envoy-upstream-service-time: 2
x-cpln-location: aws-us-west-2
x-cpln-replica: doc-test-00083-deployment-75584b7d66-f8wtb

Endpoints

Global

This URL is globally load-balanced and TLS terminated. If a domain configured for the GVC, this endpoint will use that domain.

Canonical

This URL is similar to the global endpoint but maps to the built-in Control Plane domain cpln.app. This can be used for testing if there is an issue with the custom domain that is associated with the GVC.

Location Specific

Within each deployment, a location specific URL is available that can be used for testing if there is an issue with the global/canonical URL or with a specific location.

Environment Variables

Custom environment variables can be made available to the image running within a container.

The value of the variable can be in plain text or a secret value.

Built-in Variables

Each workload has the following built-in environment variables:

Secret Variables

Sensitive values can be used as an environment variable by using a secret.

The identity of the workload must be member of a policy that has the reveal permissions on the secret.

Disallowed Variables

The following variable names are not allowed to be used as a custom environment variable:

• K_SERVICE
• K_CONFIGURATION
• K_REVISION

PORT Variable

The PORT environment variable is provided at runtime and available to a container.

It can be assigned as a custom environment variable in all cases except when the container is exposed and the value doesn't match that of the exposed port.

For example:

• If the container is exposed with a port of 3000:
  • the system will accept a PORT environment variable with the value 3000.
  • the system will deny a PORT environment variable with any value other than 3000.
• If the container is not exposed then any value is accepted for the PORT environment variable.

Import Variables

A .env file can be uploaded using the console to import multiple environment variables. Secret values are supported.

Sample .env file:

copy
URL=http://test.example.com
USERNAME=user001
PASSWORD=cpln://secret/username_secret.password
DATA=cpln://secret/opaque_secret.payload

Firewall

External

The external firewall is used to control Internet traffic to/from a workload.

Inbound Requests:

• By default, all inbound requests are disabled.
• Access is granted by explicitly adding one or more IPv4 / IPv6 / CIDR addresses or allowing all addresses.
• Using the UI:
  • Multiple address can entered within the textbox by delimiting each address with either a comma or space.
  • An import file can be uploaded containing each address on its own line or delimited with either a comma or space.

Outbound Requests:

• By default, all outbound requests are disabled.
• Access is granted by explicitly adding one or more IPv4 / IPv6 / CIDR addresses or public hostnames or allowing all addresses / hostnames.
  • When using a hostname, only ports 80, 443, and 445 will be reachable. To allow all ports, enable all outbound requests.
  • When using a IP or CIDR, all ports will be reachable.
• The IP/CIDR addresses takes precedence over hostnames.
• Using the UI:
  • Multiple address can entered within the textbox by delimiting each address with either a comma or space.
  • An import file can be uploaded containing each address on its own line or delimited with either a comma or space.

Internal

The internal firewall is used to control access between other workloads within an org.

Available Options:

• None: No access is allowed between workloads.
• Same GVC: Workloads running in the same GVC are accessible.
• Same Org: Workloads running in the same org are accessible.
• Specific Workloads: Specific workloads are allowed access this workload.
  • These workloads can be from the same or different GVCs.
  • The user configuring this setting must have the view permission, set within a policy, on the workload being specified.
• Allow to Access Itself: Enables replicas of this workload to access themselves.

Identity

Refer to the identities page for additional details.

Images

Each workload must be configured with at least one container, associated with an image.

Images can be pulled from:

• A public registry

  • If the image does not require authentication, only the image name and optional tag are required.
  • If authentication is required, a pull secret must be configured on the GVC containing the workload.

• Org's private registry

  • If the image resides in your org's private registry, no pull secret is required and you may use one of the following for the image name:
    • Full Name: /org/ORG_NAME/image/IMAGE_NAME:TAG
    • Short Name: //image/IMAGE_NAME:TAG

Lifecycle

Each Workload container can be configured to execute the PostStart lifecycle hook.

This hook is executed immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT.

This hook can be configured using the console or cpln apply.

• Using the console

  • From the workload container, select the `Lifecycle' link from the top menu bar.
  • Enter the command and optional arguments.
  • Click Save.

• Using cpln apply

  • Only the exec type is supported.

  • Example:

    Add the lifecycle section to an existing workload container.

    copy
    spec:
      containers:
        - name: advanced-options-example
          args: []
          cpu: 50m
          env: []
          image: '/org/ORG_NAME/image/IMAGE:TAG'
          memory: 128Mi
          port: 8080
          lifecycle:
            postStart:
              exec:
                command:
                  - sh
                  - '-c'
                  - sleep 10

Logs

Workload logs are consolidated from all the deployed locations and can be viewed using the UI or CLI.

Using the UI, the logs page will be prefilled with the LogQL query for the workload and GVC name.

Example LogQL Query
copy
  {gvc="test-gvc, workload="test-workload"}

Logs can be further filtered by:

• Date
• Location
• Container

Grafana can be used to view the logs by clicking the Explore on Grafana link within the console.

Refer to the logs page for additional details.

Map a Secret as a File

A Secret can be mapped as a read-only file by using a Volume.

During the configuration of a Volume using the console, the Secret reference (e.g., cpln://secret/SECRET_NAME) can be entered manually or Control-S can be pressed to view and select the available Secrets.

The Path must be a unique absolute path and, optionally, a file name (e.g., /secret/my-secret.txt) depending on the secret type. This path will be added to the container's file system and will be accessible by the running application.

NOTE: A maximum of 15 volumes can be added.

Secret Types

The secret type will dictate how the secret will be mounted to the file system.

• Opaque Secret:
  • The .payload property is not required.
  • If the payload is base-64 encoded, the secret can be decoded at runtime by selecting the Base64 decode at Runtime checkbox when configuring the secret.
  • The configured path must contain at least one subpath (e.g., /path/subpath). The last path (or file name) will be mounted as a file and contain the payload. If a subpath is not given, the payload of the secret will be mounted as a file named payload (e.g., /path/payload).
• Azure, Docker, and GCP Secrets:
  • The secret will be mounted to the specified path as the file name ___cpln___.secret.
  • The configured path must contain at least one subpath (e.g., /path/subpath). The last path will be mounted as a directory and contain the ___cpln___.secret file.

• All other Secret Types:

  • If the root secret is selected, the specified path will be mounted as a directory. The contents of the directory will contain files named as the key/property of the secret. The contents of each file will contain the value of the respective key.
  • If a key/property of a secret is selected, the secret will be mounted to the specified path as a file. The contents will include the value of the key/property.
  • The directory will include a file named ___cpln___.secret. The contents of this file will be the JSON formatted output of the secret.

Identity Configuration

A Workload that is configured with a Volume that references a Secret must be configured with an Identity bound to a policy having the reveal permission.

Metrics

Control Plane can collect custom metrics from your workload by having your application emit a Prometheus formatted list of metrics at a path and port of your choosing. The port can be different than the one serving traffic. Each container in a workload can be configured with metrics.

Sample output from the metrics endpoint:

copy
MY_COUNTER 788
MY_COUNTER_2 123
NUM_USERS 2
NUM_ORDERS 91

The platform will scrape all the replicas in the workload every 30 seconds and they must respond within 5 seconds. Metric names with the prefix cpln_ will be ignored by the scrapping process.

The collected metrics can be viewed by clicking the Metrics link on the workload page within the console. Clear any existing query and enter the name of the metric. Click Run Query to execute.

The time-series displayed will include these labels:

• org
• gvc
• location
• provider
• region
• cluster_id
• replica

Pause a Workload

To pause / suspend a workload, set the min and max scale to 0 within the Autoscaling configuration of the Options page.

Permissions

The permissions below are used to define policies together with one or more of the four principal types:

Probes

Probes are a feature of Kubernetes that are used to control the health of an application running inside a container.

Readiness Probe

The readiness probe is an action that is defined to let the workload know that the application running within the container is ready to receive traffic. For example, if the application is performing some actions during start-up and needs it to complete before serving requests, the readiness probe can fail until the actions have been completed.

Liveness Probe

The liveness probe is an action that is defined to check if the application running within the container is healthy. If the probe fails, the container will be restarted. For example, if the application code caused itself to be in a deadlock, the liveness probe can catch that the container is not healthy, since it didn't respond to the probe, and restart. This will ensure that the application is available until the bug that is causing the deadlock is fixed.

Options

Health Check Type:

• Run a Custom Command.
• HTTP
  • Scheme (either HTTP or HTTPS, default is HTTP).
  • Path.
  • Port (must be between 80 and 65535 inclusive).
  • Optional HTTP headers.
• TCP
  • Socket port (must be between 80 and 65535 inclusive. Ports 8012, 8022, 9090, 9091, 15000, 15001, 15006, 15020, 15021, 15090, 41000 are invalid).

Configurable Limits:

• Initial Delay Seconds
  • The delay to wait after the container is started before performing the first probe (must be between 0 and 120 inclusive, default is 0).
• Period Seconds
  • How often to perform the probe (must be between 1 and 60 inclusive, default is 10).
• Timeout Seconds
  • Number of seconds after which the probe times out (must be between 1 and 60 inclusive, default is 1).
• Success Threshold
  • Minimum consecutive successes for the probe to be considered successful after having failed (must be between 1 and 20 inclusive, default is 1).
• Failure Threshold
  • When a probe fails, Kubernetes will try this amount of times before giving up. For a liveness probe, the container will be restarted. For a readiness probe, the workload will be marked Unready. (must be between 1 and 20 inclusive, default is 3).

Refer to the Kubernetes probe documentation here for additional details.

Resources

The CPU and Memory of a container is configurable. Select appropriate values for each container.

If capacity AI is enabled, these values will be the maximum that the container could be provisioned with.

If capacity AI is disabled, these values will be reserved when the container is provisioned.

Memory Units:

You can express memory as a plain integer or as a fixed-point number using one of these suffixes: E, P, T, G, M, k.

You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki.

For example, the following represent roughly the same value:

128974848, 129e6, 129M, 123Mi

Refer to the Kubernetes Meaning of Memory reference page for additional details.

Storage

Each workload replica receives 1GB of local ephemeral solid state drive (SSD) storage.

If the replica uses more than 1GB, it will be replaced with a new, fresh replica.

Timeout

The maximum request duration in seconds before Control Plane will timeout. This timeout amount can be reached when Control Plane is waiting for the workload to respond or when waiting for a new workload to become available when using Autoscaling.

The minimum value is 1 second and the maximum value is 600 seconds.

Types

Serverless

Serverless workloads should be used for web applications that serve traffic on a single port, but may not need to run 100% of the time.

Serverless workloads may:

• Scale to zero.

Serverless workloads may not:

• Serve traffic on multiple ports.

Serverless workloads must:

• Expose a network endpoint.

Standard

Standard workloads have greater flexibility in network exposure, but may not scale to zero.

Standard workloads may:

• Expose no network endpoint.
• Serve traffic on multiple ports.

Standard workloads may not:

• Scale to zero.

Cron

Cron workloads should be used when you need to perform a background task on a regular schedule.

Cron workloads may not:

• Serve traffic.
• Scale to zero.
• Include a container that runs indefinitely.

Cron workloads must:

• Exit upon completion of the task at hand. Control Plane will start a new replica of your workload at the next scheduled execution time.

Configuration

• job.schedule
  • A string, which determines how often the cron workload should execute. This field uses the Kubernetes cron schedule syntax
• job.concurrencyPolicy
  • Either Forbid or Replace. This determines what Control Plane will do when a prior execution of your workload is still running when the next scheduled execution time arrives.
    • Forbid: subsequent executions will be forgone until the running execution completes.
    • Replace: the running execution will be stopped so that a new execution can begin.
• job.historyLimit
  • An integer between 1 and 10 representing the number of prior executions to be retained for reference.
• job.suspend
  • A boolean value indicating whether your workload should run. Setting this property to true will disable future executions.
• job.restartPolicy
  • Either Never or OnFailure. This determines whether your workload will be restarted when it fails on execution.

Volumes

Cloud Object and File storage, ephemeral scratch storage and Secrets can be mounted to directories of containers at runtime by adding one or more volumes.

A volume consists of a uri and a mount path. The uri is prefixed with the provider scheme followed by the bucket/storage name (e.g., s3://my-s3-bucket). The mount path must be a unique absolute path (e.g., /s3-files). This path will be added to the container's file system and accessible by the running application.

During the set up of a volume using the console, the uri name can be entered manually or an existing Cloud Account can assist looking up the name.

The identity of the workload is used to authenticate to the provider's cloud storage API, or used for authorization to access the Control Plane secret. A Cloud Account for each cloud storage provider, with the necessary access/roles, must exist and be associated with the workload identity.

Volumes can be shared between containers of the same workload. For example if two containers in a workload are each configured with the volume uri: 'scratch://volume1', path: '/my/shared/data' then changes to files in /my/shared/data will be visible to both containers.

NOTE: A maximum of 15 volumes can be added.

Volume Providers

Identity Configuration

To allow a workload identity the ability to authenticate to an object store, a cloud access rule must be created for each provider. A Cloud Account for each provider must exists in order to create the cloud access rule.

The following list contains the minimum roles/scopes that must be added to a cloud access rule:

• S3 (using an AWS Cloud Account)

  • Select Create a new AWS role with existing policies and choose AmazonS3ReadOnlyAccess.

• Google Cloud Storage (using a Google Cloud Account)

  • Select Create a new GCP service account.
  • Resource: Storage -> Global -> Bucket -> Select bucket name.
  • Role: Storage Legacy Bucket Reader and Storage Legacy Object Reader.
  • Verify that the Cloud Account for GCP is configured correctly. In particular, the Control Plane GCP service account requires the Storage Admin role.

• Azure Blob Storage and Files (using an Azure Cloud Account)

  • Scope: Storage -> Region -> Storage Accounts -> Select storage account.
  • Role (for Azure Files) : Reader and Data Access.
  • Role (for Azure Blobs) : Storage Blob Data Reader.

Firewall Configuration

To allow a Workload access to the object stores, the outbound requests of its external firewall must either be set to All Outbound Requests Allowed or the hostnames listed below for the corresponding object store must be added to the Outbound Hostname Allow List.

AWS
copy
  *.amazonaws.com

Azure Blob
copy
  *.blob.core.windows.net
  *.azure.com

Azure File
copy
  *.file.core.windows.net
  *.azure.com

GCP
copy
  *.googleapis.com

Limitations

• Volumes are read-only, except for Azure Files.
• The following Path names are reserved:
  • /dev
  • /dev/log
  • /tmp
  • /var
  • /var/log
• Authentication to a provider is only facilitated through the workload identity. The use of an AWS or Azure key to mount a bucket/container within a container will not work.
• Properties of a mounted object store, such as cache policies and timeouts, cannot be configured by the user. Control Plane has optimized those values for each cloud provider.

Metering and Billing

The CPU, memory and egress used for mounted object stores are billed to the workload. To review the costs of mounting an object store, query the container named cpln-mounter for the workload within the metrics page.

Working Directory Override

The container working directory can be overridden by entering a custom directory. The value must be an absolute path.

Workload Health

• Possible values:
  • Loading
  • Healthy
  • Unhealthy
  • Deleting
  • Unknown