Workloads

Overview

Refer to the Workload concepts page.

Create a Workload

Refer to the Create a Workload guide for additional details.

CLI

Refer to the CLI documentation for workloads.

Access Report

Displays the permissions granted to principals for the workload.

Autoscaling

Workload auto-scaling is configured by setting a strategy and target value which controls when the workload will scale up & down.

The current capacity is evaluated every 2 seconds and compared against the scale target. It averages requests completed over the previous 60 seconds to avoid rapid changes. If ever a scaling decision is made which results in a scale increase above 200% then it suspends scale down decisions and averages over 6 seconds for 60 seconds. This is to allow for rapid scaling when a burst of traffic is detected.

As the system scales up, traffic will not be sent to the new replicas until they pass the readiness probe, if configured. If there is no probe configured or if it is a basic TCP port check, the requests will hit the new replicas before they are ready to respond. This could cause a delay or errors for end-user traffic.

Selectable Scaling Strategies:

• Concurrent Requests Quantity
  • The average number of requests executing at a given point in time across all the replicas. (requests * requestDuration)/(timePeriod * replicas).
  • Example: A workload with 5 replicas received 1000 requests with an average response time of 50ms (.05 seconds) over a 1 second period. The concurrent requests metric for that period is (1000 * .05)/(1 * 5) = 10.
• Requests Per Second
  • The raw number of requests received by a workload each second divided by the number of replicas. Requests are counted even if they have not yet been completed.
• Percentage of CPU Utilization
  • The percentage of CPU consumed by system and user processes in the container(s) as specified in the container cpu field.

Options:

• Minimum Scale
  • The minimum allowed number of replicas. A workload can scale down to 0 when there is no traffic and scale-up immediately to fulfill new requests. (Must be between 0 and Maximum Scale inclusive).
• Maximum Scale
  • The maximum allowed number of replicas.
• Scale to Zero Delay
  • The amount of time (in seconds) when there are no requests received before a workload is scaled down to 0. (Must be between 30 and 3600 inclusive).
• Maximum Concurrency
  • The maximum allowed number of requests to be actively running against a single replica. If there are no replicas available that are processing less than the configured maximum number of concurrent requests, the system will queue the request and wait for a replica to be available. It will not trigger a scale decision. The purpose of this setting is to prevent a single replica from taking more traffic than it is designed to process.
  • If, for example, Max Concurrency = 100, Scaling Strategy = ‘Concurrent Requests’, and Target = 100, the results would not be desirable for most end-user traffic. When the system decides to scale up, it will queue the requests until an existing request completes or the new replica becomes available.
  • Must be between 0 and 1000 inclusive.

Capacity AI

Workloads can leverage intelligent allocation of its container's resources (CPU and Memory) by using Capacity AI.

Capacity AI uses an analysis of historical usage to adjust these resources up to a configured maximum.

This can significantly reduce cost, but may cause temporary performance issues with sudden spikes in usage.

If capacity AI is disabled, the amount of resources configured will be fully allocated.

Capacity AI must be disabled if the autoscaling strategy is set to CPU Utilization.

Minimum Capacity AI

When resources are not being used, Capacity AI will downscale CPU usage to a minimum of 25 millicores. The minimum will increase depending on the memory size being recommended by Capacity AI using a 1:3 ratio of CPU millicores to memory MiB.

Command Override

The container entrypoint can be overridden by entering a custom command value.

Command Line Arguments

Custom command line arguments can be sent to the container during deployment.

These arguments will be appended to the image ENTRYPOINT.

The argument list is ordered and will be passed to the container in the same order.

Connect

A specific replica of a workload can be connected to (similar to exec) from either the console or the CLI. This can be used for troubleshooting any issues with the replica.

To connect using the console, click the Connect link from a workload. Select the location, container, replica, and command. Click Connect to execute the command. By default, the bash shell will be executed.

To connect using the CLI, review the workload connect subcommand.

Containers

Workloads must have at least one container configured with the following:

• Image
• Port that the container exposes
• Reserved/Max Resources (CPU and Memory)
• Environment Variables
• Command Line Arguments
• Volumes
• Readiness / Liveness Probes

Deactivate

To deactivate / shut-down a workload, perform the following steps:

1. From the selected workload, click Options.
2. Set the Min Scale and Max Scale settings to 0.
3. Click Save

The workload will scale to zero and will not serve any traffic. To reactivate the workload, restore the Min Scale and Max Scale settings to their needed values.

Debug

Debug values can be included within the response headers of a workload's endpoint request.

The values will only be returned when:

1. debug is active and the header x-cpln-debug: true is in the request.
2. The global or canonical endpoint is being requested.

Using the console, debug can be activated by:

• Clicking Options.
• Clicking the Debug switch to on.
• Clicking Save.

After the workload redeploys, the response from the workload's endpoint will contain the following headers if the header x-cpln-debug: true is in the request:

• x-cpln-location: Location of the responding replica.
• x-cpln-replica: Name of the responding replica.

Sample Request Headers:

copy
GET https://doc-test-v39red0.cpln.app/ HTTP/1.1
Host: doc-test-v39red0.cpln.app
Connection: keep-alive
x-cpln-debug: true

Sample Response Headers:

copy
HTTP/1.1 200 OK
content-length: 2993
content-type: text/plain
date: Fri, 10 Sep 2021 21:34:27 GMT
x-envoy-upstream-service-time: 2
x-cpln-location: aws-us-west-2
x-cpln-replica: doc-test-00083-deployment-75584b7d66-f8wtb

Endpoints

Global

This URL is globally load-balanced and TLS terminated. If a domain configured for the GVC, this endpoint will use that domain.

Canonical

This URL is similar to the global endpoint but maps to the built-in Control Plane domain cpln.app. This can be used for testing if there is an issue with the custom domain that is associated with the GVC.

Location Specific

Within each deployment, a location specific URL is available that can be used for testing if there is an issue with the global/canonical URL or with a specific location.

Environment Variables

Custom environment variables can be made available to the image running within a container.

The value of the variable can be in plain text or a secret value.

Built-in Variables

Each workload has the following built-in environment variables:

Secret Variables

Sensitive values can be used as an environment variable by using a secret.

The identity of the workload must be member of a profile that has the reveal permissions on the secret.

Disallowed Variables

The following variable names are not allowed to be used as a custom environment variable:

• K_SERVICE
• K_CONFIGURATION
• K_REVISION

PORT Variable

The PORT environment variable is provided at runtime and available to a container.

It can be assigned as a custom environment variable in all cases except when the container is exposed and the value doesn't match that of the exposed port.

For example:

• If the container is exposed with a port of 3000:
  • the system will accept a PORT environment variable with the value 3000.
  • the system will deny a PORT environment variable with any value other than 3000.
• If the container is not exposed then any value is accepted for the PORT environment variable.

Import Variables

A .env file can be uploaded using the console to import multiple environment variables. Secret values are supported.

Sample .env file:

copy
URL=http://test.example.com
USERNAME=user001
PASSWORD=cpln://secret/username_secret.password
DATA=cpln://secret/opaque_secret.payload

Firewall

External

The external firewall is used to control Internet traffic to/from a workload.

Inbound Requests:

• By default, all inbound requests are disabled.
• Access is granted by explicitly adding one or more IPv4 / IPv6 / CIDR addresses.

Outbound Requests:

• By default, all outbound requests are disabled.
• Access is granted by explicitly adding one or more IPv4 / IPv6 / CIDR addresses or public hostnames.
  • When using a hostname, only ports 80, 443, and 445 will be reachable. To allow all ports, enable all outbound requests.
  • When using a IP or CIDR, all ports will be reachable.
• The IP/CIDR addresses takes precedence over hostnames.

Internal

The internal firewall is used to control access between other workloads within an org.

Available Options:

• None: No access is allowed between workloads.
• Same GVC: Workloads running in the same GVC are accessible.
• Same Org: Workloads running in the same org are accessible.
• Specific Workloads: Specific workloads are allowed access this workload.
  • These workloads can be from the same or different GVCs.
  • The user configuring this setting must have the view permission, set within a policy, on the workload being specified.
• Allow to Access Itself: Enables replicas of this workload to access themselves.

Identity

Refer to the identities page for additional details.

Images

Each workload must be configured with at least one container, associated with an image.

Images can be pulled from:

• A public registry

  • If the image does not require authentication, only the image name and optional tag are required.
  • If authentication is required, a pull secret must be configured on the GVC containing the workload.

• Org's private registry

  • If the image resides in your org's private registry, no pull secret is required and you may use one of the following for the image name:
    • Full Name: /org/ORG_NAME/image/IMAGE_NAME:TAG
    • Short Name: //image/IMAGE_NAME:TAG

Lifecycle

Each Workload container can be configured to execute the PostStart lifecycle hook.

This hook is executed immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT.

This hook can be configured using the console or cpln apply.

• Using the console

  • From the workload container, select the `Lifecycle' link from the top menu bar.
  • Enter the command and optional arguments.
  • Click Save.

• Using cpln apply

  • Only the exec type is supported.

  • Example:

    Add the lifecycle section to an existing workload container.

    copy
    spec:
      containers:
        - name: advanced-options-example
          args: []
          cpu: 50m
          env: []
          image: '/org/ORG_NAME/image/IMAGE:TAG'
          memory: 128Mi
          port: 8080
          lifecycle:
            postStart:
              exec:
                command:
                  - sh
                  - '-c'
                  - sleep 10

Logs

Workload logs are consolidated from all the deployed locations and can be viewed using the UI or CLI.

Using the UI, the logs page will be prefilled with the LogQL query for the workload and GVC name.

Example LogQL Query
copy
  {gvc="test-gvc, workload="test-workload"}

Logs can be further filtered by:

• Date
• Location
• Container

Grafana can be used to view the logs by clicking the Explore on Grafana link within the console.

Refer to the logs page for additional details.

Metrics

Control Plane can collect custom metrics from your workload by having your application emit a Prometheus formatted list of metrics at a path and port of your choosing. The port can be different than the one serving traffic. Each container in a workload can be configured with metrics.

Sample output from the metrics endpoint:

copy
MY_COUNTER 788
MY_COUNTER_2 123
NUM_USERS 2
NUM_ORDERS 91

The platform will scrape all the replicas in the workload every 30 seconds and they must respond within 5 seconds. Metric names with the prefix cpln_ will be ignored by the scrapping process.

The collected metrics can be viewed by clicking the Metrics link on the workload page within the console. Clear any existing query and enter the name of the metric. Click Run Query to execute.

The time-series displayed will include these labels:

• org
• gvc
• location
• provider
• region
• cluster_id
• replica

Pause a Workload

To pause / suspend a workload, set the min and max scale to 0 within the Autoscaling configuration of the Options page.

Permissions

The permissions below are used to define policies together with one or more of the four principal types:

Probes

Probes are a feature of Kubernetes that are used to control the health of an application running inside a container.

Readiness Probe

The readiness probe is an action that is defined to let the workload know that the application running within the container is ready to receive traffic. For example, if the application is performing some actions during start-up and needs it to complete before serving requests, the readiness probe can fail until the actions have been completed.

Liveness Probe

The liveness probe is an action that is defined to check if the application running within the container is healthy. If the probe fails, the container will be restarted. For example, if the application code caused itself to be in a deadlock, the liveness probe can catch that the container is not healthy, since it didn't respond to the probe, and restart. This will ensure that the application is available until the bug that is causing the deadlock is fixed.

Options

Health Check Type:

• Run a Custom Command.
• HTTP
  • Scheme (either HTTP or HTTPS, default is HTTP).
  • Path.
  • Port (must be between 80 and 65535 inclusive).
  • Optional HTTP headers.
• TCP
  • Socket port (must be between 80 and 65535 inclusive. Ports 8012, 8022, 9090, 9091, 15000, 15001, 15006, 15020, 15021, 15090, 41000 are invalid).

Configurable Limits:

• Initial Delay Seconds
  • The delay to wait after the container is started before performing the first probe (must be between 0 and 120 inclusive, default is 0).
• Period Seconds
  • How often to perform the probe (must be between 1 and 60 inclusive, default is 10).
• Timeout Seconds
  • Number of seconds after which the probe times out (must be between 1 and 60 inclusive, default is 1).
• Success Threshold
  • Minimum consecutive successes for the probe to be considered successful after having failed (must be between 1 and 20 inclusive, default is 1).
• Failure Threshold
  • When a probe fails, Kubernetes will try this amount of times before giving up. For a liveness probe, the container will be restarted. For a readiness probe, the workload will be marked Unready. (must be between 1 and 20 inclusive, default is 3).

Refer to the Kubernetes probe documentation here for additional details.

Resources

Each container can have its amount of CPU and Memory configured. Select an appropriate value for your container.

If capacity AI is enabled, these values will be the maximum that the container could be provisioned with.

If capacity AI is disabled, these values will be reserved when the container is provisioned.

Memory Units:

You can express memory as a plain integer or as a fixed-point number using one of these suffixes: E, P, T, G, M, k.

You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, Mi, Ki.

For example, the following represent roughly the same value:

128974848, 129e6, 129M, 123Mi

Refer to the Kubernetes Meaning of Memory reference page for additional details.

Storage

Each workload replica receives 1GB of local ephemeral solid state drive (SSD) storage.

If the replica uses more than 1GB, it will be replaced with a new, fresh replica.

Timeout

The maximum request duration in seconds before Control Plane will timeout. This timeout amount can be reached when Control Plane is waiting for the workload to respond or when waiting for a new workload to become available when using Autoscaling.

The minimum value is 1 second and the maximum value is 600 seconds.

Volumes

Object stores, such as S3, can be mounted as a file system on a container by adding one or more volumes.

A volume consists of a URI and a Mount Path. The URI is prefixed with the provider scheme followed by the bucket/storage name (e.g., s3://my-s3-bucket). The Mount Path must be a unique absolute path (e.g., /s3-files). This path will be added to the container's file system and accessible by the running application.

During the set up of a volume using the console, the URI name can be entered manually or an existing Cloud Account can assist looking up the name.

The identity of the workload is used to authenticate to the provider's cloud storage API when accessing a mounted path. A Cloud Account for each cloud storage provider, with the necessary access/roles, must exist and be associated with the workload identity.

Supported Providers

Identity Configuration

To allow a workload identity the ability to authenticate to an object store, a cloud access rule must be created for each provider. A Cloud Account for each provider must exists in order to create the cloud access rule.

The following list contains the minimum roles/scopes that must be added to a cloud access rule:

• S3 (using an AWS Cloud Account)

  • Select Create a new AWS role with existing policies and choose AmazonS3ReadOnlyAccess.

• Google Cloud Storage (using a Google Cloud Account)

  • Select Create a new GCP service account.
  • Resource: Storage -> Global -> Bucket -> Select bucket name.
  • Role: Storage Legacy Bucket Reader and Storage Legacy Object Reader.
  • Verify that the Cloud Account for GCP is configured correctly. In particular, the Control Plane GCP service account requires the Storage Admin role.

• Azure Blob Storage and Files (using an Azure Cloud Account)

  • Scope: Storage -> Region -> Storage Accounts -> Select storage account.
  • Role (for Azure Files) : Reader and Data Access.
  • Role (for Azure Blobs) : Storage Blob Data Reader.

Firewall Configuration

To allow a Workload access to the object stores, the outbound requests of its external firewall must either be set to All Outbound Requests Allowed or the hostnames listed below for the corresponding object store must be added to the Outbound Hostname Allow List.

AWS
copy
  *.amazonaws.com

Azure Blob
copy
  *.blob.core.windows.net
  *.azure.com

Azure File
copy
  *.file.core.windows.net
  *.azure.com

GCP
copy
  *.googleapis.com

Limitations

• Volumes are read-only, except for Azure Files.
• The following Path names are reserved:
  • /dev
  • /dev/log
  • /tmp
  • /var
  • /var/log
• Authentication to a provider is only facilitated through the workload identity. The use of an AWS or Azure key to mount a bucket/container within a container will not work.
• Properties of a mounted object store, such as cache policies and timeouts, cannot be configured by the user. Control Plane has optimized those values for each cloud provider.

Metering and Billing

The CPU, memory and egress used for mounted object stores are billed to the workload. To review the costs of mounting an object store, query the container named cpln-mounter for the workload within the metrics page.

Working Directory Override

The container working directory can be overridden by entering a custom directory. The value must be an absolute path.

Workload Health

• Possible values:
  • Loading
  • Healthy
  • Unhealthy
  • Deleting
  • Unknown