Containers
Overview
Workloads must have at least one container configured with the following:
- Image
- Resources (CPU and Memory)
- Port(s) that the container exposes
- optional Environment Variables
- optional Container Overrides
- optional Custom Metrics
- optional Readiness / Liveness Probes
- optional Volumes
- optional Lifecycle
If a workload has more than one container, only one can serve traffic on the default global endpoint, however additional containers can receive traffic using a custom domain.
The following rules apply to the name of a container:
- Cannot be: ‘istio-proxy’, ‘queue-proxy’, ‘istio-validation’.
- Cannot start with:
cpln_
.
Images
Each workload must be configured with at least one container, associated with an image.
Images can be pulled from:
-
A public registry
- If the image does not require authentication, only the image name and optional tag are required.
- If authentication is required, a pull secret must be configured on the GVC containing the workload.
-
Org’s private registry
- If the image resides in your org’s private registry, no pull secret is required and you may use one of the following for the image name:
- Full Name:
/org/ORG_NAME/image/IMAGE_NAME:TAG
- Short Name:
//image/IMAGE_NAME:TAG
- Full Name:
- If the image resides in your org’s private registry, no pull secret is required and you may use one of the following for the image name:
-
An external private registry
- Create a pull secret with the registry credentials and configure it for the GVC.
Images must be formatted for linux/amd64 to run on the Control Plane Cloud Platform managed locations. When using a BYOK location additional runtime platforms are available.
Control Plane supports the use of dynamic tags. If an image is pushed and the tag is not updated (e.g., latest
), the platform will redploy the new image within 5 minutes.
To enable this feature:
- Using the console
- Toggle the
Support Dynamic Tag
switch on theInfo
page of the selected workload.
- Toggle the
- Using cpln apply
- Add the element
supportDynamicTags: true
within the selected workload.
- Add the element
The UserID 1337 is restricted.
If used then inbound and outbound communication to the workload will be disabled.
One known case where this UserID is configured by default is with Laravel Sail.
If you need to use Laravel Sail on Control Plane then you must change the default UserID.
Contact support for assistance.
Ports
Workloads can expose ports to be accessed by other workloads internally, externally from the provided default endpoints and from Domains. Serverless workloads must expose one port. Cron workloads may not expose ports. Standard workloads can expose 0 or more ports. The default endpoints will route traffic to the first port exposed by the first container of a workload.
ports:
- number: 8080
protocol: http
Number
The port to expose externally or internally to other Control Plane workloads. Access is controlled using the workload firewall.
Ports which are not exposed or are only accessed from another container in the same workload do NOT need to be defined.
The ports listed below are blocked and are not allowed to be used.
Containers which attempt to use these ports will not be able to bind.
8012, 8022, 9090, 9091, 15000, 15001, 15006, 15020, 15021, 15090, 41000
Protocol
- grpc
- http
- http2
- tcp
If the protocol is not specified when using the ports
then http
is configured.
If the deprecated port
parameter is used then http2
is configured.
When the protocol is tcp
…
- External access to TCP ports of workloads is not available when using default external endpoints or standard custom domains.
- The inbound allow CIDR list in firewall is always restricted to TCP ports when using default external endpoints or standard custom domains.
To enable external access through TCP and inbound firewall functionality you must:
- Enable Dedicated Load Balancer on the GVC
- Configure access using a Domain and a custom port configured with TCP.
Probes
Probes are used to check the health of an application running inside a container.
Readiness Probe
The readiness probe is used to determine if the workload replica is ready to receive traffic. For example, if the application is performing some actions during start-up and needs it to complete before serving requests, the readiness probe should fail until the actions have been completed.
This check is used in two ways.
-
Determines if replicas from a new version of the workload are ready, when the check passes the rollout continues, when the check fails the rollout is paused.
-
Determines if the workload replica should receive new requests from end users. When the readiness probe is failing the replica is removed the pool of available replicas for this workload and all endpoints.
It is recommended to use an HTTP or Command probes that perform an adequate check that the workload is healthy and able to respond to requests.
Liveness Probe
The liveness probe defines when the container should be restarted.
For example, if the application code hits a deadlock condition, the liveness probe can catch that the container is not healthy, and Control Plane will restart the failing workload replica. This will ensure that the application is available as much as possible until the defect causing the deadlock is fixed.
Options
Health Check Type:
- Run a Custom Command.
- HTTP
- Scheme (either HTTP or HTTPS, default is HTTP).
- Path.
- Port (must be between 80 and 65535 inclusive).
- Optional HTTP headers.
- TCP
- Socket port (must be between 80 and 65535 inclusive. Ports 8012, 8022, 9090, 9091, 15000, 15001, 15006, 15020, 15021, 15090, 41000 are invalid).
- gRPC
- gRPC port.
Configurable Limits:
- Initial Delay Seconds
- The delay to wait after the container is started before performing the first probe (must be between 0 and 120 inclusive, default is 0).
- Period Seconds
- How often to perform the probe (must be between 1 and 60 inclusive, default is 10).
- Timeout Seconds
- Number of seconds after which the probe times out (must be between 1 and 60 inclusive, default is 1).
- Success Threshold
- Minimum consecutive successes for the probe to be considered successful after having failed (must be between 1 and 20 inclusive, default is 1).
- Failure Threshold
- When a probe fails, Kubernetes will try this amount of times before giving up. For a liveness probe, the container will be restarted. For a readiness probe, the workload will be marked
Unready
. (must be between 1 and 20 inclusive, default is 3).
- When a probe fails, Kubernetes will try this amount of times before giving up. For a liveness probe, the container will be restarted. For a readiness probe, the workload will be marked
Refer to the Kubernetes probe documentation here for additional details.
Resources
Control Plane allows you to specify CPU and memory resources for each container.
CPU
CPU is specified in millicores (m) and cores.
- Millicore: Since a whole core might be too much for many applications, the platform allows you to specify CPU resources in millicores. One core is equivalent to 1,000 millicores. Specifying
500
(in the console) or500m
(in the YAML manifiest) would allocate half a core. - Core: A core is a CPU unit. Specifying
1
(in both the console and YAML manifest) as the CPU resource means allocating the equivalent of one full CPU core.
Memory
Memory is specified in Mebibytes (Mi) and Gibibytes (Gi).
- Mebibyte: A unit of digital data that is equal to 1,048,576 bytes. Specifying
1
and selecting Mi (in the console) or1Mi
(in the YAML manifiest) would allocate 1 Mebibyte. - Gibibyte: A unit of digital data that is equal to 1,073,741,824 bytes. Specifying
1
and selecting Gi (in the console) or1Gi
(in the YAML manifiest) would allocate 1 Gibibyte.
GPU
Workloads can be attached to the following GPUs at runtime:
GPU | # per replica |
---|---|
Nvidia T4 | 1-4 |
Nvidia A10g | 1 |
The drivers for the card will be installed and available for use automatically. Driver versions are maintained by Control Plane and kept current.
When a GPU is selected, minimum values for CPU and Memory must be met and CapacityAI is not allowed to be enabled. There are no additional charges for GPUs. The standard CPU, Memory and Egress charges are used.
Example workload container with 4 Nvidia T4 GPUs:
gpu:
nvidia:
model: t4
quantity: 4
Environment Variables
Custom environment variables can be made available to the image running within a container.
The value of the variable can be in plain text or a secret value.
Secret Variables
Sensitive values can be used as an environment variable by using a secret.
The identity of the workload must be member of a policy that has the reveal
permissions on the secret.
When adding an environment variable using the UI, a list of available secrets can be accessed by pressing Control-S within the value textbox. If you do not have any secrets defined, the prefix cpln://secret/
will be inserted.
Reference Variables
Environment variables can take on values from the pod manifest.
To reference a field, prefix the field with cpln://reference/
The supported fields are: metadata.name
, metadata.namespace
, spec.nodeName
, status.hostIP
, status.podIP
, and status.podIPs
PORT Variable
The PORT
environment variable is provided at runtime and available to a container.
It can be assigned as a custom environment variable in all cases except when the container is exposed and the value doesn’t match that of the exposed port.
For example:
- If the container is exposed with a port of
3000
:- the system will accept a PORT environment variable with the value
3000
. - the system will deny a PORT environment variable with any value other than
3000
.
- the system will accept a PORT environment variable with the value
- If the container is not exposed then any value is accepted for the PORT environment variable.
Environment Variable Inheritance
Environment variables may be set at the GVC level. These variables are available to any container running in the GVC on an opt-in basis. To opt in, set the container’s inheritEnv
property to true
. You can override the value of an inherited variable by adding a local variable with the same key.
Use by Other Variables
Environment variables can contain the values of other environment variables by following the pattern $(ENV_NAME)
.
For example, the runtime value of the built-in variable CPLN_GLOBAL_ENDPOINT
can be used in another environment variable by having the value $(CPLN_GLOBAL_ENDPOINT)
added to the new variable.
Disallowed Variables
The following variable names are not allowed to be used as a custom environment variable:
K_SERVICE
K_CONFIGURATION
K_REVISION
Import Variables
A .env file can be uploaded using the console to import multiple environment variables. Secret values are supported.
URL=http://test.example.com
USERNAME=user001
PASSWORD=cpln://secret/username_secret.password
DATA=cpln://secret/opaque_secret.payload
Container overrides
A custom command
(entrypoint), args
(arguments) or workingDir
can be provided for the container which will override the startup configuration of the container.
Command
The container entrypoint can be overridden by entering a custom command value.
Arguments
The arguments will be appended to the image ENTRYPOINT
if no command is specified.
The argument list is ordered and will be passed to the container in the same order it is defined in the api.
Working Directory
The workingDir
setting changes the directory that commands are executed in when no path or relative paths are specified.
Lifecycle
Each Workload container can be configured to execute a subset of the Kubernetes lifecycle hooks. The supported hooks are:
PostStart
This hook is executed immediately after a container is created. However, there is no guarantee that the hook will execute before the container ENTRYPOINT. In the event of a failure, the relevant error message will be recorded in the corresponding deployment.
PreStop
This hook is executed immediately before a container is stopped. In the event of a failure, the relevant error message will be recorded in the workload’s event log
Setting lifecycle hooks
These hooks can be configured using the console or cpln apply.
-
Using the console
- From the workload container, select the
Lifecycle
link from the top menu bar. - Enter the command and optional arguments.
- Click
Save
.
- From the workload container, select the
-
Using cpln apply
-
Only the
exec
type is supported. -
Example:
Add the
lifecycle
section to an existing workload container.Workload Specspec: containers: - name: advanced-options-example args: [] cpu: 50m env: [] image: '//image/IMAGE:TAG' memory: 128Mi port: 8080 lifecycle: postStart: exec: command: - sh - '-c' - sleep 10 preStop: exec: command: - sh - '-c' - sleep 10
-