> ## Documentation Index
> Fetch the complete documentation index at: https://docs.controlplane.com/llms.txt
> Use this file to discover all available pages before exploring further.

# AWS Provider

> Configure and deploy Managed Kubernetes clusters on AWS with automated node scaling, load balancer provisioning, version upgrades, and storage management.

The AWS Provider for Managed Kubernetes is designed to facilitate the setup of Kubernetes clusters on AWS Cloud. It automates key cloud component management, enabling easy and rapid creation and maintenance of scalable, production-grade Kubernetes clusters. This provider simplifies various cloud infrastructure tasks, including node autoscaling, load balancer provisioning, version upgrades, and storage management. It enables centralized management of a fleet of Kubernetes clusters across single or multiple AWS accounts.

## Requirements

* [AWS Account](https://aws.amazon.com/console/)

## Supported add-ons

* [Dashboard:](/mk8s/add-ons/dashboard) Provides a Kubernetes dashboard UI for the cluster.
* [AWS Workload Identity:](/mk8s/add-ons/aws_workload_identity) Allows your pods to assume AWS IAM Roles.
* [AWS ECR:](/mk8s/add-ons/aws_ecr) Allows pulling images from private ECR registries.
* [AWS EFS:](/mk8s/add-ons/aws_efs) Provides support for persistent volumes using AWS Elastic File System.
* [Local Path Storage:](/mk8s/add-ons/local_path_storage) Create PVCs backed by local volumes.
* [Logs:](/mk8s/add-ons/logs) Enable logging for pods and cluster auditing.

## Step 1 - Preparing the AWS environment

**Ensure your AWS environment includes:**

* **A VPC with**:
  * At least one public subnet, or
  * At least one private subnet with a NAT gateway for egress traffic
  * [Guide on creating a VPC with AWS Console (External)](https://www.youtube.com/watch?v=ApGz8tpNLgo)

<Note>
  Tailored instructions for IAM Role creation are available in the [Control Plane Console](https://console.cpln.io/console/) when creating an AWS Kubernetes cluster using the UI. Alternatively, you can follow the instructions below:
</Note>

* **IAM Role:**
  1. Create an *IAM Role* with the naming pattern `cpln-mk8s-${ORG}`. For example, `cpln-mk8s-testing` if the organization name is `testing`.
  2. Create an *IAM Policy* with the naming pattern `cpln-mk8s-${ORG}`. For example, `cpln-mk8s-testing` if the organization name is `testing`.

```json Control Plane Connector Policy theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CreateIamResourcesForCplnPrefix",
      "Effect": "Allow",
      "Action": [
        "iam:Get*",
        "iam:CreateRole",
        "iam:DeleteRole",
        "iam:PassRole",
        "iam:CreateInstanceProfile",
        "iam:DeleteInstanceProfile",
        "iam:AddRoleToInstanceProfile",
        "iam:RemoveRoleFromInstanceProfile",
        "iam:CreatePolicy",
        "iam:DeletePolicy",
        "iam:CreatePolicyVersion",
        "iam:DeletePolicyVersion",
        "iam:AttachRolePolicy",
        "iam:DetachRolePolicy",
        "iam:UpdateRoleDescription",
        "iam:UpdateAssumeRolePolicy",
        "sts:AssumeRole"
      ],
      "Resource": [
        "arn:aws:iam::*:instance-profile/cpln-*",
        "arn:aws:iam::*:policy/cpln-*",
        "arn:aws:iam::*:role/cpln-*"
      ]
    },
    {
      "Sid": "ValidateConfiguration",
      "Effect": "Allow",
      "Action": [
        "kms:ListKeys",
        "kms:DescribeKey",
        "iam:ListPolicies",
        "iam:ListPolicyVersions",
        "iam:ListRolePolicies",
        "ec2:DescribeKeyPairs",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs",
        "ec2:DescribeSecurityGroups",
        "iam:ListAttachedRolePolicies",
        "iam:ListInstanceProfilesForRole",
        "iam:SimulatePrincipalPolicy"
      ],
      "Resource": "*"
    },
    {
      "Sid": "PrepareCluster",
      "Effect": "Allow",
      "Action": [
        "autoscaling:CreateAutoScalingGroup",
        "ec2:CreateLaunchTemplate",
        "ec2:ModifyLaunchTemplate",
        "ec2:CreateSecurityGroup",
        "ec2:CreateLaunchTemplate",
        "ec2:CreateTags",
        "autoscaling:CreateOrUpdateTags",
        "ec2:RunInstances",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:DescribeImages",
        "autoscaling:UpdateAutoScalingGroup",
        "autoscaling:StartInstanceRefresh",
        "ec2:CreateLaunchTemplateVersion",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:AuthorizeSecurityGroupEgress",
        "ec2:DescribeSecurityGroups"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Cleanup",
      "Effect": "Allow",
      "Action": [
        "autoscaling:DeleteAutoScalingGroup",
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DeleteLaunchConfiguration",
        "ec2:DeleteLaunchTemplate",
        "ec2:DeleteSecurityGroup",
        "ec2:DeleteVolume",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeVolumes",
        "elasticloadbalancing:DeleteLoadBalancer",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeTags"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AmiLookupInSSM",
      "Effect": "Allow",
      "Action": [
        "ssm:GetParameters",
        "ssm:DescribeParameters",
        "ssm:GetParameter"
      ],
      "Resource": [
        "arn:aws:ssm:*::parameter/aws/service/ami-amazon-linux-latest/*",
        "arn:aws:ssm:*::parameter/aws/service/canonical/*",
        "arn:aws:ssm:*::parameter/aws/service/eks/*"
      ]
    }
  ]
}
```

3. Attach the connector policy to the `cpln-mk8s-${ORG}` *IAM Role*.

4. Edit the trust policy of the `cpln-mk8s-${ORG}` *IAM Role*.

   **Important:** Replace `${ACCOUNT_ID}` with your own account id.

```json Trust Policy theme={null}
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/oidc.mk8s.cpln.io/federate"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "oidc.mk8s.cpln.io/federate:aud": "sts.amazonaws.com",
          "oidc.mk8s.cpln.io/federate:sub": "observer-org"
        }
      }
    }
  ]
}
```

5. Save the updates and copy the ARN of the *IAM Role* for use in the next step.

## Step 2 - Create a Managed Kubernetes Cluster Using a Manifest File

1. **Update the manifest below**: Modify the following `aws-mk8s-template.yaml` YAML manifest below with actual values.
   Customize the file as needed.
   Replace placeholders for `${ROLE_ARN}`, `${SSH_KEY_NAME}`, `${SUBNET_1}` and `${SUBNET_2}` and `${VPC_ID}`.
   Use the Role ARN of the IAM Role created in the previous step in place of `${ROLE_ARN}`.

```yaml YAML theme={null}
kind: mk8s
name: aws-mk8s-example
description: aws-mk8s-example cluster
tags: {}
spec:
  provider:
    aws:
      deployRoleArn: ${ROLE_ARN}
      image:
        recommended: ubuntu/jammy-22.04
      # keyPair: ${SSH_KEY_NAME} # Optional
      nodePools:
        - name: general
          bootDiskSize: 25
          extraSecurityGroupIds: []
          instanceTypes:
            - m7g.large
            - m6gd.large
            - c7g.xlarge
          labels: {}
          maxSize: 4
          minSize: 1
          onDemandBaseCapacity: 0
          onDemandPercentageAboveBaseCapacity: 0
          # overrideImage:
          #   recommended: amazon/amzn2
          spotAllocationStrategy: lowest-price
          subnetIds:
            - ${SUBNET_1}
            - ${SUBNET_2}
          taints: []
      region: eu-central-1
      securityGroupIds: []
      skipCreateRoles: false
      vpcId: ${VPC_ID}
  addOns:
    awsECR: {}
    dashboard: {}
    localPathStorage: {}
    awsWorkloadIdentity: {}
  firewall:
    - description: default
      sourceCIDR: 0.0.0.0/0
  version: 1.30.3
```

This example creates a managed Kubernetes cluster in AWS with the following configurations:

* **[Add-ons](#supported-add-ons)**: Includes Dashboard, Local Path Storage, AWS Workload Identity and awsECR.
* **Location**: The cluster's Kubernetes control plane is managed in the `eu-central-1` region. Placing worker nodes close to the control plane is recommended for optimal performance. Full list of supported regions: `[af-south-1, ap-east-1, ap-northeast-1, ap-northeast-2, ap-northeast-3, ap-south-1, ap-south-2, ap-southeast-1, ap-southeast-2, ap-southeast-3, ap-southeast-4, ca-central-1, eu-central-1, eu-central-2, eu-north-1, eu-south-1, eu-south-2, eu-west-1, eu-west-2, eu-west-3, me-central-1, me-south-1, sa-east-1, us-east-1, us-east-2, us-west-1, us-west-2]`.
* **Kubernetes API Firewall**: Utilizes the `Default` rule, allowing public access to the Kubernetes API. It is advisable to restrict API access to a known IP range for security purposes.
* **Kubernetes Version**: 1.30.3.
* **Node Pool**: A single `general` node pool, scaling on-demand between 1 and 4 nodes.
* **Server Image**: ubuntu/jammy-22.04.

2. **Create the Cluster**: Deploy the `aws-mk8s-example` cluster by applying the manifest.

   * **Console**: Apply the `aws-mk8s-template.yaml` file using the `cpln apply >_` option in the upper right corner.
   * **CLI**: Execute `cpln apply -f aws-mk8s-template.yaml --org YOUR_ORG_HERE`.

   **Wait until the cluster is initialized.**

## Step 3 - Accessing the Cluster

### 1. Using the Terminal

1. **Obtain the Cluster's Kubeconfig File**: Execute the command `cpln mk8s kubeconfig aws-mk8s-example -f /tmp/aws-mk8s-example-conf`.
2. **Access the Cluster with `kubectl`**: Use the obtained kubeconfig file by running `export KUBECONFIG=/tmp/aws-mk8s-example-conf` for the current shell session.

### 2. Using Kubernetes Dashboard

1. **Navigate to Control Plane Console**: Visit the [Control Plane Console](https://console.cpln.io/console/).
2. **Access the Dashboard**: In the Control Plane Console, navigate to `Kubernetes` in the left sidebar panel and click on `Open` under `Dashboard` for the cluster `aws-mk8s-example`.

## Advanced Configuration Options

### Deploy Role Chain

AWS accounts with high security requirements may not allow a direct trust policy to allow direct access from Control Plane for setup and management.
In this situation, you can configure the initial login from Control Plane into a different intermediary account and add a chain of roles that will be assumed to reach the target account.

### `.spec.provider.aws.deployRoleChain`

Each record in the `deployRoleChain` array is assumed **after** the initial login into the `deployRole`.
The chain of roles is processed in the order listed in the array, from top to bottom.
Each step in the chain must have a trust policy in place to allow `sts:assumeRole` from the previous step.

| Property            | Type              | Description                                                                                        |
| ------------------- | ----------------- | -------------------------------------------------------------------------------------------------- |
| `roleArn`           | string            | The role that will be assumed using `sts:assumeRole`.                                              |
| `externalId`        | string (optional) | The externalId to use during the assume role.                                                      |
| `sessionNamePrefix` | string (optional) | A prefix to use for the session name. The rest of the session will be generated to make it unique. |

```json example trust policy theme={null}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::333377778888:role/ExampleTrustedRole"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "ExampleExternalId"
        }
      }
    }
  ]
}

```

### Instance rotation/refresh

Sometimes an explicit rotation (or refresh) of the instances in the Auto-Scaling Group is needed. For example to get the latest image or new boot disk size.

To do this per node pool, add a label `cpln.io/refreshToken` with an arbitrary valid value. Every time the value of this label changes Control Plane will start a conservative instance replacement (1 instance at a time). If you remove that label it's the same as having an empty value and that will trigger refresh too.