mirror of
https://github.com/terraform-aws-modules/terraform-aws-eks.git
synced 2025-09-09 19:32:58 +08:00
Bryant Biggs
416515a0da
* feat!: Upgrade min AWS provider and Terraform versions to `6.0` and `1.5.7` respectively * fix: Remove deprecated arguments in AWS v6.0 provider, upgrade Helm provider to v3.0, bump VPC module to v6.0 * fix: Remove `aws-auth` sub-module * fix: Remove `platform` and `cluster_service_ipv4_cidr` variables from `user-data` sub-module * fix: Resolve all marked `todos` that have been accumulated * fix: Set default `http_put_response_hop_limit` to `1` * fix: Remove IRSA support from Karpenter sub-module * fix: Avoid making GET requests from data sources unless absolutely necessary * feat: Add variable optional attribute definitions * feat: Bump KMS key module version to latest, add remaining variable attribute definitions * fix: Remove `cluster_` prefix from variable names to better match the underlying API * fix: Move all EFA logic to the nodegroup itself * fix: Remove arguments that do not make sense in EKS * fix: Updates from plan validation * fix: Remove more self-managed node group attributes that are commonly not used in EKS clusters * fix: Remove data plane compute `*_defaults` variables that do not work with variable optional attributes * fix: Ignore changes to `bootstrap_self_managed_addons` to aid in upgrade * feat: Add support for `region` argument on relevant resources * feat: Initial pass on upgrade guide * fix: Updates from testing and validating EKS managed node group * fix: Updates from testing and validating self-managed node group * docs: Ensure addon ussage documented is aligned * feat: Switch to dualstack OIDC issuer URL * feat: Allow sourcing over overriding the Karpenter assume role policy * fix: Use `Bool` instead of `StringEquals` for DenyHTTP queue policy * fix: Correct use of `nullable` and default value propagation
Karpenter Example
Configuration in this directory creates an AWS EKS cluster with Karpenter provisioned for managing compute resource scaling. In the example provided, Karpenter is provisioned on top of an EKS Managed Node Group.
Usage
To provision the provided configurations you need to execute:
$ terraform init
$ terraform plan
$ terraform apply --auto-approve
Once the cluster is up and running, you can check that Karpenter is functioning as intended with the following command:
# First, make sure you have updated your local kubeconfig
aws eks --region eu-west-1 update-kubeconfig --name ex-karpenter
# Second, deploy the Karpenter NodeClass/NodePool
kubectl apply -f karpenter.yaml
# Second, deploy the example deployment
kubectl apply -f inflate.yaml
# You can watch Karpenter's controller logs with
kubectl logs -f -n kube-system -l app.kubernetes.io/name=karpenter -c controller
Validate if the Amazon EKS Addons Pods are running in the Managed Node Group and the inflate application Pods are running on Karpenter provisioned Nodes.
kubectl get nodes -L karpenter.sh/registered
NAME STATUS ROLES AGE VERSION REGISTERED
ip-10-0-13-51.eu-west-1.compute.internal Ready <none> 29s v1.31.1-eks-1b3e656 true
ip-10-0-41-242.eu-west-1.compute.internal Ready <none> 35m v1.31.1-eks-1b3e656
ip-10-0-8-151.eu-west-1.compute.internal Ready <none> 35m v1.31.1-eks-1b3e656
kubectl get pods -A -o custom-columns=NAME:.metadata.name,NODE:.spec.nodeName
NAME NODE
inflate-67cd5bb766-hvqfn ip-10-0-13-51.eu-west-1.compute.internal
inflate-67cd5bb766-jnsdp ip-10-0-13-51.eu-west-1.compute.internal
inflate-67cd5bb766-k4gwf ip-10-0-41-242.eu-west-1.compute.internal
inflate-67cd5bb766-m49f6 ip-10-0-13-51.eu-west-1.compute.internal
inflate-67cd5bb766-pgzx9 ip-10-0-8-151.eu-west-1.compute.internal
aws-node-58m4v ip-10-0-3-57.eu-west-1.compute.internal
aws-node-pj2gc ip-10-0-8-151.eu-west-1.compute.internal
aws-node-thffj ip-10-0-41-242.eu-west-1.compute.internal
aws-node-vh66d ip-10-0-13-51.eu-west-1.compute.internal
coredns-844dbb9f6f-9g9lg ip-10-0-41-242.eu-west-1.compute.internal
coredns-844dbb9f6f-fmzfq ip-10-0-41-242.eu-west-1.compute.internal
eks-pod-identity-agent-jr2ns ip-10-0-8-151.eu-west-1.compute.internal
eks-pod-identity-agent-mpjkq ip-10-0-13-51.eu-west-1.compute.internal
eks-pod-identity-agent-q4tjc ip-10-0-3-57.eu-west-1.compute.internal
eks-pod-identity-agent-zzfdj ip-10-0-41-242.eu-west-1.compute.internal
karpenter-5b8965dc9b-rx9bx ip-10-0-8-151.eu-west-1.compute.internal
karpenter-5b8965dc9b-xrfnx ip-10-0-41-242.eu-west-1.compute.internal
kube-proxy-2xf42 ip-10-0-41-242.eu-west-1.compute.internal
kube-proxy-kbfc8 ip-10-0-8-151.eu-west-1.compute.internal
kube-proxy-kt8zn ip-10-0-13-51.eu-west-1.compute.internal
kube-proxy-sl6bz ip-10-0-3-57.eu-west-1.compute.internal
Tear Down & Clean-Up
Because Karpenter manages the state of node resources outside of Terraform, Karpenter created resources will need to be de-provisioned first before removing the remaining resources with Terraform.
- Remove the example deployment created above and any nodes created by Karpenter
kubectl delete deployment inflate
- Remove the resources created by Terraform
terraform destroy --auto-approve
Note that this example may create resources which cost money. Run terraform destroy when you don't need these resources.
Requirements
| Name | Version |
|---|---|
| terraform | >= 1.5.7 |
| aws | >= 6.0 |
| helm | >= 3.0.0 |
Providers
| Name | Version |
|---|---|
| aws | >= 6.0 |
| aws.virginia | >= 6.0 |
| helm | >= 3.0.0 |
Modules
| Name | Source | Version |
|---|---|---|
| eks | ../.. | n/a |
| karpenter | ../../modules/karpenter | n/a |
| karpenter_disabled | ../../modules/karpenter | n/a |
| vpc | terraform-aws-modules/vpc/aws | ~> 6.0 |
Resources
| Name | Type |
|---|---|
| helm_release.karpenter | resource |
| aws_availability_zones.available | data source |
| aws_ecrpublic_authorization_token.token | data source |
Inputs
No inputs.
Outputs
No outputs.