{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Terraform AWS EKS module","text":"

Moar content coming soon!

"},{"location":"UPGRADE-17.0/","title":"How to handle the terraform-aws-eks module upgrade","text":""},{"location":"UPGRADE-17.0/#upgrade-module-to-v1700-for-managed-node-groups","title":"Upgrade module to v17.0.0 for Managed Node Groups","text":"

In this release, we now decided to remove random_pet resources in Managed Node Groups (MNG). Those were used to recreate MNG if something changed. But they were causing a lot of issues. To upgrade the module without recreating your MNG, you will need to explicitly reuse their previous name and set them in your MNG name argument.

  1. Run terraform apply with the module version v16.2.0
  2. Get your worker group names
~ terraform state show 'module.eks.module.node_groups.aws_eks_node_group.workers[\"example\"]' | grep node_group_name\nnode_group_name = \"test-eks-mwIwsvui-example-sincere-squid\"\n
  1. Upgrade your module and configure your node groups to use existing names
module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"17.0.0\"\n\n  cluster_name    = \"test-eks-mwIwsvui\"\n  cluster_version = \"1.20\"\n  # ...\n\n  node_groups = {\n    example = {\n      name = \"test-eks-mwIwsvui-example-sincere-squid\"\n\n      # ...\n    }\n  }\n  # ...\n}\n
  1. Run terraform plan, you should see that only random_pets will be destroyed
Terraform will perform the following actions:\n\n  # module.eks.module.node_groups.random_pet.node_groups[\"example\"] will be destroyed\n  - resource \"random_pet\" \"node_groups\" {\n      - id        = \"sincere-squid\" -> null\n      - keepers   = {\n          - \"ami_type\"                  = \"AL2_x86_64\"\n          - \"capacity_type\"             = \"SPOT\"\n          - \"disk_size\"                 = \"50\"\n          - \"iam_role_arn\"              = \"arn:aws:iam::123456789123:role/test-eks-mwIwsvui20210527220853611600000009\"\n          - \"instance_types\"            = \"t3.large\"\n          - \"key_name\"                  = \"\"\n          - \"node_group_name\"           = \"test-eks-mwIwsvui-example\"\n          - \"source_security_group_ids\" = \"\"\n          - \"subnet_ids\"                = \"subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx\"\n        } -> null\n      - length    = 2 -> null\n      - separator = \"-\" -> null\n    }\n\nPlan: 0 to add, 0 to change, 1 to destroy.\n
  1. If everything sounds good to you, run terraform apply

After the first apply, we recommend you to create a new node group and let the module use the node_group_name_prefix (by removing the name argument) to generate names and avoid collision during node groups re-creation if needed, because the lifecycle is create_before_destroy = true.

"},{"location":"UPGRADE-18.0/","title":"Upgrade from v17.x to v18.x","text":"

Please consult the examples directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.

Note: please see https://github.com/terraform-aws-modules/terraform-aws-eks/issues/1744 where users have shared the steps/changes that have worked for their configurations to upgrade. Due to the numerous configuration possibilities, it is difficult to capture specific steps that will work for all; this has proven to be a useful thread to share collective information from the broader community regarding v18.x upgrades.

For most users, adding the following to your v17.x configuration will preserve the state of your cluster control plane when upgrading to v18.x:

prefix_separator                   = \"\"\niam_role_name                      = $CLUSTER_NAME\ncluster_security_group_name        = $CLUSTER_NAME\ncluster_security_group_description = \"EKS cluster security group.\"\n

This configuration assumes that create_iam_role is set to true, which is the default value.

As the location of the Terraform state of the IAM role has been changed from 17.x to 18.x, you'll also have to move the state before running terraform apply by calling:

terraform state mv 'module.eks.aws_iam_role.cluster[0]' 'module.eks.aws_iam_role.this[0]'\n

See more information here

"},{"location":"UPGRADE-18.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":""},{"location":"UPGRADE-18.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-18.0/#added","title":"Added","text":""},{"location":"UPGRADE-18.0/#modified","title":"Modified","text":""},{"location":"UPGRADE-18.0/#removed","title":"Removed","text":""},{"location":"UPGRADE-18.0/#variable-and-output-changes","title":"Variable and output changes","text":"

  1. Removed variables:

  2. Renamed variables:

  3. Added variables:

  4. Removed outputs:

  5. Renamed outputs:

  6. Added outputs:

"},{"location":"UPGRADE-18.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-18.0/#before-17x-example","title":"Before 17.x Example","text":"
module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~> 17.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id  = module.vpc.vpc_id\n  subnets = module.vpc.private_subnets\n\n  # Managed Node Groups\n  node_groups_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  node_groups = {\n    node_group = {\n      min_capacity     = 1\n      max_capacity     = 10\n      desired_capacity = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      k8s_labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      additional_tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Worker groups\n  worker_additional_security_group_ids = [aws_security_group.additional.id]\n\n  worker_groups_launch_template = [\n    {\n      name                    = \"worker-group\"\n      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n      spot_instance_pools     = 4\n      asg_max_size            = 5\n      asg_desired_capacity    = 2\n      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n      public_ip               = true\n    },\n  ]\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n
"},{"location":"UPGRADE-18.0/#after-18x-example","title":"After 18.x Example","text":"
module \"cluster_after\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~> 18.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id     = module.vpc.vpc_id\n  subnet_ids = module.vpc.private_subnets\n\n  eks_managed_node_group_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  eks_managed_node_groups = {\n    node_group = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n  }\n\n  self_managed_node_groups = {\n    worker_group = {\n      name = \"worker-group\"\n\n      min_size      = 1\n      max_size      = 5\n      desired_size  = 2\n      instance_type = \"m4.large\"\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      block_device_mappings = {\n        xvda = {\n          device_name = \"/dev/xvda\"\n          ebs = {\n            delete_on_termination = true\n            encrypted             = false\n            volume_size           = 100\n            volume_type           = \"gp2\"\n          }\n\n        }\n      }\n\n      use_mixed_instances_policy = true\n      mixed_instances_policy = {\n        instances_distribution = {\n          spot_instance_pools = 4\n        }\n\n        override = [\n          { instance_type = \"m5.large\" },\n          { instance_type = \"m5a.large\" },\n          { instance_type = \"m5d.large\" },\n          { instance_type = \"m5ad.large\" },\n        ]\n      }\n    }\n  }\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n
"},{"location":"UPGRADE-18.0/#diff-of-before-after","title":"Diff of before <> after","text":"
 module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~> 17.0\"\n+  version = \"~> 18.0\"\n\n   cluster_name                    = local.name\n   cluster_version                 = local.cluster_version\n   cluster_endpoint_private_access = true\n   cluster_endpoint_public_access  = true\n\n   vpc_id  = module.vpc.vpc_id\n-  subnets = module.vpc.private_subnets\n+  subnet_ids = module.vpc.private_subnets\n\n-  # Managed Node Groups\n-  node_groups_defaults = {\n+  eks_managed_node_group_defaults = {\n     ami_type  = \"AL2_x86_64\"\n     disk_size = 50\n   }\n\n-  node_groups = {\n+  eks_managed_node_groups = {\n     node_group = {\n-      min_capacity     = 1\n-      max_capacity     = 10\n-      desired_capacity = 1\n+      min_size     = 1\n+      max_size     = 10\n+      desired_size = 1\n\n       instance_types = [\"t3.large\"]\n       capacity_type  = \"SPOT\"\n\n       update_config = {\n         max_unavailable_percentage = 50\n       }\n\n-      k8s_labels = {\n+      labels = {\n         Environment = \"test\"\n         GithubRepo  = \"terraform-aws-eks\"\n         GithubOrg   = \"terraform-aws-modules\"\n       }\n\n       taints = [\n         {\n           key    = \"dedicated\"\n           value  = \"gpuGroup\"\n           effect = \"NO_SCHEDULE\"\n         }\n       ]\n\n-      additional_tags = {\n+      tags = {\n         ExtraTag = \"example\"\n       }\n     }\n   }\n\n-  # Worker groups\n-  worker_additional_security_group_ids = [aws_security_group.additional.id]\n-\n-  worker_groups_launch_template = [\n-    {\n-      name                    = \"worker-group\"\n-      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n-      spot_instance_pools     = 4\n-      asg_max_size            = 5\n-      asg_desired_capacity    = 2\n-      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n-      public_ip               = true\n-    },\n-  ]\n+  self_managed_node_group_defaults = {\n+    vpc_security_group_ids = [aws_security_group.additional.id]\n+  }\n+\n+  self_managed_node_groups = {\n+    worker_group = {\n+      name = \"worker-group\"\n+\n+      min_size      = 1\n+      max_size      = 5\n+      desired_size  = 2\n+      instance_type = \"m4.large\"\n+\n+      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n+\n+      block_device_mappings = {\n+        xvda = {\n+          device_name = \"/dev/xvda\"\n+          ebs = {\n+            delete_on_termination = true\n+            encrypted             = false\n+            volume_size           = 100\n+            volume_type           = \"gp2\"\n+          }\n+\n+        }\n+      }\n+\n+      use_mixed_instances_policy = true\n+      mixed_instances_policy = {\n+        instances_distribution = {\n+          spot_instance_pools = 4\n+        }\n+\n+        override = [\n+          { instance_type = \"m5.large\" },\n+          { instance_type = \"m5a.large\" },\n+          { instance_type = \"m5d.large\" },\n+          { instance_type = \"m5ad.large\" },\n+        ]\n+      }\n+    }\n+  }\n\n   # Fargate\n   fargate_profiles = {\n     default = {\n       name = \"default\"\n       selectors = [\n         {\n           namespace = \"kube-system\"\n           labels = {\n             k8s-app = \"kube-dns\"\n           }\n         },\n         {\n           namespace = \"default\"\n         }\n       ]\n\n       tags = {\n         Owner = \"test\"\n       }\n\n       timeouts = {\n         create = \"20m\"\n         delete = \"20m\"\n       }\n     }\n   }\n\n   tags = {\n     Environment = \"test\"\n     GithubRepo  = \"terraform-aws-eks\"\n     GithubOrg   = \"terraform-aws-modules\"\n   }\n }\n
"},{"location":"UPGRADE-18.0/#attaching-an-iam-role-policy-to-a-fargate-profile","title":"Attaching an IAM role policy to a Fargate profile","text":""},{"location":"UPGRADE-18.0/#before-17x","title":"Before 17.x","text":"
resource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n
"},{"location":"UPGRADE-18.0/#after-18x","title":"After 18.x","text":"
# Attach the policy to an \"example\" Fargate profile\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_profiles[\"example\"].iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n

Or:

# Attach the policy to all Fargate profiles\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  for_each = module.eks.fargate_profiles\n\n  role       = each.value.iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n
"},{"location":"UPGRADE-19.0/","title":"Upgrade from v18.x to v19.x","text":"

Please consult the examples directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.

"},{"location":"UPGRADE-19.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":""},{"location":"UPGRADE-19.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-19.0/#added","title":"Added","text":""},{"location":"UPGRADE-19.0/#modified","title":"Modified","text":""},{"location":"UPGRADE-19.0/#removed","title":"Removed","text":""},{"location":"UPGRADE-19.0/#variable-and-output-changes","title":"Variable and output changes","text":"

  1. Removed variables:

  2. node_security_group_ntp_ipv4_cidr_block - default security group settings have an egress rule for ALL to 0.0.0.0/0/::/0

  3. node_security_group_ntp_ipv6_cidr_block - default security group settings have an egress rule for ALL to 0.0.0.0/0/::/0
  4. Self-managed node groups:

  5. EKS managed node groups:

  6. Renamed variables:

  7. N/A

  8. Added variables:

  9. provision_on_outpostfor Outposts support

  10. outpost_config for Outposts support
  11. cluster_addons_timeouts for setting a common set of timeouts for all addons (unless a specific value is provided within the addon configuration)
  12. service_ipv6_cidr for setting the IPv6 CIDR block for the Kubernetes service addresses

  13. node_security_group_enable_recommended_rules for enabling recommended node security group rules for common access patterns

  14. Self-managed node groups:

  15. EKS managed node groups:

  16. Removed outputs:

  17. Self-managed node groups:

  18. EKS managed node groups:

  19. Renamed outputs:

  20. cluster_id is not renamed but the value it returns is now different. For standard EKS clusters created in the AWS cloud, the value returned at the time of this writing is null/empty. For local EKS clusters created on Outposts, the value returned will look like a UUID/GUID. Users should switch all instances of cluster_id to use cluster_name before upgrading to v19. Reference

  21. Added outputs:

  22. cluster_name - The cluster_id currently set by the AWS provider is actually the cluster name, but in the future, this will change and there will be a distinction between the cluster_name and cluster_id. Reference

"},{"location":"UPGRADE-19.0/#upgrade-migrations","title":"Upgrade Migrations","text":"
  1. Before upgrading your module definition to v19.x, please see below for both EKS managed node group(s) and self-managed node groups and remove the node group(s) security group prior to upgrading.
"},{"location":"UPGRADE-19.0/#self-managed-node-groups","title":"Self-Managed Node Groups","text":"

Self-managed node groups on v18.x by default create a security group that does not specify any rules. In v19.x, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version v18.x of your module definition, remove this security group from your node groups by setting create_security_group = false.

"},{"location":"UPGRADE-19.0/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"

EKS managed node groups on v18.x by default create a security group that does not specify any rules. In v19.x, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version v18.x of your module definition, remove this security group from your node groups by setting create_security_group = false.

"},{"location":"UPGRADE-19.0/#diff-of-before-v18x-vs-after-v19x","title":"Diff of Before (v18.x) vs After (v19.x)","text":"
 module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~> 18.0\"\n+  version = \"~> 19.0\"\n\n  cluster_name                    = local.name\n+ cluster_endpoint_public_access  = true\n- cluster_endpoint_private_access = true # now the default\n\n  cluster_addons = {\n-   resolve_conflicts = \"OVERWRITE\" # now the default\n+   preserve          = true\n+   most_recent       = true\n\n+   timeouts = {\n+     create = \"25m\"\n+     delete = \"10m\"\n    }\n    kube-proxy = {}\n    vpc-cni = {\n-     resolve_conflicts = \"OVERWRITE\" # now the default\n    }\n  }\n\n  # Encryption key\n  create_kms_key = true\n- cluster_encryption_config = [{\n-   resources = [\"secrets\"]\n- }]\n+ cluster_encryption_config = {\n+   resources = [\"secrets\"]\n+ }\n  kms_key_deletion_window_in_days = 7\n  enable_kms_key_rotation         = true\n\n- iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+ iam_role_additional_policies = {\n+   additional = aws_iam_policy.additional.arn\n+ }\n\n  vpc_id                   = module.vpc.vpc_id\n  subnet_ids               = module.vpc.private_subnets\n  control_plane_subnet_ids = module.vpc.intra_subnets\n\n  # Extend node-to-node security group rules\n- node_security_group_ntp_ipv4_cidr_block = [\"169.254.169.123/32\"] # now the default\n  node_security_group_additional_rules = {\n-    ingress_self_ephemeral = {\n-      description = \"Node to node ephemeral ports\"\n-      protocol    = \"tcp\"\n-      from_port   = 0\n-      to_port     = 0\n-      type        = \"ingress\"\n-      self        = true\n-    }\n-    egress_all = {\n-      description      = \"Node all egress\"\n-      protocol         = \"-1\"\n-      from_port        = 0\n-      to_port          = 0\n-      type             = \"egress\"\n-      cidr_blocks      = [\"0.0.0.0/0\"]\n-      ipv6_cidr_blocks = [\"::/0\"]\n-    }\n  }\n\n  # Self-Managed Node Group(s)\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  self_managed_node_groups = {\n    spot = {\n      instance_type = \"m5.large\"\n      instance_market_options = {\n        market_type = \"spot\"\n      }\n\n      pre_bootstrap_user_data = <<-EOT\n        echo \"foo\"\n        export FOO=bar\n      EOT\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      post_bootstrap_user_data = <<-EOT\n        cd /tmp\n        sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm\n        sudo systemctl enable amazon-ssm-agent\n        sudo systemctl start amazon-ssm-agent\n      EOT\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n    }\n  }\n\n  # EKS Managed Node Group(s)\n  eks_managed_node_group_defaults = {\n    ami_type       = \"AL2_x86_64\"\n    instance_types = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n\n    attach_cluster_primary_security_group = true\n    vpc_security_group_ids                = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  eks_managed_node_groups = {\n    blue = {}\n    green = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = {\n        dedicated = {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      }\n\n      update_config = {\n        max_unavailable_percentage = 33 # or set `max_unavailable`\n      }\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Fargate Profile(s)\n  fargate_profile_defaults = {\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  # OIDC Identity provider\n  cluster_identity_providers = {\n    cognito = {\n      client_id      = \"702vqsrjicklgb7c5b7b50i1gc\"\n      issuer_url     = \"https://cognito-idp.us-west-2.amazonaws.com/us-west-2_re1u6bpRA\"\n      username_claim = \"email\"\n      groups_claim   = \"cognito:groups\"\n      groups_prefix  = \"gid:\"\n    }\n  }\n\n  # aws-auth configmap\n  manage_aws_auth_configmap = true\n\n  aws_auth_node_iam_role_arns_non_windows = [\n    module.eks_managed_node_group.iam_role_arn,\n    module.self_managed_node_group.iam_role_arn,\n  ]\n  aws_auth_fargate_profile_pod_execution_role_arns = [\n    module.fargate_profile.fargate_profile_pod_execution_role_arn\n  ]\n\n  aws_auth_roles = [\n    {\n      rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n      username = \"role1\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_users = [\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user1\"\n      username = \"user1\"\n      groups   = [\"system:masters\"]\n    },\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user2\"\n      username = \"user2\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_accounts = [\n    \"777777777777\",\n    \"888888888888\",\n  ]\n\n  tags = local.tags\n}\n
"},{"location":"UPGRADE-19.0/#terraform-state-moves","title":"Terraform State Moves","text":"

The following Terraform state move commands are optional but recommended if you are providing additional IAM policies that are to be attached to IAM roles created by this module (cluster IAM role, node group IAM role, Fargate profile IAM role). Because the resources affected are aws_iam_role_policy_attachment, in theory, you could get away with simply applying the configuration and letting Terraform detach and re-attach the policies. However, during this brief period of update, you could experience permission failures as the policy is detached and re-attached, and therefore the state move route is recommended.

Where \"<POLICY_ARN>\" is specified, this should be replaced with the full ARN of the policy, and \"<POLICY_MAP_KEY>\" should be replaced with the key used in the iam_role_additional_policies map for the associated policy. For example, if you have the followingv19.x configuration:

  ...\n  # This is demonstrating the cluster IAM role additional policies\n  iam_role_additional_policies = {\n    additional = aws_iam_policy.additional.arn\n  }\n  ...\n

The associated state move command would look similar to (albeit with your correct policy ARN):

terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"arn:aws:iam::111111111111:policy/ex-complete-additional\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"additional\"]'\n

If you are not providing any additional IAM policies, no actions are required.

"},{"location":"UPGRADE-19.0/#cluster-iam-role","title":"Cluster IAM Role","text":"

Repeat for each policy provided in iam_role_additional_policies:

terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"<POLICY_ARN>\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"<POLICY_MAP_KEY>\"]'\n
"},{"location":"UPGRADE-19.0/#eks-managed-node-group-iam-role","title":"EKS Managed Node Group IAM Role","text":"

Where \"<NODE_GROUP_KEY>\" is the key used in the eks_managed_node_groups map for the associated node group. Repeat for each policy provided in iam_role_additional_policies in either/or eks_managed_node_group_defaults or the individual node group definitions:

terraform state mv 'module.eks.module.eks_managed_node_group[\"<NODE_GROUP_KEY>\"].aws_iam_role_policy_attachment.this[\"<POLICY_ARN>\"]' 'module.eks.module.eks_managed_node_group[\"<NODE_GROUP_KEY>\"].aws_iam_role_policy_attachment.additional[\"<POLICY_MAP_KEY>\"]'\n
"},{"location":"UPGRADE-19.0/#self-managed-node-group-iam-role","title":"Self-Managed Node Group IAM Role","text":"

Where \"<NODE_GROUP_KEY>\" is the key used in the self_managed_node_groups map for the associated node group. Repeat for each policy provided in iam_role_additional_policies in either/or self_managed_node_group_defaults or the individual node group definitions:

terraform state mv 'module.eks.module.self_managed_node_group[\"<NODE_GROUP_KEY>\"].aws_iam_role_policy_attachment.this[\"<POLICY_ARN>\"]' 'module.eks.module.self_managed_node_group[\"<NODE_GROUP_KEY>\"].aws_iam_role_policy_attachment.additional[\"<POLICY_MAP_KEY>\"]'\n
"},{"location":"UPGRADE-19.0/#fargate-profile-iam-role","title":"Fargate Profile IAM Role","text":"

Where \"<FARGATE_PROFILE_KEY>\" is the key used in the fargate_profiles map for the associated profile. Repeat for each policy provided in iam_role_additional_policies in either/or fargate_profile_defaults or the individual profile definitions:

terraform state mv 'module.eks.module.fargate_profile[\"<FARGATE_PROFILE_KEY>\"].aws_iam_role_policy_attachment.this[\"<POLICY_ARN>\"]' 'module.eks.module.fargate_profile[\"<FARGATE_PROFILE_KEY>\"].aws_iam_role_policy_attachment.additional[\"<POLICY_MAP_KEY>\"]'\n
"},{"location":"UPGRADE-20.0/","title":"Upgrade from v19.x to v20.x","text":"

Please consult the examples directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.

"},{"location":"UPGRADE-20.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":""},{"location":"UPGRADE-20.0/#upcoming-changes-planned-in-v210","title":"\u26a0\ufe0f Upcoming Changes Planned in v21.0 \u26a0\ufe0f","text":"

To give users advanced notice and provide some future direction for this module, these are the following changes we will be looking to make in the next major release of this module:

  1. The aws-auth sub-module will be removed entirely from the project. Since this sub-module is captured in the v20.x releases, users can continue using it even after the module moves forward with the next major version. The long term strategy and direction is cluster access entry and to rely only on the AWS Terraform provider.
  2. The default value for authentication_mode will change to API. Aligning with point 1 above, this is a one way change, but users are free to specify the value of their choosing in place of this default (when the change is made). This module will proceed with an EKS API first strategy.
  3. The launch template and autoscaling group usage contained within the EKS managed node group and self-managed node group sub-modules *might be replaced with the terraform-aws-autoscaling module. At minimum, it makes sense to replace most of functionality in the self-managed node group module with this external module, but its not yet clear if there is any benefit of using it in the EKS managed node group sub-module. The interface that users interact with will stay the same, the changes will be internal to the implementation and we will do everything we can to keep the disruption to a minimum.
  4. The platform variable will be replaced and instead ami_type will become the standard across both self-managed node group(s) and EKS managed node group(s). As EKS expands its portfolio of supported operating systems, the ami_type is better suited to associate the correct user data format to the respective OS. The platform variable is a legacy artifact of self-managed node groups but not as descriptive as the ami_type, and therefore it will be removed in favor of ami_type.
"},{"location":"UPGRADE-20.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-20.0/#added","title":"Added","text":""},{"location":"UPGRADE-20.0/#modified","title":"Modified","text":""},{"location":"UPGRADE-20.0/#removed","title":"Removed","text":""},{"location":"UPGRADE-20.0/#variable-and-output-changes","title":"Variable and output changes","text":"

  1. Removed variables:

  2. cluster_iam_role_dns_suffix - replaced with a static string of amazonaws.com

  3. manage_aws_auth_configmap
  4. create_aws_auth_configmap
  5. aws_auth_node_iam_role_arns_non_windows
  6. aws_auth_node_iam_role_arns_windows
  7. aws_auth_fargate_profile_pod_execution_role_arn
  8. aws_auth_roles
  9. aws_auth_users

  10. aws_auth_accounts

  11. Karpenter

  12. Renamed variables:

  13. Karpenter

  14. Added variables:

  15. create_access_entry

  16. enable_cluster_creator_admin_permissions
  17. authentication_mode
  18. access_entries

  19. cloudwatch_log_group_class

  20. Karpenter

  21. Self-managed node group

  22. Removed outputs:

  23. aws_auth_configmap_yaml

  24. Renamed outputs:

  25. Karpenter

  26. Added outputs:

  27. access_entries

  28. Karpenter

  29. Self-managed node group

"},{"location":"UPGRADE-20.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-20.0/#diff-of-before-v1921-vs-after-v200","title":"Diff of Before (v19.21) vs After (v20.0)","text":"
 module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~> 19.21\"\n+  version = \"~> 20.0\"\n\n# If you want to maintain the current default behavior of v19.x\n+  kms_key_enable_default_policy = false\n\n-   manage_aws_auth_configmap = true\n\n-   aws_auth_roles = [\n-     {\n-       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n-       username = \"role1\"\n-       groups   = [\"custom-role-group\"]\n-     },\n-   ]\n\n-   aws_auth_users = [\n-     {\n-       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n-       username = \"user1\"\n-       groups   = [\"custom-users-group\"]\n-     },\n-   ]\n}\n\n+ module \"eks_aws_auth\" {\n+   source  = \"terraform-aws-modules/eks/aws//modules/aws-auth\"\n+   version = \"~> 20.0\"\n\n+   manage_aws_auth_configmap = true\n\n+   aws_auth_roles = [\n+     {\n+       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n+       username = \"role1\"\n+       groups   = [\"custom-role-group\"]\n+     },\n+   ]\n\n+   aws_auth_users = [\n+     {\n+       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n+       username = \"user1\"\n+       groups   = [\"custom-users-group\"]\n+     },\n+   ]\n+ }\n
"},{"location":"UPGRADE-20.0/#karpenter-diff-of-before-v1921-vs-after-v200","title":"Karpenter Diff of Before (v19.21) vs After (v20.0)","text":"
 module \"eks_karpenter\" {\n   source  = \"terraform-aws-modules/eks/aws//modules/karpenter\"\n-  version = \"~> 19.21\"\n+  version = \"~> 20.0\"\n\n# If you wish to maintain the current default behavior of v19.x\n+  enable_irsa             = true\n+  create_instance_profile = true\n\n# To avoid any resource re-creation\n+  iam_role_name          = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_role_description   = \"Karpenter IAM role for service account\"\n+  iam_policy_name        = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_policy_description = \"Karpenter IAM role for service account\"\n}\n
"},{"location":"UPGRADE-20.0/#terraform-state-moves","title":"Terraform State Moves","text":""},{"location":"UPGRADE-20.0/#authentication-mode-changes","title":"\u26a0\ufe0f Authentication Mode Changes \u26a0\ufe0f","text":"

Changing the authentication_mode is a one-way decision. See announcement blog for further details:

Switching authentication modes on an existing cluster is a one-way operation. You can switch from CONFIG_MAP to API_AND_CONFIG_MAP. You can then switch from API_AND_CONFIG_MAP to API. You cannot revert these operations in the opposite direction. Meaning you cannot switch back to CONFIG_MAP or API_AND_CONFIG_MAP from API. And you cannot switch back to CONFIG_MAP from API_AND_CONFIG_MAP.

[!IMPORTANT] If migrating to cluster access entries and you will NOT have any entries that remain in the aws-auth ConfigMap, you do not need to remove the configmap from the statefile. You can simply follow the migration guide and once access entries have been created, you can let Terraform remove/delete the aws-auth ConfigMap.

If you WILL have entries that remain in the aws-auth ConfigMap, then you will need to remove the ConfigMap resources from the statefile to avoid any disruptions. When you add the new aws-auth sub-module and apply the changes, the sub-module will upsert the ConfigMap on the cluster. Provided the necessary entries are defined in that sub-module's definition, it will \"re-adopt\" the ConfigMap under Terraform's control.

"},{"location":"UPGRADE-20.0/#authentication_mode-config_map","title":"authentication_mode = \"CONFIG_MAP\"","text":"

If using authentication_mode = \"CONFIG_MAP\", before making any changes, you will first need to remove the configmap from the statefile to avoid any disruptions:

terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n

Once the configmap has been removed from the statefile, you can add the new aws-auth sub-module and copy the relevant definitions from the EKS module over to the new aws-auth sub-module definition (see before after diff above).

[!CAUTION] You will need to add entries to the aws-auth sub-module for any IAM roles used by node groups and/or Fargate profiles - the module no longer handles this in the background on behalf of users.

When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes.

"},{"location":"UPGRADE-20.0/#authentication_mode-api_and_config_map","title":"authentication_mode = \"API_AND_CONFIG_MAP\"","text":"

When using authentication_mode = \"API_AND_CONFIG_MAP\" and there are entries that will remain in the configmap (entries that cannot be replaced by cluster access entry), you will first need to update the authentication_mode on the cluster to \"API_AND_CONFIG_MAP\". To help make this upgrade process easier, a copy of the changes defined in the v20.0.0 PR have been captured here but with the aws-auth components still provided in the module. This means you get the equivalent of the v20.0.0 module, but it still includes support for the aws-auth configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the authentication_mode has been updated to \"API_AND_CONFIG_MAP\". Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.

Once the authentication_mode has been updated, next you will need to remove the configmap from the statefile to avoid any disruptions:

[!NOTE] This is only required if there are entries that will remain in the aws-auth ConfigMap after migrating. Otherwise, you can skip this step and let Terraform destroy the ConfigMap.

terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n
"},{"location":"UPGRADE-20.0/#i-terraform-17-users","title":"\u2139\ufe0f Terraform 1.7+ users","text":"

If you are using Terraform v1.7+, you can utilize the remove to facilitate both the removal of the configmap through code. You can create a fork/clone of the provided migration module and add the remove blocks and apply those changes before proceeding. We do not want to force users onto the bleeding edge with this module, so we have not included remove support at this time.

Once the configmap has been removed from the statefile, you can add the new aws-auth sub-module and copy the relevant definitions from the EKS module over to the new aws-auth sub-module definition (see before after diff above). When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes. In the before/example above - the configmap would remove any entries for roles used by node groups and/or Fargate Profiles, but maintain the custom entries for users and roles passed into the module definition.

"},{"location":"UPGRADE-20.0/#authentication_mode-api","title":"authentication_mode = \"API\"","text":"

In order to switch to API only using cluster access entry, you first need to update the authentication_mode on the cluster to API_AND_CONFIG_MAP without modifying the aws-auth configmap. To help make this upgrade process easier, a copy of the changes defined in the v20.0.0 PR have been captured here but with the aws-auth components still provided in the module. This means you get the equivalent of the v20.0.0 module, but it still includes support for the aws-auth configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the authentication_mode has been updated to \"API_AND_CONFIG_MAP\". Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.

Once the authentication_mode has been updated, you can update the authentication_mode on the cluster to API and remove the aws-auth configmap components.

"},{"location":"compute_resources/","title":"Compute Resources","text":""},{"location":"compute_resources/#table-of-contents","title":"Table of Contents","text":"

\u2139\ufe0f Only the pertinent attributes are shown below for brevity

"},{"location":"compute_resources/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"

Refer to the EKS Managed Node Group documentation documentation for service related details.

  1. The module creates a custom launch template by default to ensure settings such as tags are propagated to instances. Please note that many of the customization options listed here are only available when a custom launch template is created. To use the default template provided by the AWS EKS managed node group service, disable the launch template creation by setting use_custom_launch_template to false:
  eks_managed_node_groups = {\n    default = {\n      use_custom_launch_template = false\n    }\n  }\n
  1. Native support for Bottlerocket OS is provided by providing the respective AMI type:
  eks_managed_node_groups = {\n    bottlerocket_default = {\n      use_custom_launch_template = false\n\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n
  1. Bottlerocket OS is supported in a similar manner. However, note that the user data for Bottlerocket OS uses the TOML format:
  eks_managed_node_groups = {\n    bottlerocket_prepend_userdata = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      bootstrap_extra_args = <<-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n      EOT\n    }\n  }\n
  1. When using a custom AMI, the AWS EKS Managed Node Group service will NOT inject the necessary bootstrap script into the supplied user data. Users can elect to provide their own user data to bootstrap and connect or opt in to use the module provided user data:
  eks_managed_node_groups = {\n    custom_ami = {\n      ami_id = \"ami-0caf35bc73450c396\"\n\n      # By default, EKS managed node groups will not append bootstrap script;\n      # this adds it back in using the default template provided by the module\n      # Note: this assumes the AMI provided is an EKS optimized AMI derivative\n      enable_bootstrap_user_data = true\n\n      pre_bootstrap_user_data = <<-EOT\n        export FOO=bar\n      EOT\n\n      # Because we have full control over the user data supplied, we can also run additional\n      # scripts/configuration changes after the bootstrap script has been run\n      post_bootstrap_user_data = <<-EOT\n        echo \"you are free little kubelet!\"\n      EOT\n    }\n  }\n
  1. There is similar support for Bottlerocket OS:
  eks_managed_node_groups = {\n    bottlerocket_custom_ami = {\n      ami_id   = \"ami-0ff61e0bcfc81dc94\"\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      # use module user data template to bootstrap\n      enable_bootstrap_user_data = true\n      # this will get added to the template\n      bootstrap_extra_args = <<-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n\n        [settings.kubernetes.node-labels]\n        \"label1\" = \"foo\"\n        \"label2\" = \"bar\"\n\n        [settings.kubernetes.node-taints]\n        \"dedicated\" = \"experimental:PreferNoSchedule\"\n        \"special\" = \"true:NoSchedule\"\n      EOT\n    }\n  }\n

See the examples/eks-managed-node-group/ example for a working example of various configurations.

"},{"location":"compute_resources/#self-managed-node-groups","title":"Self Managed Node Groups","text":"

Refer to the Self Managed Node Group documentation documentation for service related details.

  1. The self-managed-node-group uses the latest AWS EKS Optimized AMI (Linux) for the given Kubernetes version by default:
  cluster_version = \"1.31\"\n\n  # This self managed node group will use the latest AWS EKS Optimized AMI for Kubernetes 1.27\n  self_managed_node_groups = {\n    default = {}\n  }\n
  1. To use Bottlerocket, specify the ami_type as one of the respective \"BOTTLEROCKET_*\" types and supply a Bottlerocket OS AMI:
  cluster_version = \"1.31\"\n\n  self_managed_node_groups = {\n    bottlerocket = {\n      ami_id   = data.aws_ami.bottlerocket_ami.id\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n

See the examples/self-managed-node-group/ example for a working example of various configurations.

"},{"location":"compute_resources/#fargate-profiles","title":"Fargate Profiles","text":"

Fargate profiles are straightforward to use and therefore no further details are provided here. See the tests/fargate-profile/ tests for a working example of various configurations.

"},{"location":"compute_resources/#default-configurations","title":"Default Configurations","text":"

Each type of compute resource (EKS managed node group, self managed node group, or Fargate profile) provides the option for users to specify a default configuration. These default configurations can be overridden from within the compute resource's individual definition. The order of precedence for configurations (from highest to least precedence):

For example, the following creates 4 AWS EKS Managed Node Groups:

  eks_managed_node_group_defaults = {\n    ami_type               = \"AL2_x86_64\"\n    disk_size              = 50\n    instance_types         = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n  }\n\n  eks_managed_node_groups = {\n    # Uses module default configurations overridden by configuration above\n    default = {}\n\n    # This further overrides the instance types used\n    compute = {\n      instance_types = [\"c5.large\", \"c6i.large\", \"c6d.large\"]\n    }\n\n    # This further overrides the instance types and disk size used\n    persistent = {\n      disk_size = 1024\n      instance_types = [\"r5.xlarge\", \"r6i.xlarge\", \"r5b.xlarge\"]\n    }\n\n    # This overrides the OS used\n    bottlerocket = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n
"},{"location":"faq/","title":"Frequently Asked Questions","text":""},{"location":"faq/#setting-disk_size-or-remote_access-does-not-make-any-changes","title":"Setting disk_size or remote_access does not make any changes","text":"

disk_size, and remote_access can only be set when using the EKS managed node group default launch template. This module defaults to providing a custom launch template to allow for custom security groups, tag propagation, etc. If you wish to forgo the custom launch template route, you can set use_custom_launch_template = false and then you can set disk_size and remote_access.

"},{"location":"faq/#i-received-an-error-expect-exactly-one-securitygroup-tagged-with-kubernetesioclustername","title":"I received an error: expect exactly one securityGroup tagged with kubernetes.io/cluster/<NAME> ...","text":"

By default, EKS creates a cluster primary security group that is created outside of the module and the EKS service adds the tag { \"kubernetes.io/cluster/<CLUSTER_NAME>\" = \"owned\" }. This on its own does not cause any conflicts for addons such as the AWS Load Balancer Controller until users decide to attach both the cluster primary security group and the shared node security group created by the module (by setting attach_cluster_primary_security_group = true). The issue is not with having multiple security groups in your account with this tag key:value combination, but having multiple security groups with this tag key:value combination attached to nodes in the same cluster. There are a few ways to resolve this depending on your use case/intentions:

\u26a0\ufe0f <CLUSTER_NAME> below needs to be replaced with the name of your cluster

  1. If you want to use the cluster primary security group, you can disable the creation of the shared node security group with:
  create_node_security_group            = false # default is true\n  attach_cluster_primary_security_group = true # default is false\n
  1. By not attaching the cluster primary security group. The cluster primary security group has quite broad access and the module has instead provided a security group with the minimum amount of access to launch an empty EKS cluster successfully and users are encouraged to open up access when necessary to support their workload.
  attach_cluster_primary_security_group = false # this is the default for the module\n

In theory, if you are attaching the cluster primary security group, you shouldn't need to use the shared node security group created by the module. However, this is left up to users to decide for their requirements and use case.

If you choose to use Custom Networking, make sure to only attach the security groups matching your choice above in your ENIConfig resources. This will ensure you avoid redundant tags.

"},{"location":"faq/#why-are-nodes-not-being-registered","title":"Why are nodes not being registered?","text":"

Nodes not being able to register with the EKS control plane is generally due to networking mis-configurations.

  1. At least one of the cluster endpoints (public or private) must be enabled.

If you require a public endpoint, setting up both (public and private) and restricting the public endpoint via setting cluster_endpoint_public_access_cidrs is recommended. More info regarding communication with an endpoint is available here.

  1. Nodes need to be able to contact the EKS cluster endpoint. By default, the module only creates a public endpoint. To access the endpoint, the nodes need outgoing internet access:

  2. Nodes in private subnets: via a NAT gateway or instance along with the appropriate routing rules

  3. Nodes in public subnets: ensure that nodes are launched with public IPs (enable through either the module here or your subnet setting defaults)

Important: If you apply only the public endpoint and configure the cluster_endpoint_public_access_cidrs to restrict access, know that EKS nodes will also use the public endpoint and you must allow access to the endpoint. If not, then your nodes will fail to work correctly.

  1. The private endpoint can also be enabled by setting cluster_endpoint_private_access = true. Ensure that VPC DNS resolution and hostnames are also enabled for your VPC when the private endpoint is enabled.

  2. Nodes need to be able to connect to other AWS services to function (download container images, make API calls to assume roles, etc.). If for some reason you cannot enable public internet access for nodes you can add VPC endpoints to the relevant services: EC2 API, ECR API, ECR DKR and S3.

"},{"location":"faq/#why-are-there-no-changes-when-a-node-groups-desired_size-is-modified","title":"Why are there no changes when a node group's desired_size is modified?","text":"

The module is configured to ignore this value. Unfortunately, Terraform does not support variables within the lifecycle block. The setting is ignored to allow autoscaling via controllers such as cluster autoscaler or Karpenter to work properly and without interference by Terraform. Changing the desired count must be handled outside of Terraform once the node group is created.

"},{"location":"faq/#how-do-i-access-compute-resource-attributes","title":"How do I access compute resource attributes?","text":"

Examples of accessing the attributes of the compute resource(s) created by the root module are shown below. Note - the assumption is that your cluster module definition is named eks as in module \"eks\" { ... }:

eks_managed_role_arns = [for group in module.eks_managed_node_group : group.iam_role_arn]\n
self_managed_role_arns = [for group in module.self_managed_node_group : group.iam_role_arn]\n
fargate_profile_pod_execution_role_arns = [for group in module.fargate_profile : group.fargate_profile_pod_execution_role_arn]\n
"},{"location":"faq/#what-add-ons-are-available","title":"What add-ons are available?","text":"

The available EKS add-ons can be found here. You can also retrieve the available addons from the API using:

aws eks describe-addon-versions --query 'addons[*].addonName'\n
"},{"location":"faq/#what-configuration-values-are-available-for-an-add-on","title":"What configuration values are available for an add-on?","text":"

You can retrieve the configuration value schema for a given addon using the following command:

aws eks describe-addon-configuration --addon-name <value> --addon-version <value> --query 'configurationSchema' --output text | jq\n

For example:

aws eks describe-addon-configuration --addon-name coredns --addon-version v1.11.1-eksbuild.8 --query 'configurationSchema' --output text | jq\n

Returns (at the time of writing):

{\n  \"$ref\": \"#/definitions/Coredns\",\n  \"$schema\": \"http://json-schema.org/draft-06/schema#\",\n  \"definitions\": {\n    \"Coredns\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"affinity\": {\n          \"default\": {\n            \"affinity\": {\n              \"nodeAffinity\": {\n                \"requiredDuringSchedulingIgnoredDuringExecution\": {\n                  \"nodeSelectorTerms\": [\n                    {\n                      \"matchExpressions\": [\n                        {\n                          \"key\": \"kubernetes.io/os\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"linux\"\n                          ]\n                        },\n                        {\n                          \"key\": \"kubernetes.io/arch\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"amd64\",\n                            \"arm64\"\n                          ]\n                        }\n                      ]\n                    }\n                  ]\n                }\n              },\n              \"podAntiAffinity\": {\n                \"preferredDuringSchedulingIgnoredDuringExecution\": [\n                  {\n                    \"podAffinityTerm\": {\n                      \"labelSelector\": {\n                        \"matchExpressions\": [\n                          {\n                            \"key\": \"k8s-app\",\n                            \"operator\": \"In\",\n                            \"values\": [\n                              \"kube-dns\"\n                            ]\n                          }\n                        ]\n                      },\n                      \"topologyKey\": \"kubernetes.io/hostname\"\n                    },\n                    \"weight\": 100\n                  }\n                ]\n              }\n            }\n          },\n          \"description\": \"Affinity of the coredns pods\",\n          \"type\": [\n            \"object\",\n            \"null\"\n          ]\n        },\n        \"computeType\": {\n          \"type\": \"string\"\n        },\n        \"corefile\": {\n          \"description\": \"Entire corefile contents to use with installation\",\n          \"type\": \"string\"\n        },\n        \"nodeSelector\": {\n          \"additionalProperties\": {\n            \"type\": \"string\"\n          },\n          \"type\": \"object\"\n        },\n        \"podAnnotations\": {\n          \"properties\": {},\n          \"title\": \"The podAnnotations Schema\",\n          \"type\": \"object\"\n        },\n        \"podDisruptionBudget\": {\n          \"description\": \"podDisruptionBudget configurations\",\n          \"enabled\": {\n            \"default\": true,\n            \"description\": \"the option to enable managed PDB\",\n            \"type\": \"boolean\"\n          },\n          \"maxUnavailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"default\": 1,\n            \"description\": \"minAvailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"minAvailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"description\": \"maxUnavailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"type\": \"object\"\n        },\n        \"podLabels\": {\n          \"properties\": {},\n          \"title\": \"The podLabels Schema\",\n          \"type\": \"object\"\n        },\n        \"replicaCount\": {\n          \"type\": \"integer\"\n        },\n        \"resources\": {\n          \"$ref\": \"#/definitions/Resources\"\n        },\n        \"tolerations\": {\n          \"default\": [\n            {\n              \"key\": \"CriticalAddonsOnly\",\n              \"operator\": \"Exists\"\n            },\n            {\n              \"effect\": \"NoSchedule\",\n              \"key\": \"node-role.kubernetes.io/control-plane\"\n            }\n          ],\n          \"description\": \"Tolerations of the coredns pod\",\n          \"items\": {\n            \"type\": \"object\"\n          },\n          \"type\": \"array\"\n        },\n        \"topologySpreadConstraints\": {\n          \"description\": \"The coredns pod topology spread constraints\",\n          \"type\": \"array\"\n        }\n      },\n      \"title\": \"Coredns\",\n      \"type\": \"object\"\n    },\n    \"Limits\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"cpu\": {\n          \"type\": \"string\"\n        },\n        \"memory\": {\n          \"type\": \"string\"\n        }\n      },\n      \"title\": \"Limits\",\n      \"type\": \"object\"\n    },\n    \"Resources\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"limits\": {\n          \"$ref\": \"#/definitions/Limits\"\n        },\n        \"requests\": {\n          \"$ref\": \"#/definitions/Limits\"\n        }\n      },\n      \"title\": \"Resources\",\n      \"type\": \"object\"\n    }\n  }\n}\n

[!NOTE] The available configuration values will vary between add-on versions, typically more configuration values will be added in later versions as functionality is enabled by EKS.

"},{"location":"local/","title":"Local Development","text":""},{"location":"local/#documentation-site","title":"Documentation Site","text":"

In order to run the documentation site locally, you will need to have the following installed locally:

To run the documentation site locally, run the following command from the root of the repository:

mkdocs serve\n

Opening the documentation at the link posted in the terminal output (i.e. - http://127.0.0.1:8000/terraform-aws-eks/)

"},{"location":"network_connectivity/","title":"Network Connectivity","text":""},{"location":"network_connectivity/#cluster-endpoint","title":"Cluster Endpoint","text":""},{"location":"network_connectivity/#public-endpoint-w-restricted-cidrs","title":"Public Endpoint w/ Restricted CIDRs","text":"

When restricting the clusters public endpoint to only the CIDRs specified by users, it is recommended that you also enable the private endpoint, or ensure that the CIDR blocks that you specify include the addresses that nodes and Fargate pods (if you use them) access the public endpoint from.

Please refer to the AWS documentation for further information

"},{"location":"network_connectivity/#security-groups","title":"Security Groups","text":"

See the example snippet below which adds additional security group rules to the cluster security group as well as the shared node security group (for node-to-node access). Users can use this extensibility to open up network access as they see fit using the security groups provided by the module:

  ...\n  # Extend cluster security group rules\n  cluster_security_group_additional_rules = {\n    egress_nodes_ephemeral_ports_tcp = {\n      description                = \"To node 1025-65535\"\n      protocol                   = \"tcp\"\n      from_port                  = 1025\n      to_port                    = 65535\n      type                       = \"egress\"\n      source_node_security_group = true\n    }\n  }\n\n  # Extend node-to-node security group rules\n  node_security_group_additional_rules = {\n    ingress_self_all = {\n      description = \"Node to node all ports/protocols\"\n      protocol    = \"-1\"\n      from_port   = 0\n      to_port     = 0\n      type        = \"ingress\"\n      self        = true\n    }\n    egress_all = {\n      description      = \"Node all egress\"\n      protocol         = \"-1\"\n      from_port        = 0\n      to_port          = 0\n      type             = \"egress\"\n      cidr_blocks      = [\"0.0.0.0/0\"]\n      ipv6_cidr_blocks = [\"::/0\"]\n    }\n  }\n  ...\n
The security groups created by this module are depicted in the image shown below along with their default inbound/outbound rules:

"},{"location":"user_data/","title":"User Data & Bootstrapping","text":"

Users can see the various methods of using and providing user data through the user data tests as well more detailed information on the design and possible configurations via the user data module itself

"},{"location":"user_data/#summary","title":"Summary","text":"

The templates provided by the module can be found under the templates directory

"},{"location":"user_data/#eks-managed-node-group","title":"EKS Managed Node Group","text":"

When using an EKS managed node group, users have 2 primary routes for interacting with the bootstrap user data:

  1. If a value for ami_id is not provided, users can supply additional user data that is pre-pended before the EKS Managed Node Group bootstrap user data. You can read more about this process from the AWS supplied documentation

  2. Users can use the following variables to facilitate this process:

    pre_bootstrap_user_data = \"...\"\n

  3. If a custom AMI is used, then per the AWS documentation, users will need to supply the necessary user data to bootstrap and register nodes with the cluster when launched. There are two routes to facilitate this bootstrapping process:

  4. If the AMI used is a derivative of the AWS EKS Optimized AMI , users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:
  5. If the AMI is NOT an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the templatefile() for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.
\u2139\ufe0f When using bottlerocket, the supplied user data (TOML format) is merged in with the values supplied by EKS. Therefore, pre_bootstrap_user_data and post_bootstrap_user_data are not valid since the bottlerocket OS handles when various settings are applied. If you wish to supply additional configuration settings when using bottlerocket, supply them via the bootstrap_extra_args variable. For the AL2_* AMI types, bootstrap_extra_args are settings that will be supplied to the AWS EKS Optimized AMI bootstrap script such as kubelet extra args, etc. See the bottlerocket GitHub repository documentation for more details on what settings can be supplied via the bootstrap_extra_args variable."},{"location":"user_data/#self-managed-node-group","title":"Self Managed Node Group","text":"

Self managed node groups require users to provide the necessary bootstrap user data. Users can elect to use the user data template provided by the module for their respective AMI type or provide their own user data template for rendering by the module.

"},{"location":"user_data/#logic-diagram","title":"Logic Diagram","text":"

The rough flow of logic that is encapsulated within the _user_data module can be represented by the following diagram to better highlight the various manners in which user data can be populated.

"}]}