From 48fa419bba0abf4f24a17584abf055fc8852c292 Mon Sep 17 00:00:00 2001
From: GitHub Action <action@github.com>
Date: Tue, 22 Oct 2024 15:08:38 +0000
Subject: [PATCH] Deployed 4abc779 with MkDocs version: 1.6.1

---
 UPGRADE-18.0/index.html      |   2 +-
 compute_resources/index.html |   6 +++---
 search/search_index.json     |   2 +-
 sitemap.xml                  |  20 ++++++++++----------
 sitemap.xml.gz               | Bin 281 -> 281 bytes
 user_data/index.html         |   2 +-
 6 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/UPGRADE-18.0/index.html b/UPGRADE-18.0/index.html
index 9321ad57..aa80f7ae 100644
--- a/UPGRADE-18.0/index.html
+++ b/UPGRADE-18.0/index.html
@@ -525,7 +525,7 @@
 <li>The previous iteration used a count over a list of node group definitions which was prone to disruptive updates; this is now replaced with a map/for_each to align with that of the EKS managed node group and Fargate profile behaviors/style</li>
 </ul>
 </li>
-<li>The user data configuration supported across the module has been completely revamped. A new <code>_user_data</code> internal sub-module has been created to consolidate all user data configuration in one location which provides better support for testability (via the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/user_data"><code>examples/user_data</code></a> example). The new sub-module supports nearly all possible combinations including the ability to allow users to provide their own user data template which will be rendered by the module. See the <code>examples/user_data</code> example project for the full plethora of example configuration possibilities and more details on the logic of the design can be found in the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/modules/_user_data_"><code>modules/_user_data</code></a> directory.</li>
+<li>The user data configuration supported across the module has been completely revamped. A new <code>_user_data</code> internal sub-module has been created to consolidate all user data configuration in one location which provides better support for testability (via the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/tests/user-data"><code>tests/user-data</code></a> example). The new sub-module supports nearly all possible combinations including the ability to allow users to provide their own user data template which will be rendered by the module. See the <code>tests/user-data</code> example project for the full plethora of example configuration possibilities and more details on the logic of the design can be found in the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/modules/_user_data_"><code>modules/_user_data</code></a> directory.</li>
 <li>Resource name changes may cause issues with existing resources. For example, security groups and IAM roles cannot be renamed, they must be recreated. Recreation of these resources may also trigger a recreation of the cluster. To use the legacy (&lt; 18.x) resource naming convention, set <code>prefix_separator</code> to "".</li>
 <li>Security group usage has been overhauled to provide only the bare minimum network connectivity required to launch a bare bones cluster. See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks#security-groups">security group documentation section</a> for more details. Users upgrading to v18.x will want to review the rules they have in place today versus the rules provisioned by the v18.x module and ensure to make any necessary adjustments for their specific workload.</li>
 </ul>
diff --git a/compute_resources/index.html b/compute_resources/index.html
index 7fa57084..d58c910b 100644
--- a/compute_resources/index.html
+++ b/compute_resources/index.html
@@ -493,7 +493,7 @@
 </span><span id="__span-4-22"><a id="__codelineno-4-22" name="__codelineno-4-22" href="#__codelineno-4-22"></a><span class="w">    </span><span class="p">}</span>
 </span><span id="__span-4-23"><a id="__codelineno-4-23" name="__codelineno-4-23" href="#__codelineno-4-23"></a><span class="w">  </span><span class="p">}</span>
 </span></code></pre></div>
-<p>See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/eks_managed_node_group"><code>examples/eks_managed_node_group/</code> example</a> for a working example of various configurations.</p>
+<p>See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/eks-managed-node-group"><code>examples/eks-managed-node-group/</code> example</a> for a working example of various configurations.</p>
 <h3 id="self-managed-node-groups">Self Managed Node Groups<a class="headerlink" href="#self-managed-node-groups" title="Permanent link">&para;</a></h3>
 <p>Refer to the <a href="https://docs.aws.amazon.com/eks/latest/userguide/worker.html">Self Managed Node Group documentation</a> documentation for service related details.</p>
 <ol>
@@ -518,9 +518,9 @@
 </span><span id="__span-6-7"><a id="__codelineno-6-7" name="__codelineno-6-7" href="#__codelineno-6-7"></a><span class="w">    </span><span class="p">}</span>
 </span><span id="__span-6-8"><a id="__codelineno-6-8" name="__codelineno-6-8" href="#__codelineno-6-8"></a><span class="w">  </span><span class="p">}</span>
 </span></code></pre></div>
-<p>See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/self_managed_node_group"><code>examples/self_managed_node_group/</code> example</a> for a working example of various configurations.</p>
+<p>See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/self-managed-node-group"><code>examples/self-managed-node-group/</code> example</a> for a working example of various configurations.</p>
 <h3 id="fargate-profiles">Fargate Profiles<a class="headerlink" href="#fargate-profiles" title="Permanent link">&para;</a></h3>
-<p>Fargate profiles are straightforward to use and therefore no further details are provided here. See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/fargate_profile"><code>examples/fargate_profile/</code> example</a> for a working example of various configurations.</p>
+<p>Fargate profiles are straightforward to use and therefore no further details are provided here. See the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/tests/fargate-profile"><code>tests/fargate-profile/</code> tests</a> for a working example of various configurations.</p>
 <h3 id="default-configurations">Default Configurations<a class="headerlink" href="#default-configurations" title="Permanent link">&para;</a></h3>
 <p>Each type of compute resource (EKS managed node group, self managed node group, or Fargate profile) provides the option for users to specify a default configuration. These default configurations can be overridden from within the compute resource's individual definition. The order of precedence for configurations (from highest to least precedence):</p>
 <ul>
diff --git a/search/search_index.json b/search/search_index.json
index a8b88522..02509a4e 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Terraform AWS EKS module","text":"<p>Moar content coming soon!</p>"},{"location":"UPGRADE-17.0/","title":"How to handle the terraform-aws-eks module upgrade","text":""},{"location":"UPGRADE-17.0/#upgrade-module-to-v1700-for-managed-node-groups","title":"Upgrade module to v17.0.0 for Managed Node Groups","text":"<p>In this release, we now decided to remove random_pet resources in Managed Node Groups (MNG). Those were used to recreate MNG if something changed. But they were causing a lot of issues. To upgrade the module without recreating your MNG, you will need to explicitly reuse their previous name and set them in your MNG <code>name</code> argument.</p> <ol> <li>Run <code>terraform apply</code> with the module version v16.2.0</li> <li>Get your worker group names</li> </ol> <pre><code>~ terraform state show 'module.eks.module.node_groups.aws_eks_node_group.workers[\"example\"]' | grep node_group_name\nnode_group_name = \"test-eks-mwIwsvui-example-sincere-squid\"\n</code></pre> <ol> <li>Upgrade your module and configure your node groups to use existing names</li> </ol> <pre><code>module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"17.0.0\"\n\n  cluster_name    = \"test-eks-mwIwsvui\"\n  cluster_version = \"1.20\"\n  # ...\n\n  node_groups = {\n    example = {\n      name = \"test-eks-mwIwsvui-example-sincere-squid\"\n\n      # ...\n    }\n  }\n  # ...\n}\n</code></pre> <ol> <li>Run <code>terraform plan</code>, you should see that only <code>random_pets</code> will be destroyed</li> </ol> <pre><code>Terraform will perform the following actions:\n\n  # module.eks.module.node_groups.random_pet.node_groups[\"example\"] will be destroyed\n  - resource \"random_pet\" \"node_groups\" {\n      - id        = \"sincere-squid\" -&gt; null\n      - keepers   = {\n          - \"ami_type\"                  = \"AL2_x86_64\"\n          - \"capacity_type\"             = \"SPOT\"\n          - \"disk_size\"                 = \"50\"\n          - \"iam_role_arn\"              = \"arn:aws:iam::123456789123:role/test-eks-mwIwsvui20210527220853611600000009\"\n          - \"instance_types\"            = \"t3.large\"\n          - \"key_name\"                  = \"\"\n          - \"node_group_name\"           = \"test-eks-mwIwsvui-example\"\n          - \"source_security_group_ids\" = \"\"\n          - \"subnet_ids\"                = \"subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx\"\n        } -&gt; null\n      - length    = 2 -&gt; null\n      - separator = \"-\" -&gt; null\n    }\n\nPlan: 0 to add, 0 to change, 1 to destroy.\n</code></pre> <ol> <li>If everything sounds good to you, run <code>terraform apply</code></li> </ol> <p>After the first apply, we recommend you to create a new node group and let the module use the <code>node_group_name_prefix</code> (by removing the <code>name</code> argument) to generate names and avoid collision during node groups re-creation if needed, because the lifecycle is <code>create_before_destroy = true</code>.</p>"},{"location":"UPGRADE-18.0/","title":"Upgrade from v17.x to v18.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p> <p>Note: please see https://github.com/terraform-aws-modules/terraform-aws-eks/issues/1744 where users have shared the steps/changes that have worked for their configurations to upgrade. Due to the numerous configuration possibilities, it is difficult to capture specific steps that will work for all; this has proven to be a useful thread to share collective information from the broader community regarding v18.x upgrades.</p> <p>For most users, adding the following to your v17.x configuration will preserve the state of your cluster control plane when upgrading to v18.x:</p> <pre><code>prefix_separator                   = \"\"\niam_role_name                      = $CLUSTER_NAME\ncluster_security_group_name        = $CLUSTER_NAME\ncluster_security_group_description = \"EKS cluster security group.\"\n</code></pre> <p>This configuration assumes that <code>create_iam_role</code> is set to <code>true</code>, which is the default value.</p> <p>As the location of the Terraform state of the IAM role has been changed from 17.x to 18.x, you'll also have to move the state before running <code>terraform apply</code> by calling:</p> <pre><code>terraform state mv 'module.eks.aws_iam_role.cluster[0]' 'module.eks.aws_iam_role.this[0]'\n</code></pre> <p>See more information here</p>"},{"location":"UPGRADE-18.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>Launch configuration support has been removed and only launch template is supported going forward. AWS is no longer adding new features back into launch configuration and their docs state <code>We strongly recommend that you do not use launch configurations. They do not provide full functionality for Amazon EC2 Auto Scaling or Amazon EC2. We provide information about launch configurations for customers who have not yet migrated from launch configurations to launch templates.</code></li> <li>Support for managing aws-auth configmap has been removed. This change also removes the dependency on the Kubernetes Terraform provider, the local dependency on aws-iam-authenticator for users, as well as the reliance on the forked http provider to wait and poll on cluster creation. To aid users in this change, an output variable <code>aws_auth_configmap_yaml</code> has been provided which renders the aws-auth configmap necessary to support at least the IAM roles used by the module (additional mapRoles/mapUsers definitions to be provided by users)</li> <li>Support for managing kubeconfig and its associated <code>local_file</code> resources have been removed; users are able to use the awscli provided <code>aws eks update-kubeconfig --name &lt;cluster_name&gt;</code> to update their local kubeconfig as necessary</li> <li>The terminology used in the module has been modified to reflect that used by the AWS documentation.</li> <li>AWS EKS Managed Node Group, <code>eks_managed_node_groups</code>, was previously referred to as simply node group, <code>node_groups</code></li> <li>Self Managed Node Group Group, <code>self_managed_node_groups</code>, was previously referred to as worker group, <code>worker_groups</code></li> <li>AWS Fargate Profile, <code>fargate_profiles</code>, remains unchanged in terms of naming and terminology</li> <li>The three different node group types supported by AWS and the module have been refactored into standalone sub-modules that are both used by the root <code>eks</code> module as well as available for individual, standalone consumption if desired.</li> <li>The previous <code>node_groups</code> sub-module is now named <code>eks-managed-node-group</code> and provisions a single AWS EKS Managed Node Group per sub-module definition (previous version utilized <code>for_each</code> to create 0 or more node groups)<ul> <li>Additional changes for the <code>eks-managed-node-group</code> sub-module over the previous <code>node_groups</code> module include:</li> <li>Variable name changes defined in section <code>Variable and output changes</code> below</li> <li>Support for nearly full control of the IAM role created, or provide the ARN of an existing IAM role, has been added</li> <li>Support for nearly full control of the security group created, or provide the ID of an existing security group, has been added</li> <li>User data has been revamped and all user data logic moved to the <code>_user_data</code> internal sub-module; the local <code>userdata.sh.tpl</code> has been removed entirely</li> </ul> </li> <li>The previous <code>fargate</code> sub-module is now named <code>fargate-profile</code> and provisions a single AWS EKS Fargate Profile per sub-module definition (previous version utilized <code>for_each</code> to create 0 or more profiles)<ul> <li>Additional changes for the <code>fargate-profile</code> sub-module over the previous <code>fargate</code> module include:</li> <li>Variable name changes defined in section <code>Variable and output changes</code> below</li> <li>Support for nearly full control of the IAM role created, or provide the ARN of an existing IAM role, has been added</li> <li>Similar to the <code>eks_managed_node_group_defaults</code> and <code>self_managed_node_group_defaults</code>, a <code>fargate_profile_defaults</code> has been provided to allow users to control the default configurations for the Fargate profiles created</li> </ul> </li> <li>A sub-module for <code>self-managed-node-group</code> has been created and provisions a single self managed node group (autoscaling group) per sub-module definition<ul> <li>Additional changes for the <code>self-managed-node-group</code> sub-module over the previous <code>node_groups</code> variable include:</li> <li>The underlying autoscaling group and launch template have been updated to more closely match that of the <code>terraform-aws-autoscaling</code> module and the features it offers</li> <li>The previous iteration used a count over a list of node group definitions which was prone to disruptive updates; this is now replaced with a map/for_each to align with that of the EKS managed node group and Fargate profile behaviors/style</li> </ul> </li> <li>The user data configuration supported across the module has been completely revamped. A new <code>_user_data</code> internal sub-module has been created to consolidate all user data configuration in one location which provides better support for testability (via the <code>examples/user_data</code> example). The new sub-module supports nearly all possible combinations including the ability to allow users to provide their own user data template which will be rendered by the module. See the <code>examples/user_data</code> example project for the full plethora of example configuration possibilities and more details on the logic of the design can be found in the <code>modules/_user_data</code> directory.</li> <li>Resource name changes may cause issues with existing resources. For example, security groups and IAM roles cannot be renamed, they must be recreated. Recreation of these resources may also trigger a recreation of the cluster. To use the legacy (&lt; 18.x) resource naming convention, set <code>prefix_separator</code> to \"\".</li> <li>Security group usage has been overhauled to provide only the bare minimum network connectivity required to launch a bare bones cluster. See the security group documentation section for more details. Users upgrading to v18.x will want to review the rules they have in place today versus the rules provisioned by the v18.x module and ensure to make any necessary adjustments for their specific workload.</li> </ul>"},{"location":"UPGRADE-18.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-18.0/#added","title":"Added","text":"<ul> <li>Support for AWS EKS Addons has been added</li> <li>Support for AWS EKS Cluster Identity Provider Configuration has been added</li> <li>AWS Terraform provider minimum required version has been updated to 3.64 to support the changes made and additional resources supported</li> <li>An example <code>user_data</code> project has been added to aid in demonstrating, testing, and validating the various methods of configuring user data with the <code>_user_data</code> sub-module as well as the root <code>eks</code> module</li> <li>Template for rendering the aws-auth configmap output - <code>aws_auth_cm.tpl</code></li> <li>Template for Bottlerocket OS user data bootstrapping - <code>bottlerocket_user_data.tpl</code></li> </ul>"},{"location":"UPGRADE-18.0/#modified","title":"Modified","text":"<ul> <li>The previous <code>fargate</code> example has been renamed to <code>fargate_profile</code></li> <li>The previous <code>irsa</code> and <code>instance_refresh</code> examples have been merged into one example <code>irsa_autoscale_refresh</code></li> <li>The previous <code>managed_node_groups</code> example has been renamed to <code>self_managed_node_group</code></li> <li>The previously hardcoded EKS OIDC root CA thumbprint value and variable has been replaced with a <code>tls_certificate</code> data source that refers to the cluster OIDC issuer url. Thumbprint values should remain unchanged however</li> <li>Individual cluster security group resources have been replaced with a single security group resource that takes a map of rules as input. The default ingress/egress rules have had their scope reduced in order to provide the bare minimum of access to permit successful cluster creation and allow users to opt in to any additional network access as needed for a better security posture. This means the <code>0.0.0.0/0</code> egress rule has been removed, instead TCP/443 and TCP/10250 egress rules to the node group security group are used instead</li> <li>The Linux/bash user data template has been updated to include the bare minimum necessary for bootstrapping AWS EKS Optimized AMI derivative nodes with provisions for providing additional user data and configurations; was named <code>userdata.sh.tpl</code> and is now named <code>linux_user_data.tpl</code></li> <li>The Windows user data template has been renamed from <code>userdata_windows.tpl</code> to <code>windows_user_data.tpl</code></li> </ul>"},{"location":"UPGRADE-18.0/#removed","title":"Removed","text":"<ul> <li>Miscellaneous documents on how to configure Kubernetes cluster internals have been removed. Documentation related to how to configure the AWS EKS Cluster and its supported infrastructure resources provided by the module are supported, while cluster internal configuration is out of scope for this project</li> <li>The previous <code>bottlerocket</code> example has been removed in favor of demonstrating the use and configuration of Bottlerocket nodes via the respective <code>eks_managed_node_group</code> and <code>self_managed_node_group</code> examples</li> <li>The previous <code>launch_template</code> and <code>launch_templates_with_managed_node_groups</code> examples have been removed; only launch templates are now supported (default) and launch configuration support has been removed</li> <li>The previous <code>secrets_encryption</code> example has been removed; the functionality has been demonstrated in several of the new examples rendering this standalone example redundant</li> <li>The additional, custom IAM role policy for the cluster role has been removed. The permissions are either now provided in the attached managed AWS permission policies used or are no longer required</li> <li>The <code>kubeconfig.tpl</code> template; kubeconfig management is no longer supported under this module</li> <li>The HTTP Terraform provider (forked copy) dependency has been removed</li> </ul>"},{"location":"UPGRADE-18.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> <ul> <li><code>cluster_create_timeout</code>, <code>cluster_update_timeout</code>, and <code>cluster_delete_timeout</code> have been replaced with <code>cluster_timeouts</code></li> <li><code>kubeconfig_name</code></li> <li><code>kubeconfig_output_path</code></li> <li><code>kubeconfig_file_permission</code></li> <li><code>kubeconfig_api_version</code></li> <li><code>kubeconfig_aws_authenticator_command</code></li> <li><code>kubeconfig_aws_authenticator_command_args</code></li> <li><code>kubeconfig_aws_authenticator_additional_args</code></li> <li><code>kubeconfig_aws_authenticator_env_variables</code></li> <li><code>write_kubeconfig</code></li> <li><code>default_platform</code></li> <li><code>manage_aws_auth</code></li> <li><code>aws_auth_additional_labels</code></li> <li><code>map_accounts</code></li> <li><code>map_roles</code></li> <li><code>map_users</code></li> <li><code>fargate_subnets</code></li> <li><code>worker_groups_launch_template</code></li> <li><code>worker_security_group_id</code></li> <li><code>worker_ami_name_filter</code></li> <li><code>worker_ami_name_filter_windows</code></li> <li><code>worker_ami_owner_id</code></li> <li><code>worker_ami_owner_id_windows</code></li> <li><code>worker_additional_security_group_ids</code></li> <li><code>worker_sg_ingress_from_port</code></li> <li><code>workers_additional_policies</code></li> <li><code>worker_create_security_group</code></li> <li><code>worker_create_initial_lifecycle_hooks</code></li> <li><code>worker_create_cluster_primary_security_group_rules</code></li> <li><code>cluster_create_endpoint_private_access_sg_rule</code></li> <li><code>cluster_endpoint_private_access_cidrs</code></li> <li><code>cluster_endpoint_private_access_sg</code></li> <li><code>manage_worker_iam_resources</code></li> <li><code>workers_role_name</code></li> <li><code>attach_worker_cni_policy</code></li> <li><code>eks_oidc_root_ca_thumbprint</code></li> <li><code>create_fargate_pod_execution_role</code></li> <li><code>fargate_pod_execution_role_name</code></li> <li><code>cluster_egress_cidrs</code></li> <li><code>workers_egress_cidrs</code></li> <li><code>wait_for_cluster_timeout</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>default_iam_role_arn</code></li> <li><code>workers_group_defaults</code></li> <li><code>worker_security_group_id</code></li> <li><code>node_groups_defaults</code></li> <li><code>node_groups</code></li> <li><code>ebs_optimized_not_supported</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>create_eks</code> and <code>create_fargate_pod_execution_role</code> have been replaced with simply <code>create</code></li> </ul> </li> <li> <p>Renamed variables:</p> <ul> <li><code>create_eks</code> -&gt; <code>create</code></li> <li><code>subnets</code> -&gt; <code>subnet_ids</code></li> <li><code>cluster_create_security_group</code> -&gt; <code>create_cluster_security_group</code></li> <li><code>cluster_log_retention_in_days</code> -&gt; <code>cloudwatch_log_group_retention_in_days</code></li> <li><code>cluster_log_kms_key_id</code> -&gt; <code>cloudwatch_log_group_kms_key_id</code></li> <li><code>manage_cluster_iam_resources</code> -&gt; <code>create_iam_role</code></li> <li><code>cluster_iam_role_name</code> -&gt; <code>iam_role_name</code></li> <li><code>permissions_boundary</code> -&gt; <code>iam_role_permissions_boundary</code></li> <li><code>iam_path</code> -&gt; <code>iam_role_path</code></li> <li><code>pre_userdata</code> -&gt; <code>pre_bootstrap_user_data</code></li> <li><code>additional_userdata</code> -&gt; <code>post_bootstrap_user_data</code></li> <li><code>worker_groups</code> -&gt; <code>self_managed_node_groups</code></li> <li><code>workers_group_defaults</code> -&gt; <code>self_managed_node_group_defaults</code></li> <li><code>node_groups</code> -&gt; <code>eks_managed_node_groups</code></li> <li><code>node_groups_defaults</code> -&gt; <code>eks_managed_node_group_defaults</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>create_eks</code> -&gt; <code>create</code></li> <li><code>worker_additional_security_group_ids</code> -&gt; <code>vpc_security_group_ids</code></li> <li>Fargate profile sub-module</li> <li><code>fargate_pod_execution_role_name</code> -&gt; <code>name</code></li> <li><code>create_fargate_pod_execution_role</code> -&gt; <code>create_iam_role</code></li> <li><code>subnets</code> -&gt; <code>subnet_ids</code></li> <li><code>iam_path</code> -&gt; <code>iam_role_path</code></li> <li><code>permissions_boundary</code> -&gt; <code>iam_role_permissions_boundary</code></li> </ul> </li> <li> <p>Added variables:</p> <ul> <li><code>cluster_additional_security_group_ids</code> added to allow users to add additional security groups to the cluster as needed</li> <li><code>cluster_security_group_name</code></li> <li><code>cluster_security_group_use_name_prefix</code> added to allow users to use either the name as specified or default to using the name specified as a prefix</li> <li><code>cluster_security_group_description</code></li> <li><code>cluster_security_group_additional_rules</code></li> <li><code>cluster_security_group_tags</code></li> <li><code>create_cloudwatch_log_group</code> added in place of the logic that checked if any cluster log types were enabled to allow users to opt in as they see fit</li> <li><code>create_node_security_group</code> added to create single security group that connects node groups and cluster in central location</li> <li><code>node_security_group_id</code></li> <li><code>node_security_group_name</code></li> <li><code>node_security_group_use_name_prefix</code></li> <li><code>node_security_group_description</code></li> <li><code>node_security_group_additional_rules</code></li> <li><code>node_security_group_tags</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li><code>cluster_addons</code></li> <li><code>cluster_identity_providers</code></li> <li><code>fargate_profile_defaults</code></li> <li><code>prefix_separator</code> added to support legacy behavior of not having a prefix separator</li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>platform</code></li> <li><code>enable_bootstrap_user_data</code></li> <li><code>pre_bootstrap_user_data</code></li> <li><code>post_bootstrap_user_data</code></li> <li><code>bootstrap_extra_args</code></li> <li><code>user_data_template_path</code></li> <li><code>create_launch_template</code></li> <li><code>launch_template_name</code></li> <li><code>launch_template_use_name_prefix</code></li> <li><code>description</code></li> <li><code>ebs_optimized</code></li> <li><code>ami_id</code></li> <li><code>key_name</code></li> <li><code>launch_template_default_version</code></li> <li><code>update_launch_template_default_version</code></li> <li><code>disable_api_termination</code></li> <li><code>kernel_id</code></li> <li><code>ram_disk_id</code></li> <li><code>block_device_mappings</code></li> <li><code>capacity_reservation_specification</code></li> <li><code>cpu_options</code></li> <li><code>credit_specification</code></li> <li><code>elastic_gpu_specifications</code></li> <li><code>elastic_inference_accelerator</code></li> <li><code>enclave_options</code></li> <li><code>instance_market_options</code></li> <li><code>license_specifications</code></li> <li><code>metadata_options</code></li> <li><code>enable_monitoring</code></li> <li><code>network_interfaces</code></li> <li><code>placement</code></li> <li><code>min_size</code></li> <li><code>max_size</code></li> <li><code>desired_size</code></li> <li><code>use_name_prefix</code></li> <li><code>ami_type</code></li> <li><code>ami_release_version</code></li> <li><code>capacity_type</code></li> <li><code>disk_size</code></li> <li><code>force_update_version</code></li> <li><code>instance_types</code></li> <li><code>labels</code></li> <li><code>cluster_version</code></li> <li><code>launch_template_version</code></li> <li><code>remote_access</code></li> <li><code>taints</code></li> <li><code>update_config</code></li> <li><code>timeouts</code></li> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>vpc_id</code></li> <li><code>security_group_rules</code></li> <li><code>cluster_security_group_id</code></li> <li><code>security_group_tags</code></li> <li><code>create_iam_role</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_name</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_path</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_permissions_boundary</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>iam_role_arn</code> (for if <code>create_iam_role</code> is <code>false</code> to bring your own externally created role)</li> <li><code>iam_role_name</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li><code>selectors</code></li> <li><code>timeouts</code></li> </ul> </li> <li> <p>Removed outputs:</p> <ul> <li><code>cluster_version</code></li> <li><code>kubeconfig</code></li> <li><code>kubeconfig_filename</code></li> <li><code>workers_asg_arns</code></li> <li><code>workers_asg_names</code></li> <li><code>workers_user_data</code></li> <li><code>workers_default_ami_id</code></li> <li><code>workers_default_ami_id_windows</code></li> <li><code>workers_launch_template_ids</code></li> <li><code>workers_launch_template_arns</code></li> <li><code>workers_launch_template_latest_versions</code></li> <li><code>worker_security_group_id</code></li> <li><code>worker_iam_instance_profile_arns</code></li> <li><code>worker_iam_instance_profile_names</code></li> <li><code>worker_iam_role_name</code></li> <li><code>worker_iam_role_arn</code></li> <li><code>fargate_profile_ids</code></li> <li><code>fargate_profile_arns</code></li> <li><code>fargate_iam_role_name</code></li> <li><code>fargate_iam_role_arn</code></li> <li><code>node_groups</code></li> <li><code>security_group_rule_cluster_https_worker_ingress</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>node_groups</code></li> <li><code>aws_auth_roles</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>aws_auth_roles</code></li> </ul> </li> <li> <p>Renamed outputs:</p> <ul> <li><code>config_map_aws_auth</code> -&gt; <code>aws_auth_configmap_yaml</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>fargate_profile_ids</code> -&gt; <code>fargate_profile_id</code></li> <li><code>fargate_profile_arns</code> -&gt; <code>fargate_profile_arn</code></li> </ul> </li> <li> <p>Added outputs:</p> <ul> <li><code>cluster_platform_version</code></li> <li><code>cluster_status</code></li> <li><code>cluster_security_group_arn</code></li> <li><code>cluster_security_group_id</code></li> <li><code>node_security_group_arn</code></li> <li><code>node_security_group_id</code></li> <li><code>cluster_iam_role_unique_id</code></li> <li><code>cluster_addons</code></li> <li><code>cluster_identity_providers</code></li> <li><code>fargate_profiles</code></li> <li><code>eks_managed_node_groups</code></li> <li><code>self_managed_node_groups</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>launch_template_id</code></li> <li><code>launch_template_arn</code></li> <li><code>launch_template_latest_version</code></li> <li><code>node_group_arn</code></li> <li><code>node_group_id</code></li> <li><code>node_group_resources</code></li> <li><code>node_group_status</code></li> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> <li><code>iam_role_name</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_unique_id</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>iam_role_unique_id</code></li> <li><code>fargate_profile_status</code></li> </ul> </li> </ol>"},{"location":"UPGRADE-18.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-18.0/#before-17x-example","title":"Before 17.x Example","text":"<pre><code>module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~&gt; 17.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id  = module.vpc.vpc_id\n  subnets = module.vpc.private_subnets\n\n  # Managed Node Groups\n  node_groups_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  node_groups = {\n    node_group = {\n      min_capacity     = 1\n      max_capacity     = 10\n      desired_capacity = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      k8s_labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      additional_tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Worker groups\n  worker_additional_security_group_ids = [aws_security_group.additional.id]\n\n  worker_groups_launch_template = [\n    {\n      name                    = \"worker-group\"\n      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n      spot_instance_pools     = 4\n      asg_max_size            = 5\n      asg_desired_capacity    = 2\n      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n      public_ip               = true\n    },\n  ]\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#after-18x-example","title":"After 18.x Example","text":"<pre><code>module \"cluster_after\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~&gt; 18.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id     = module.vpc.vpc_id\n  subnet_ids = module.vpc.private_subnets\n\n  eks_managed_node_group_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  eks_managed_node_groups = {\n    node_group = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n  }\n\n  self_managed_node_groups = {\n    worker_group = {\n      name = \"worker-group\"\n\n      min_size      = 1\n      max_size      = 5\n      desired_size  = 2\n      instance_type = \"m4.large\"\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      block_device_mappings = {\n        xvda = {\n          device_name = \"/dev/xvda\"\n          ebs = {\n            delete_on_termination = true\n            encrypted             = false\n            volume_size           = 100\n            volume_type           = \"gp2\"\n          }\n\n        }\n      }\n\n      use_mixed_instances_policy = true\n      mixed_instances_policy = {\n        instances_distribution = {\n          spot_instance_pools = 4\n        }\n\n        override = [\n          { instance_type = \"m5.large\" },\n          { instance_type = \"m5a.large\" },\n          { instance_type = \"m5d.large\" },\n          { instance_type = \"m5ad.large\" },\n        ]\n      }\n    }\n  }\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#diff-of-before-after","title":"Diff of before &lt;&gt; after","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 17.0\"\n+  version = \"~&gt; 18.0\"\n\n   cluster_name                    = local.name\n   cluster_version                 = local.cluster_version\n   cluster_endpoint_private_access = true\n   cluster_endpoint_public_access  = true\n\n   vpc_id  = module.vpc.vpc_id\n-  subnets = module.vpc.private_subnets\n+  subnet_ids = module.vpc.private_subnets\n\n-  # Managed Node Groups\n-  node_groups_defaults = {\n+  eks_managed_node_group_defaults = {\n     ami_type  = \"AL2_x86_64\"\n     disk_size = 50\n   }\n\n-  node_groups = {\n+  eks_managed_node_groups = {\n     node_group = {\n-      min_capacity     = 1\n-      max_capacity     = 10\n-      desired_capacity = 1\n+      min_size     = 1\n+      max_size     = 10\n+      desired_size = 1\n\n       instance_types = [\"t3.large\"]\n       capacity_type  = \"SPOT\"\n\n       update_config = {\n         max_unavailable_percentage = 50\n       }\n\n-      k8s_labels = {\n+      labels = {\n         Environment = \"test\"\n         GithubRepo  = \"terraform-aws-eks\"\n         GithubOrg   = \"terraform-aws-modules\"\n       }\n\n       taints = [\n         {\n           key    = \"dedicated\"\n           value  = \"gpuGroup\"\n           effect = \"NO_SCHEDULE\"\n         }\n       ]\n\n-      additional_tags = {\n+      tags = {\n         ExtraTag = \"example\"\n       }\n     }\n   }\n\n-  # Worker groups\n-  worker_additional_security_group_ids = [aws_security_group.additional.id]\n-\n-  worker_groups_launch_template = [\n-    {\n-      name                    = \"worker-group\"\n-      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n-      spot_instance_pools     = 4\n-      asg_max_size            = 5\n-      asg_desired_capacity    = 2\n-      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n-      public_ip               = true\n-    },\n-  ]\n+  self_managed_node_group_defaults = {\n+    vpc_security_group_ids = [aws_security_group.additional.id]\n+  }\n+\n+  self_managed_node_groups = {\n+    worker_group = {\n+      name = \"worker-group\"\n+\n+      min_size      = 1\n+      max_size      = 5\n+      desired_size  = 2\n+      instance_type = \"m4.large\"\n+\n+      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n+\n+      block_device_mappings = {\n+        xvda = {\n+          device_name = \"/dev/xvda\"\n+          ebs = {\n+            delete_on_termination = true\n+            encrypted             = false\n+            volume_size           = 100\n+            volume_type           = \"gp2\"\n+          }\n+\n+        }\n+      }\n+\n+      use_mixed_instances_policy = true\n+      mixed_instances_policy = {\n+        instances_distribution = {\n+          spot_instance_pools = 4\n+        }\n+\n+        override = [\n+          { instance_type = \"m5.large\" },\n+          { instance_type = \"m5a.large\" },\n+          { instance_type = \"m5d.large\" },\n+          { instance_type = \"m5ad.large\" },\n+        ]\n+      }\n+    }\n+  }\n\n   # Fargate\n   fargate_profiles = {\n     default = {\n       name = \"default\"\n       selectors = [\n         {\n           namespace = \"kube-system\"\n           labels = {\n             k8s-app = \"kube-dns\"\n           }\n         },\n         {\n           namespace = \"default\"\n         }\n       ]\n\n       tags = {\n         Owner = \"test\"\n       }\n\n       timeouts = {\n         create = \"20m\"\n         delete = \"20m\"\n       }\n     }\n   }\n\n   tags = {\n     Environment = \"test\"\n     GithubRepo  = \"terraform-aws-eks\"\n     GithubOrg   = \"terraform-aws-modules\"\n   }\n }\n</code></pre>"},{"location":"UPGRADE-18.0/#attaching-an-iam-role-policy-to-a-fargate-profile","title":"Attaching an IAM role policy to a Fargate profile","text":""},{"location":"UPGRADE-18.0/#before-17x","title":"Before 17.x","text":"<pre><code>resource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#after-18x","title":"After 18.x","text":"<pre><code># Attach the policy to an \"example\" Fargate profile\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_profiles[\"example\"].iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre> <p>Or:</p> <pre><code># Attach the policy to all Fargate profiles\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  for_each = module.eks.fargate_profiles\n\n  role       = each.value.iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre>"},{"location":"UPGRADE-19.0/","title":"Upgrade from v18.x to v19.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p>"},{"location":"UPGRADE-19.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>The <code>cluster_id</code> output used to output the name of the cluster. This is due to the fact that the cluster name is a unique constraint and therefore its set as the unique identifier within Terraform's state map. However, starting with local EKS clusters created on Outposts, there is now an attribute returned from the <code>aws eks create-cluster</code> API named <code>id</code>. The <code>cluster_id</code> has been updated to return this value which means that for current, standard EKS clusters created in the AWS cloud, no value will be returned (at the time of this writing) for <code>cluster_id</code> and only local EKS clusters on Outposts will return a value that looks like a UUID/GUID. Users should switch all instances of <code>cluster_id</code> to use <code>cluster_name</code> before upgrading to v19. Reference</li> <li>Minimum supported version of Terraform AWS provider updated to v4.45 to support the latest features provided via the resources utilized.</li> <li>Minimum supported version of Terraform updated to v1.0</li> <li>Individual security group created per EKS managed node group or self-managed node group has been removed. This configuration went mostly unused and would often cause confusion (\"Why is there an empty security group attached to my nodes?\"). This functionality can easily be replicated by user's providing one or more externally created security groups to attach to nodes launched from the node group.</li> <li>Previously, <code>var.iam_role_additional_policies</code> (one for each of the following: cluster IAM role, EKS managed node group IAM role, self-managed node group IAM role, and Fargate Profile IAM role) accepted a list of strings. This worked well for policies that already existed but failed for policies being created at the same time as the cluster due to the well-known issue of unknown values used in a <code>for_each</code> loop. To rectify this issue in <code>v19.x</code>, two changes were made:</li> <li><code>var.iam_role_additional_policies</code> was changed from type <code>list(string)</code> to type <code>map(string)</code> -&gt; this is a breaking change. More information on managing this change can be found below, under <code>Terraform State Moves</code></li> <li>The logic used in the root module for this variable was changed to replace the use of <code>try()</code> with <code>lookup()</code>. More details on why can be found here</li> <li>The cluster name has been removed from the Karpenter module event rule names. Due to the use of long cluster names appending to the provided naming scheme, the cluster name has moved to a <code>ClusterName</code> tag and the event rule name is now a prefix. This guarantees that users can have multiple instances of Karpenter with their respective event rules/SQS queue without name collisions, while also still being able to identify which queues and event rules belong to which cluster.</li> <li>The new variable <code>node_security_group_enable_recommended_rules</code> is set to true by default and may conflict with any custom ingress/egress rules. Please ensure that any duplicates from the <code>node_security_group_additional_rules</code> are removed before upgrading, or set <code>node_security_group_enable_recommended_rules</code> to false. Reference</li> </ul>"},{"location":"UPGRADE-19.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-19.0/#added","title":"Added","text":"<ul> <li>Support for setting <code>preserve</code> as well as <code>most_recent</code> on addons.</li> <li><code>preserve</code> indicates if you want to preserve the created resources when deleting the EKS add-on</li> <li><code>most_recent</code> indicates if you want to use the most recent revision of the add-on or the default version (default)</li> <li>Support for setting default node security group rules for common access patterns required:</li> <li>Egress all for <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Ingress from cluster security group for 8443/TCP and 9443/TCP for common applications such as ALB Ingress Controller, Karpenter, OPA Gatekeeper, etc. These are commonly used as webhook ports for validating and mutating webhooks</li> </ul>"},{"location":"UPGRADE-19.0/#modified","title":"Modified","text":"<ul> <li><code>cluster_security_group_additional_rules</code> and <code>node_security_group_additional_rules</code> have been modified to use <code>lookup()</code> instead of <code>try()</code> to avoid the well-known issue of unknown values within a <code>for_each</code> loop</li> <li>Default cluster security group rules have removed egress rules for TCP/443 and TCP/10250 to node groups since the cluster primary security group includes a default rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Default node security group rules have removed egress rules have been removed since the default security group settings have egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li><code>block_device_mappings</code> previously required a map of maps but has since changed to an array of maps. Users can remove the outer key for each block device mapping and replace the outermost map <code>{}</code> with an array <code>[]</code>. There are no state changes required for this change.</li> <li><code>create_kms_key</code> previously defaulted to <code>false</code> and now defaults to <code>true</code>. Clusters created with this module now default to enabling secret encryption by default with a customer-managed KMS key created by this module</li> <li><code>cluster_encryption_config</code> previously used a type of <code>list(any)</code> and now uses a type of <code>any</code> -&gt; users can simply remove the outer <code>[</code>...<code>]</code> brackets on <code>v19.x</code></li> <li><code>cluster_encryption_config</code> previously defaulted to <code>[]</code> and now defaults to <code>{resources = [\"secrets\"]}</code> to encrypt secrets by default</li> <li><code>cluster_endpoint_public_access</code> previously defaulted to <code>true</code> and now defaults to <code>false</code>. Clusters created with this module now default to private-only access to the cluster endpoint</li> <li><code>cluster_endpoint_private_access</code> previously defaulted to <code>false</code> and now defaults to <code>true</code></li> <li>The addon configuration now sets <code>\"OVERWRITE\"</code> as the default value for <code>resolve_conflicts</code> to ease add-on upgrade management. Users can opt out of this by instead setting <code>\"NONE\"</code> as the value for <code>resolve_conflicts</code></li> <li>The <code>kms</code> module used has been updated from <code>v1.0.2</code> to <code>v1.1.0</code> - no material changes other than updated to latest</li> <li>The default value for EKS managed node group <code>update_config</code> has been updated to the recommended <code>{ max_unavailable_percentage = 33 }</code></li> <li>The default value for the self-managed node group <code>instance_refresh</code> has been updated to the recommended:     <pre><code>{\n  strategy = \"Rolling\"\n  preferences = {\n    min_healthy_percentage = 66\n  }\n}\n</code></pre></li> </ul>"},{"location":"UPGRADE-19.0/#removed","title":"Removed","text":"<ul> <li>Remove all references of <code>aws_default_tags</code> to avoid update conflicts; this is the responsibility of the provider and should be handled at the provider level</li> <li>https://github.com/terraform-aws-modules/terraform-aws-eks/issues?q=is%3Aissue+default_tags+is%3Aclosed</li> <li>https://github.com/terraform-aws-modules/terraform-aws-eks/pulls?q=is%3Apr+default_tags+is%3Aclosed</li> </ul>"},{"location":"UPGRADE-19.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> </li> <li> <p><code>node_security_group_ntp_ipv4_cidr_block</code> - default security group settings have an egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></p> </li> <li><code>node_security_group_ntp_ipv6_cidr_block</code> - default security group settings have an egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Self-managed node groups:<ul> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>security_group_rules</code></li> <li><code>security_group_tags</code></li> <li><code>cluster_security_group_id</code></li> <li><code>vpc_id</code></li> </ul> </li> <li> <p>EKS managed node groups:</p> <ul> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>security_group_rules</code></li> <li><code>security_group_tags</code></li> <li><code>cluster_security_group_id</code></li> <li><code>vpc_id</code></li> </ul> </li> <li> <p>Renamed variables:</p> </li> <li> <p>N/A</p> </li> <li> <p>Added variables:</p> </li> <li> <p><code>provision_on_outpost</code>for Outposts support</p> </li> <li><code>outpost_config</code> for Outposts support</li> <li><code>cluster_addons_timeouts</code> for setting a common set of timeouts for all addons (unless a specific value is provided within the addon configuration)</li> <li><code>service_ipv6_cidr</code> for setting the IPv6 CIDR block for the Kubernetes service addresses</li> <li> <p><code>node_security_group_enable_recommended_rules</code> for enabling recommended node security group rules for common access patterns</p> </li> <li> <p>Self-managed node groups:</p> <ul> <li><code>launch_template_id</code> for use when using an existing/externally created launch template (Ref: https://github.com/terraform-aws-modules/terraform-aws-autoscaling/pull/204)</li> <li><code>maintenance_options</code></li> <li><code>private_dns_name_options</code></li> <li><code>instance_requirements</code></li> <li><code>context</code></li> <li><code>default_instance_warmup</code></li> <li><code>force_delete_warm_pool</code></li> </ul> </li> <li>EKS managed node groups:<ul> <li><code>use_custom_launch_template</code> was added to better clarify how users can switch between a custom launch template or the default launch template provided by the EKS managed node group. Previously, to achieve this same functionality of using the default launch template, users needed to set <code>create_launch_template = false</code> and <code>launch_template_name = \"\"</code> which is not very intuitive.</li> <li><code>launch_template_id</code> for use when using an existing/externally created launch template (Ref: https://github.com/terraform-aws-modules/terraform-aws-autoscaling/pull/204)</li> <li><code>maintenance_options</code></li> <li><code>private_dns_name_options</code>  -</li> </ul> </li> <li> <p>Removed outputs:</p> </li> <li> <p>Self-managed node groups:</p> <ul> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> </ul> </li> <li> <p>EKS managed node groups:</p> <ul> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> </ul> </li> <li> <p>Renamed outputs:</p> </li> <li> <p><code>cluster_id</code> is not renamed but the value it returns is now different. For standard EKS clusters created in the AWS cloud, the value returned at the time of this writing is <code>null</code>/empty. For local EKS clusters created on Outposts, the value returned will look like a UUID/GUID. Users should switch all instances of <code>cluster_id</code> to use <code>cluster_name</code> before upgrading to v19. Reference</p> </li> <li> <p>Added outputs:</p> </li> <li> <p><code>cluster_name</code> - The <code>cluster_id</code> currently set by the AWS provider is actually the cluster name, but in the future, this will change and there will be a distinction between the <code>cluster_name</code> and <code>cluster_id</code>. Reference</p> </li> </ol>"},{"location":"UPGRADE-19.0/#upgrade-migrations","title":"Upgrade Migrations","text":"<ol> <li>Before upgrading your module definition to <code>v19.x</code>, please see below for both EKS managed node group(s) and self-managed node groups and remove the node group(s) security group prior to upgrading.</li> </ol>"},{"location":"UPGRADE-19.0/#self-managed-node-groups","title":"Self-Managed Node Groups","text":"<p>Self-managed node groups on <code>v18.x</code> by default create a security group that does not specify any rules. In <code>v19.x</code>, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version <code>v18.x</code> of your module definition, remove this security group from your node groups by setting <code>create_security_group = false</code>.</p> <ul> <li>If you are currently utilizing this security group, it is recommended to create an additional security group that matches the rules/settings of the security group created by the node group, and specify that security group ID in <code>vpc_security_group_ids</code>. Once this is in place, you can proceed with the original security group removal.</li> <li>For most users, the security group is not used and can be safely removed. However, deployed instances will have the security group attached to nodes and require the security group to be disassociated before the security group can be deleted. Because instances are deployed via autoscaling groups, we cannot simply remove the security group from the code and have those changes reflected on the instances. Instead, we have to update the code and then trigger the autoscaling groups to cycle the instances deployed so that new instances are provisioned without the security group attached. You can utilize the <code>instance_refresh</code> parameter of Autoscaling groups to force nodes to re-deploy when removing the security group since changes to launch templates automatically trigger an instance refresh. An example configuration is provided below.</li> <li>Add the following to either/or <code>self_managed_node_group_defaults</code> or the individual self-managed node group definitions:     <pre><code>create_security_group = false\ninstance_refresh = {\n  strategy = \"Rolling\"\n  preferences = {\n    min_healthy_percentage = 66\n  }\n}\n</code></pre></li> <li>It is recommended to use the <code>aws-node-termination-handler</code> while performing this update. Please refer to the <code>irsa-autoscale-refresh</code> example for usage. This will ensure that pods are safely evicted in a controlled manner to avoid service disruptions.</li> <li>Once the necessary configurations are in place, you can apply the changes which will:</li> <li>Create a new launch template (version) without the self-managed node group security group</li> <li>Replace instances based on the <code>instance_refresh</code> configuration settings</li> <li>New instances will launch without the self-managed node group security group, and prior instances will be terminated</li> <li>Once the self-managed node group has cycled, the security group will be deleted</li> </ul>"},{"location":"UPGRADE-19.0/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"<p>EKS managed node groups on <code>v18.x</code> by default create a security group that does not specify any rules. In <code>v19.x</code>, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version <code>v18.x</code> of your module definition, remove this security group from your node groups by setting <code>create_security_group = false</code>.</p> <ul> <li>If you are currently utilizing this security group, it is recommended to create an additional security group that matches the rules/settings of the security group created by the node group, and specify that security group ID in <code>vpc_security_group_ids</code>. Once this is in place, you can proceed with the original security group removal.</li> <li>EKS managed node groups rollout changes using a rolling update strategy that can be influenced through <code>update_config</code>. No additional changes are required for removing the security group created by node groups (unlike self-managed node groups which should utilize the <code>instance_refresh</code> setting of Autoscaling groups).</li> <li>Once <code>create_security_group = false</code> has been set, you can apply the changes which will:</li> <li>Create a new launch template (version) without the EKS managed node group security group</li> <li>Replace instances based on the <code>update_config</code> configuration settings</li> <li>New instances will launch without the EKS managed node group security group, and prior instances will be terminated</li> <li> <p>Once the EKS managed node group has cycled, the security group will be deleted</p> </li> <li> <p>Once the node group security group(s) have been removed, you can update your module definition to specify the <code>v19.x</code> version of the module</p> </li> <li>Run <code>terraform init -upgrade=true</code> to update your configuration and pull in the v19 changes</li> <li>Using the documentation provided above, update your module definition to reflect the changes in the module from <code>v18.x</code> to <code>v19.x</code>. You can utilize <code>terraform plan</code> as you go to help highlight any changes that you wish to make. See below for <code>terraform state mv ...</code> commands related to the use of <code>iam_role_additional_policies</code>. If you are not providing any values to these variables, you can skip this section.</li> <li>Once you are satisfied with the changes and the <code>terraform plan</code> output, you can apply the changes to sync your infrastructure with the updated module definition (or vice versa).</li> </ul>"},{"location":"UPGRADE-19.0/#diff-of-before-v18x-vs-after-v19x","title":"Diff of Before (v18.x) vs After (v19.x)","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 18.0\"\n+  version = \"~&gt; 19.0\"\n\n  cluster_name                    = local.name\n+ cluster_endpoint_public_access  = true\n- cluster_endpoint_private_access = true # now the default\n\n  cluster_addons = {\n-   resolve_conflicts = \"OVERWRITE\" # now the default\n+   preserve          = true\n+   most_recent       = true\n\n+   timeouts = {\n+     create = \"25m\"\n+     delete = \"10m\"\n    }\n    kube-proxy = {}\n    vpc-cni = {\n-     resolve_conflicts = \"OVERWRITE\" # now the default\n    }\n  }\n\n  # Encryption key\n  create_kms_key = true\n- cluster_encryption_config = [{\n-   resources = [\"secrets\"]\n- }]\n+ cluster_encryption_config = {\n+   resources = [\"secrets\"]\n+ }\n  kms_key_deletion_window_in_days = 7\n  enable_kms_key_rotation         = true\n\n- iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+ iam_role_additional_policies = {\n+   additional = aws_iam_policy.additional.arn\n+ }\n\n  vpc_id                   = module.vpc.vpc_id\n  subnet_ids               = module.vpc.private_subnets\n  control_plane_subnet_ids = module.vpc.intra_subnets\n\n  # Extend node-to-node security group rules\n- node_security_group_ntp_ipv4_cidr_block = [\"169.254.169.123/32\"] # now the default\n  node_security_group_additional_rules = {\n-    ingress_self_ephemeral = {\n-      description = \"Node to node ephemeral ports\"\n-      protocol    = \"tcp\"\n-      from_port   = 0\n-      to_port     = 0\n-      type        = \"ingress\"\n-      self        = true\n-    }\n-    egress_all = {\n-      description      = \"Node all egress\"\n-      protocol         = \"-1\"\n-      from_port        = 0\n-      to_port          = 0\n-      type             = \"egress\"\n-      cidr_blocks      = [\"0.0.0.0/0\"]\n-      ipv6_cidr_blocks = [\"::/0\"]\n-    }\n  }\n\n  # Self-Managed Node Group(s)\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  self_managed_node_groups = {\n    spot = {\n      instance_type = \"m5.large\"\n      instance_market_options = {\n        market_type = \"spot\"\n      }\n\n      pre_bootstrap_user_data = &lt;&lt;-EOT\n        echo \"foo\"\n        export FOO=bar\n      EOT\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      post_bootstrap_user_data = &lt;&lt;-EOT\n        cd /tmp\n        sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm\n        sudo systemctl enable amazon-ssm-agent\n        sudo systemctl start amazon-ssm-agent\n      EOT\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n    }\n  }\n\n  # EKS Managed Node Group(s)\n  eks_managed_node_group_defaults = {\n    ami_type       = \"AL2_x86_64\"\n    instance_types = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n\n    attach_cluster_primary_security_group = true\n    vpc_security_group_ids                = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  eks_managed_node_groups = {\n    blue = {}\n    green = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = {\n        dedicated = {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      }\n\n      update_config = {\n        max_unavailable_percentage = 33 # or set `max_unavailable`\n      }\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Fargate Profile(s)\n  fargate_profile_defaults = {\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  # OIDC Identity provider\n  cluster_identity_providers = {\n    cognito = {\n      client_id      = \"702vqsrjicklgb7c5b7b50i1gc\"\n      issuer_url     = \"https://cognito-idp.us-west-2.amazonaws.com/us-west-2_re1u6bpRA\"\n      username_claim = \"email\"\n      groups_claim   = \"cognito:groups\"\n      groups_prefix  = \"gid:\"\n    }\n  }\n\n  # aws-auth configmap\n  manage_aws_auth_configmap = true\n\n  aws_auth_node_iam_role_arns_non_windows = [\n    module.eks_managed_node_group.iam_role_arn,\n    module.self_managed_node_group.iam_role_arn,\n  ]\n  aws_auth_fargate_profile_pod_execution_role_arns = [\n    module.fargate_profile.fargate_profile_pod_execution_role_arn\n  ]\n\n  aws_auth_roles = [\n    {\n      rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n      username = \"role1\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_users = [\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user1\"\n      username = \"user1\"\n      groups   = [\"system:masters\"]\n    },\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user2\"\n      username = \"user2\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_accounts = [\n    \"777777777777\",\n    \"888888888888\",\n  ]\n\n  tags = local.tags\n}\n</code></pre>"},{"location":"UPGRADE-19.0/#terraform-state-moves","title":"Terraform State Moves","text":"<p>The following Terraform state move commands are optional but recommended if you are providing additional IAM policies that are to be attached to IAM roles created by this module (cluster IAM role, node group IAM role, Fargate profile IAM role). Because the resources affected are <code>aws_iam_role_policy_attachment</code>, in theory, you could get away with simply applying the configuration and letting Terraform detach and re-attach the policies. However, during this brief period of update, you could experience permission failures as the policy is detached and re-attached, and therefore the state move route is recommended.</p> <p>Where <code>\"&lt;POLICY_ARN&gt;\"</code> is specified, this should be replaced with the full ARN of the policy, and <code>\"&lt;POLICY_MAP_KEY&gt;\"</code> should be replaced with the key used in the <code>iam_role_additional_policies</code> map for the associated policy. For example, if you have the following<code>v19.x</code> configuration:</p> <pre><code>  ...\n  # This is demonstrating the cluster IAM role additional policies\n  iam_role_additional_policies = {\n    additional = aws_iam_policy.additional.arn\n  }\n  ...\n</code></pre> <p>The associated state move command would look similar to (albeit with your correct policy ARN):</p> <pre><code>terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"arn:aws:iam::111111111111:policy/ex-complete-additional\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"additional\"]'\n</code></pre> <p>If you are not providing any additional IAM policies, no actions are required.</p>"},{"location":"UPGRADE-19.0/#cluster-iam-role","title":"Cluster IAM Role","text":"<p>Repeat for each policy provided in <code>iam_role_additional_policies</code>:</p> <pre><code>terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#eks-managed-node-group-iam-role","title":"EKS Managed Node Group IAM Role","text":"<p>Where <code>\"&lt;NODE_GROUP_KEY&gt;\"</code> is the key used in the <code>eks_managed_node_groups</code> map for the associated node group. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>eks_managed_node_group_defaults</code> or the individual node group definitions:</p> <pre><code>terraform state mv 'module.eks.module.eks_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.eks_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#self-managed-node-group-iam-role","title":"Self-Managed Node Group IAM Role","text":"<p>Where <code>\"&lt;NODE_GROUP_KEY&gt;\"</code> is the key used in the <code>self_managed_node_groups</code> map for the associated node group. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>self_managed_node_group_defaults</code> or the individual node group definitions:</p> <pre><code>terraform state mv 'module.eks.module.self_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.self_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#fargate-profile-iam-role","title":"Fargate Profile IAM Role","text":"<p>Where <code>\"&lt;FARGATE_PROFILE_KEY&gt;\"</code> is the key used in the <code>fargate_profiles</code> map for the associated profile. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>fargate_profile_defaults</code> or the individual profile definitions:</p> <pre><code>terraform state mv 'module.eks.module.fargate_profile[\"&lt;FARGATE_PROFILE_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.fargate_profile[\"&lt;FARGATE_PROFILE_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-20.0/","title":"Upgrade from v19.x to v20.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p>"},{"location":"UPGRADE-20.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>Minium supported AWS provider version increased to <code>v5.34</code></li> <li>Minimum supported Terraform version increased to <code>v1.3</code> to support Terraform state <code>moved</code> blocks as well as other advanced features</li> <li>The <code>resolve_conflicts</code> argument within the <code>cluster_addons</code> configuration has been replaced with <code>resolve_conflicts_on_create</code> and <code>resolve_conflicts_on_update</code> now that <code>resolve_conflicts</code> is deprecated</li> <li>The default/fallback value for the <code>preserve</code> argument of <code>cluster_addons</code>is now set to <code>true</code>. This has shown to be useful for users deprovisioning clusters while avoiding the situation where the CNI is deleted too early and causes resources to be left orphaned resulting in conflicts.</li> <li>The Karpenter sub-module's use of the <code>irsa</code> naming convention has been removed, along with an update to the Karpenter controller IAM policy to align with Karpenter's <code>v1beta1</code>/<code>v0.32</code> changes. Instead of referring to the role as <code>irsa</code> or <code>pod_identity</code>, its simply just an IAM role used by the Karpenter controller and there is support for use with either IRSA and/or Pod Identity (default) at this time</li> <li>The <code>aws-auth</code> ConfigMap resources have been moved to a standalone sub-module. This removes the Kubernetes provider requirement from the main module and allows for the <code>aws-auth</code> ConfigMap to be managed independently of the main module. This sub-module will be removed entirely in the next major release.</li> <li>Support for cluster access management has been added with the default authentication mode set as <code>API_AND_CONFIG_MAP</code>. This is a one way change if applied; if you wish to use <code>CONFIG_MAP</code>, you will need to set <code>authentication_mode = \"CONFIG_MAP\"</code> explicitly when upgrading.</li> <li>Karpenter EventBridge rule key <code>spot_interrupt</code> updated to correct mis-spelling (was <code>spot_interupt</code>). This will cause the rule to be replaced</li> </ul>"},{"location":"UPGRADE-20.0/#upcoming-changes-planned-in-v210","title":"\u26a0\ufe0f Upcoming Changes Planned in v21.0 \u26a0\ufe0f","text":"<p>To give users advanced notice and provide some future direction for this module, these are the following changes we will be looking to make in the next major release of this module:</p> <ol> <li>The <code>aws-auth</code> sub-module will be removed entirely from the project. Since this sub-module is captured in the v20.x releases, users can continue using it even after the module moves forward with the next major version. The long term strategy and direction is cluster access entry and to rely only on the AWS Terraform provider.</li> <li>The default value for <code>authentication_mode</code> will change to <code>API</code>. Aligning with point 1 above, this is a one way change, but users are free to specify the value of their choosing in place of this default (when the change is made). This module will proceed with an EKS API first strategy.</li> <li>The launch template and autoscaling group usage contained within the EKS managed node group and self-managed node group sub-modules *might be replaced with the <code>terraform-aws-autoscaling</code> module. At minimum, it makes sense to replace most of functionality in the self-managed node group module with this external module, but its not yet clear if there is any benefit of using it in the EKS managed node group sub-module. The interface that users interact with will stay the same, the changes will be internal to the implementation and we will do everything we can to keep the disruption to a minimum.</li> <li>The <code>platform</code> variable will be replaced and instead <code>ami_type</code> will become the standard across both self-managed node group(s) and EKS managed node group(s). As EKS expands its portfolio of supported operating systems, the <code>ami_type</code> is better suited to associate the correct user data format to the respective OS. The <code>platform</code> variable is a legacy artifact of self-managed node groups but not as descriptive as the <code>ami_type</code>, and therefore it will be removed in favor of <code>ami_type</code>.</li> </ol>"},{"location":"UPGRADE-20.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-20.0/#added","title":"Added","text":"<ul> <li>A module tag has been added to the cluster control plane</li> <li>Support for cluster access entries. The <code>bootstrap_cluster_creator_admin_permissions</code> setting on the control plane has been hardcoded to <code>false</code> since this operation is a one time operation only at cluster creation per the EKS API. Instead, users can enable/disable <code>enable_cluster_creator_admin_permissions</code> at any time to achieve the same functionality. This takes the identity that Terraform is using to make API calls and maps it into a cluster admin via an access entry. For users on existing clusters, you will need to remove the default cluster administrator that was created by EKS prior to the cluster access entry APIs - see the section <code>Removing the default cluster administrator</code> for more details.</li> <li>Support for specifying the CloudWatch log group class (standard or infrequent access)</li> <li>Native support for Windows based managed node groups similar to AL2 and Bottlerocket</li> <li>Self-managed node groups now support <code>instance_maintenance_policy</code> and have added <code>max_healthy_percentage</code>, <code>scale_in_protected_instances</code>, and <code>standby_instances</code> arguments to the <code>instance_refresh.preferences</code> block</li> </ul>"},{"location":"UPGRADE-20.0/#modified","title":"Modified","text":"<ul> <li>For <code>sts:AssumeRole</code> permissions by services, the use of dynamically looking up the DNS suffix has been replaced with the static value of <code>amazonaws.com</code>. This does not appear to change by partition and instead requires users to set this manually for non-commercial regions.</li> <li>The default value for <code>kms_key_enable_default_policy</code> has changed from <code>false</code> to <code>true</code> to align with the default behavior of the <code>aws_kms_key</code> resource</li> <li>The Karpenter default value for <code>create_instance_profile</code> has changed from <code>true</code> to <code>false</code> to align with the changes in Karpenter v0.32</li> <li>The Karpenter variable <code>create_instance_profile</code> default value has changed from <code>true</code> to <code>false</code>. Starting with Karpenter <code>v0.32.0</code>, Karpenter accepts an IAM role and creates the EC2 instance profile used by the nodes</li> </ul>"},{"location":"UPGRADE-20.0/#removed","title":"Removed","text":"<ul> <li>The <code>complete</code> example has been removed due to its redundancy with the other examples</li> <li>References to the IRSA sub-module in the IAM repository have been removed. Once https://github.com/clowdhaus/terraform-aws-eks-pod-identity has been updated and moved into the organization, the documentation here will be updated to mention the new module.</li> </ul>"},{"location":"UPGRADE-20.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> </li> <li> <p><code>cluster_iam_role_dns_suffix</code> - replaced with a static string of <code>amazonaws.com</code></p> </li> <li><code>manage_aws_auth_configmap</code></li> <li><code>create_aws_auth_configmap</code></li> <li><code>aws_auth_node_iam_role_arns_non_windows</code></li> <li><code>aws_auth_node_iam_role_arns_windows</code></li> <li><code>aws_auth_fargate_profile_pod_execution_role_arn</code></li> <li><code>aws_auth_roles</code></li> <li><code>aws_auth_users</code></li> <li> <p><code>aws_auth_accounts</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>irsa_tag_key</code></li> <li><code>irsa_tag_values</code></li> <li><code>irsa_subnet_account_id</code></li> <li><code>enable_karpenter_instance_profile_creation</code></li> </ul> </li> <li> <p>Renamed variables:</p> </li> <li> <p>Karpenter</p> <ul> <li><code>create_irsa</code> -&gt; <code>create_iam_role</code></li> <li><code>irsa_name</code> -&gt; <code>iam_role_name</code></li> <li><code>irsa_use_name_prefix</code> -&gt; <code>iam_role_name_prefix</code></li> <li><code>irsa_path</code> -&gt; <code>iam_role_path</code></li> <li><code>irsa_description</code> -&gt; <code>iam_role_description</code></li> <li><code>irsa_max_session_duration</code> -&gt; <code>iam_role_max_session_duration</code></li> <li><code>irsa_permissions_boundary_arn</code> -&gt; <code>iam_role_permissions_boundary_arn</code></li> <li><code>irsa_tags</code> -&gt; <code>iam_role_tags</code></li> <li><code>policies</code> -&gt; <code>iam_role_policies</code></li> <li><code>irsa_policy_name</code> -&gt; <code>iam_policy_name</code></li> <li><code>irsa_ssm_parameter_arns</code> -&gt; <code>ami_id_ssm_parameter_arns</code></li> <li><code>create_iam_role</code> -&gt; <code>create_node_iam_role</code></li> <li><code>iam_role_additional_policies</code> -&gt; <code>node_iam_role_additional_policies</code></li> <li><code>policies</code> -&gt; <code>iam_role_policies</code></li> <li><code>iam_role_arn</code> -&gt; <code>node_iam_role_arn</code></li> <li><code>iam_role_name</code> -&gt; <code>node_iam_role_name</code></li> <li><code>iam_role_name_prefix</code> -&gt; <code>node_iam_role_name_prefix</code></li> <li><code>iam_role_path</code> -&gt; <code>node_iam_role_path</code></li> <li><code>iam_role_description</code> -&gt; <code>node_iam_role_description</code></li> <li><code>iam_role_max_session_duration</code> -&gt; <code>node_iam_role_max_session_duration</code></li> <li><code>iam_role_permissions_boundary_arn</code> -&gt; <code>node_iam_role_permissions_boundary_arn</code></li> <li><code>iam_role_attach_cni_policy</code> -&gt; <code>node_iam_role_attach_cni_policy</code></li> <li><code>iam_role_additional_policies</code> -&gt; <code>node_iam_role_additional_policies</code></li> <li><code>iam_role_tags</code> -&gt; <code>node_iam_role_tags</code></li> </ul> </li> <li> <p>Added variables:</p> </li> <li> <p><code>create_access_entry</code></p> </li> <li><code>enable_cluster_creator_admin_permissions</code></li> <li><code>authentication_mode</code></li> <li><code>access_entries</code></li> <li> <p><code>cloudwatch_log_group_class</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>iam_policy_name</code></li> <li><code>iam_policy_use_name_prefix</code></li> <li><code>iam_policy_description</code></li> <li><code>iam_policy_path</code></li> <li><code>enable_irsa</code></li> <li><code>create_access_entry</code></li> <li><code>access_entry_type</code></li> </ul> </li> <li> <p>Self-managed node group</p> <ul> <li><code>instance_maintenance_policy</code></li> <li><code>create_access_entry</code></li> <li><code>iam_role_arn</code></li> </ul> </li> <li> <p>Removed outputs:</p> </li> <li> <p><code>aws_auth_configmap_yaml</code></p> </li> <li> <p>Renamed outputs:</p> </li> <li> <p>Karpenter</p> <ul> <li><code>irsa_name</code> -&gt; <code>iam_role_name</code></li> <li><code>irsa_arn</code> -&gt; <code>iam_role_arn</code></li> <li><code>irsa_unique_id</code> -&gt; <code>iam_role_unique_id</code></li> <li><code>role_name</code> -&gt; <code>node_iam_role_name</code></li> <li><code>role_arn</code> -&gt; <code>node_iam_role_arn</code></li> <li><code>role_unique_id</code> -&gt; <code>node_iam_role_unique_id</code></li> </ul> </li> <li> <p>Added outputs:</p> </li> <li> <p><code>access_entries</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>node_access_entry_arn</code></li> </ul> </li> <li> <p>Self-managed node group</p> <ul> <li><code>access_entry_arn</code></li> </ul> </li> </ol>"},{"location":"UPGRADE-20.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-20.0/#diff-of-before-v1921-vs-after-v200","title":"Diff of Before (v19.21) vs After (v20.0)","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 19.21\"\n+  version = \"~&gt; 20.0\"\n\n# If you want to maintain the current default behavior of v19.x\n+  kms_key_enable_default_policy = false\n\n-   manage_aws_auth_configmap = true\n\n-   aws_auth_roles = [\n-     {\n-       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n-       username = \"role1\"\n-       groups   = [\"custom-role-group\"]\n-     },\n-   ]\n\n-   aws_auth_users = [\n-     {\n-       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n-       username = \"user1\"\n-       groups   = [\"custom-users-group\"]\n-     },\n-   ]\n}\n\n+ module \"eks_aws_auth\" {\n+   source  = \"terraform-aws-modules/eks/aws//modules/aws-auth\"\n+   version = \"~&gt; 20.0\"\n\n+   manage_aws_auth_configmap = true\n\n+   aws_auth_roles = [\n+     {\n+       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n+       username = \"role1\"\n+       groups   = [\"custom-role-group\"]\n+     },\n+   ]\n\n+   aws_auth_users = [\n+     {\n+       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n+       username = \"user1\"\n+       groups   = [\"custom-users-group\"]\n+     },\n+   ]\n+ }\n</code></pre>"},{"location":"UPGRADE-20.0/#karpenter-diff-of-before-v1921-vs-after-v200","title":"Karpenter Diff of Before (v19.21) vs After (v20.0)","text":"<pre><code> module \"eks_karpenter\" {\n   source  = \"terraform-aws-modules/eks/aws//modules/karpenter\"\n-  version = \"~&gt; 19.21\"\n+  version = \"~&gt; 20.0\"\n\n# If you wish to maintain the current default behavior of v19.x\n+  enable_irsa             = true\n+  create_instance_profile = true\n\n# To avoid any resource re-creation\n+  iam_role_name          = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_role_description   = \"Karpenter IAM role for service account\"\n+  iam_policy_name        = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_policy_description = \"Karpenter IAM role for service account\"\n}\n</code></pre>"},{"location":"UPGRADE-20.0/#terraform-state-moves","title":"Terraform State Moves","text":""},{"location":"UPGRADE-20.0/#authentication-mode-changes","title":"\u26a0\ufe0f Authentication Mode Changes \u26a0\ufe0f","text":"<p>Changing the <code>authentication_mode</code> is a one-way decision. See announcement blog for further details:</p> <p>Switching authentication modes on an existing cluster is a one-way operation. You can switch from CONFIG_MAP to API_AND_CONFIG_MAP. You can then switch from API_AND_CONFIG_MAP to API. You cannot revert these operations in the opposite direction. Meaning you cannot switch back to CONFIG_MAP or API_AND_CONFIG_MAP from API. And you cannot switch back to CONFIG_MAP from API_AND_CONFIG_MAP.</p> <p>[!IMPORTANT] If migrating to cluster access entries and you will NOT have any entries that remain in the <code>aws-auth</code> ConfigMap, you do not need to remove the configmap from the statefile. You can simply follow the migration guide and once access entries have been created, you can let Terraform remove/delete the <code>aws-auth</code> ConfigMap.</p> <p>If you WILL have entries that remain in the <code>aws-auth</code> ConfigMap, then you will need to remove the ConfigMap resources from the statefile to avoid any disruptions. When you add the new <code>aws-auth</code> sub-module and apply the changes, the sub-module will upsert the ConfigMap on the cluster. Provided the necessary entries are defined in that sub-module's definition, it will \"re-adopt\" the ConfigMap under Terraform's control.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-config_map","title":"authentication_mode = \"CONFIG_MAP\"","text":"<p>If using <code>authentication_mode = \"CONFIG_MAP\"</code>, before making any changes, you will first need to remove the configmap from the statefile to avoid any disruptions:</p> <pre><code>terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n</code></pre> <p>Once the configmap has been removed from the statefile, you can add the new <code>aws-auth</code> sub-module and copy the relevant definitions from the EKS module over to the new <code>aws-auth</code> sub-module definition (see before after diff above).</p> <p>[!CAUTION] You will need to add entries to the <code>aws-auth</code> sub-module for any IAM roles used by node groups and/or Fargate profiles - the module no longer handles this in the background on behalf of users.</p> <p>When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-api_and_config_map","title":"authentication_mode = \"API_AND_CONFIG_MAP\"","text":"<p>When using <code>authentication_mode = \"API_AND_CONFIG_MAP\"</code> and there are entries that will remain in the configmap (entries that cannot be replaced by cluster access entry), you will first need to update the <code>authentication_mode</code> on the cluster to <code>\"API_AND_CONFIG_MAP\"</code>. To help make this upgrade process easier, a copy of the changes defined in the <code>v20.0.0</code> PR have been captured here but with the <code>aws-auth</code> components still provided in the module. This means you get the equivalent of the <code>v20.0.0</code> module, but it still includes support for the <code>aws-auth</code> configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the <code>authentication_mode</code> has been updated to <code>\"API_AND_CONFIG_MAP\"</code>. Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.</p> <p>Once the <code>authentication_mode</code> has been updated, next you will need to remove the configmap from the statefile to avoid any disruptions:</p> <p>[!NOTE] This is only required if there are entries that will remain in the <code>aws-auth</code> ConfigMap after migrating. Otherwise, you can skip this step and let Terraform destroy the ConfigMap.</p> <pre><code>terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n</code></pre>"},{"location":"UPGRADE-20.0/#i-terraform-17-users","title":"\u2139\ufe0f Terraform 1.7+ users","text":"<p>If you are using Terraform <code>v1.7+</code>, you can utilize the <code>remove</code> to facilitate both the removal of the configmap through code. You can create a fork/clone of the provided migration module and add the <code>remove</code> blocks and apply those changes before proceeding. We do not want to force users onto the bleeding edge with this module, so we have not included <code>remove</code> support at this time.</p> <p>Once the configmap has been removed from the statefile, you can add the new <code>aws-auth</code> sub-module and copy the relevant definitions from the EKS module over to the new <code>aws-auth</code> sub-module definition (see before after diff above). When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes. In the before/example above - the configmap would remove any entries for roles used by node groups and/or Fargate Profiles, but maintain the custom entries for users and roles passed into the module definition.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-api","title":"authentication_mode = \"API\"","text":"<p>In order to switch to <code>API</code> only using cluster access entry, you first need to update the <code>authentication_mode</code> on the cluster to <code>API_AND_CONFIG_MAP</code> without modifying the <code>aws-auth</code> configmap. To help make this upgrade process easier, a copy of the changes defined in the <code>v20.0.0</code> PR have been captured here but with the <code>aws-auth</code> components still provided in the module. This means you get the equivalent of the <code>v20.0.0</code> module, but it still includes support for the <code>aws-auth</code> configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the <code>authentication_mode</code> has been updated to <code>\"API_AND_CONFIG_MAP\"</code>. Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.</p> <p>Once the <code>authentication_mode</code> has been updated, you can update the <code>authentication_mode</code> on the cluster to <code>API</code> and remove the <code>aws-auth</code> configmap components.</p>"},{"location":"compute_resources/","title":"Compute Resources","text":""},{"location":"compute_resources/#table-of-contents","title":"Table of Contents","text":"<ul> <li>EKS Managed Node Groups</li> <li>Self Managed Node Groups</li> <li>Fargate Profiles</li> <li>Default Configurations</li> </ul> <p>\u2139\ufe0f Only the pertinent attributes are shown below for brevity</p>"},{"location":"compute_resources/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"<p>Refer to the EKS Managed Node Group documentation documentation for service related details.</p> <ol> <li>The module creates a custom launch template by default to ensure settings such as tags are propagated to instances. Please note that many of the customization options listed here are only available when a custom launch template is created. To use the default template provided by the AWS EKS managed node group service, disable the launch template creation by setting <code>use_custom_launch_template</code> to <code>false</code>:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    default = {\n      use_custom_launch_template = false\n    }\n  }\n</code></pre> <ol> <li>Native support for Bottlerocket OS is provided by providing the respective AMI type:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_default = {\n      use_custom_launch_template = false\n\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre> <ol> <li>Bottlerocket OS is supported in a similar manner. However, note that the user data for Bottlerocket OS uses the TOML format:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_prepend_userdata = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      bootstrap_extra_args = &lt;&lt;-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n      EOT\n    }\n  }\n</code></pre> <ol> <li>When using a custom AMI, the AWS EKS Managed Node Group service will NOT inject the necessary bootstrap script into the supplied user data. Users can elect to provide their own user data to bootstrap and connect or opt in to use the module provided user data:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    custom_ami = {\n      ami_id = \"ami-0caf35bc73450c396\"\n\n      # By default, EKS managed node groups will not append bootstrap script;\n      # this adds it back in using the default template provided by the module\n      # Note: this assumes the AMI provided is an EKS optimized AMI derivative\n      enable_bootstrap_user_data = true\n\n      pre_bootstrap_user_data = &lt;&lt;-EOT\n        export FOO=bar\n      EOT\n\n      # Because we have full control over the user data supplied, we can also run additional\n      # scripts/configuration changes after the bootstrap script has been run\n      post_bootstrap_user_data = &lt;&lt;-EOT\n        echo \"you are free little kubelet!\"\n      EOT\n    }\n  }\n</code></pre> <ol> <li>There is similar support for Bottlerocket OS:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_custom_ami = {\n      ami_id   = \"ami-0ff61e0bcfc81dc94\"\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      # use module user data template to bootstrap\n      enable_bootstrap_user_data = true\n      # this will get added to the template\n      bootstrap_extra_args = &lt;&lt;-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n\n        [settings.kubernetes.node-labels]\n        \"label1\" = \"foo\"\n        \"label2\" = \"bar\"\n\n        [settings.kubernetes.node-taints]\n        \"dedicated\" = \"experimental:PreferNoSchedule\"\n        \"special\" = \"true:NoSchedule\"\n      EOT\n    }\n  }\n</code></pre> <p>See the <code>examples/eks_managed_node_group/</code> example for a working example of various configurations.</p>"},{"location":"compute_resources/#self-managed-node-groups","title":"Self Managed Node Groups","text":"<p>Refer to the Self Managed Node Group documentation documentation for service related details.</p> <ol> <li>The <code>self-managed-node-group</code> uses the latest AWS EKS Optimized AMI (Linux) for the given Kubernetes version by default:</li> </ol> <pre><code>  cluster_version = \"1.31\"\n\n  # This self managed node group will use the latest AWS EKS Optimized AMI for Kubernetes 1.27\n  self_managed_node_groups = {\n    default = {}\n  }\n</code></pre> <ol> <li>To use Bottlerocket, specify the <code>ami_type</code> as one of the respective <code>\"BOTTLEROCKET_*\" types</code> and supply a Bottlerocket OS AMI:</li> </ol> <pre><code>  cluster_version = \"1.31\"\n\n  self_managed_node_groups = {\n    bottlerocket = {\n      ami_id   = data.aws_ami.bottlerocket_ami.id\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre> <p>See the <code>examples/self_managed_node_group/</code> example for a working example of various configurations.</p>"},{"location":"compute_resources/#fargate-profiles","title":"Fargate Profiles","text":"<p>Fargate profiles are straightforward to use and therefore no further details are provided here. See the <code>examples/fargate_profile/</code> example for a working example of various configurations.</p>"},{"location":"compute_resources/#default-configurations","title":"Default Configurations","text":"<p>Each type of compute resource (EKS managed node group, self managed node group, or Fargate profile) provides the option for users to specify a default configuration. These default configurations can be overridden from within the compute resource's individual definition. The order of precedence for configurations (from highest to least precedence):</p> <ul> <li>Compute resource individual configuration</li> <li>Compute resource family default configuration (<code>eks_managed_node_group_defaults</code>, <code>self_managed_node_group_defaults</code>, <code>fargate_profile_defaults</code>)<ul> <li>Module default configuration (see <code>variables.tf</code> and <code>node_groups.tf</code>)</li> </ul> </li> </ul> <p>For example, the following creates 4 AWS EKS Managed Node Groups:</p> <pre><code>  eks_managed_node_group_defaults = {\n    ami_type               = \"AL2_x86_64\"\n    disk_size              = 50\n    instance_types         = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n  }\n\n  eks_managed_node_groups = {\n    # Uses module default configurations overridden by configuration above\n    default = {}\n\n    # This further overrides the instance types used\n    compute = {\n      instance_types = [\"c5.large\", \"c6i.large\", \"c6d.large\"]\n    }\n\n    # This further overrides the instance types and disk size used\n    persistent = {\n      disk_size = 1024\n      instance_types = [\"r5.xlarge\", \"r6i.xlarge\", \"r5b.xlarge\"]\n    }\n\n    # This overrides the OS used\n    bottlerocket = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre>"},{"location":"faq/","title":"Frequently Asked Questions","text":"<ul> <li>Setting <code>disk_size</code> or <code>remote_access</code> does not make any changes</li> <li>I received an error: <code>expect exactly one securityGroup tagged with kubernetes.io/cluster/&lt;NAME&gt; ...</code></li> <li>Why are nodes not being registered?</li> <li>Why are there no changes when a node group's <code>desired_size</code> is modified?</li> <li>How do I access compute resource attributes?</li> <li>What add-ons are available?</li> <li>What configuration values are available for an add-on?</li> </ul>"},{"location":"faq/#setting-disk_size-or-remote_access-does-not-make-any-changes","title":"Setting <code>disk_size</code> or <code>remote_access</code> does not make any changes","text":"<p><code>disk_size</code>, and <code>remote_access</code> can only be set when using the EKS managed node group default launch template. This module defaults to providing a custom launch template to allow for custom security groups, tag propagation, etc. If you wish to forgo the custom launch template route, you can set <code>use_custom_launch_template = false</code> and then you can set <code>disk_size</code> and <code>remote_access</code>.</p>"},{"location":"faq/#i-received-an-error-expect-exactly-one-securitygroup-tagged-with-kubernetesioclustername","title":"I received an error: <code>expect exactly one securityGroup tagged with kubernetes.io/cluster/&lt;NAME&gt; ...</code>","text":"<p>By default, EKS creates a cluster primary security group that is created outside of the module and the EKS service adds the tag <code>{ \"kubernetes.io/cluster/&lt;CLUSTER_NAME&gt;\" = \"owned\" }</code>. This on its own does not cause any conflicts for addons such as the AWS Load Balancer Controller until users decide to attach both the cluster primary security group and the shared node security group created by the module (by setting <code>attach_cluster_primary_security_group = true</code>). The issue is not with having multiple security groups in your account with this tag key:value combination, but having multiple security groups with this tag key:value combination attached to nodes in the same cluster. There are a few ways to resolve this depending on your use case/intentions:</p> <p>\u26a0\ufe0f <code>&lt;CLUSTER_NAME&gt;</code> below needs to be replaced with the name of your cluster</p> <ol> <li>If you want to use the cluster primary security group, you can disable the creation of the shared node security group with:</li> </ol> <pre><code>  create_node_security_group            = false # default is true\n  attach_cluster_primary_security_group = true # default is false\n</code></pre> <ol> <li>By not attaching the cluster primary security group. The cluster primary security group has quite broad access and the module has instead provided a security group with the minimum amount of access to launch an empty EKS cluster successfully and users are encouraged to open up access when necessary to support their workload.</li> </ol> <pre><code>  attach_cluster_primary_security_group = false # this is the default for the module\n</code></pre> <p>In theory, if you are attaching the cluster primary security group, you shouldn't need to use the shared node security group created by the module. However, this is left up to users to decide for their requirements and use case.</p> <p>If you choose to use Custom Networking, make sure to only attach the security groups matching your choice above in your ENIConfig resources. This will ensure you avoid redundant tags.</p>"},{"location":"faq/#why-are-nodes-not-being-registered","title":"Why are nodes not being registered?","text":"<p>Nodes not being able to register with the EKS control plane is generally due to networking mis-configurations.</p> <ol> <li>At least one of the cluster endpoints (public or private) must be enabled.</li> </ol> <p>If you require a public endpoint, setting up both (public and private) and restricting the public endpoint via setting <code>cluster_endpoint_public_access_cidrs</code> is recommended. More info regarding communication with an endpoint is available here.</p> <ol> <li> <p>Nodes need to be able to contact the EKS cluster endpoint. By default, the module only creates a public endpoint. To access the endpoint, the nodes need outgoing internet access:</p> </li> <li> <p>Nodes in private subnets: via a NAT gateway or instance along with the appropriate routing rules</p> </li> <li>Nodes in public subnets: ensure that nodes are launched with public IPs (enable through either the module here or your subnet setting defaults)</li> </ol> <p>Important: If you apply only the public endpoint and configure the <code>cluster_endpoint_public_access_cidrs</code> to restrict access, know that EKS nodes will also use the public endpoint and you must allow access to the endpoint. If not, then your nodes will fail to work correctly.</p> <ol> <li> <p>The private endpoint can also be enabled by setting <code>cluster_endpoint_private_access = true</code>. Ensure that VPC DNS resolution and hostnames are also enabled for your VPC when the private endpoint is enabled.</p> </li> <li> <p>Nodes need to be able to connect to other AWS services to function (download container images, make API calls to assume roles, etc.). If for some reason you cannot enable public internet access for nodes you can add VPC endpoints to the relevant services: EC2 API, ECR API, ECR DKR and S3.</p> </li> </ol>"},{"location":"faq/#why-are-there-no-changes-when-a-node-groups-desired_size-is-modified","title":"Why are there no changes when a node group's <code>desired_size</code> is modified?","text":"<p>The module is configured to ignore this value. Unfortunately, Terraform does not support variables within the <code>lifecycle</code> block. The setting is ignored to allow autoscaling via controllers such as cluster autoscaler or Karpenter to work properly and without interference by Terraform. Changing the desired count must be handled outside of Terraform once the node group is created.</p>"},{"location":"faq/#how-do-i-access-compute-resource-attributes","title":"How do I access compute resource attributes?","text":"<p>Examples of accessing the attributes of the compute resource(s) created by the root module are shown below. Note - the assumption is that your cluster module definition is named <code>eks</code> as in <code>module \"eks\" { ... }</code>:</p> <ul> <li>EKS Managed Node Group attributes</li> </ul> <pre><code>eks_managed_role_arns = [for group in module.eks_managed_node_group : group.iam_role_arn]\n</code></pre> <ul> <li>Self Managed Node Group attributes</li> </ul> <pre><code>self_managed_role_arns = [for group in module.self_managed_node_group : group.iam_role_arn]\n</code></pre> <ul> <li>Fargate Profile attributes</li> </ul> <pre><code>fargate_profile_pod_execution_role_arns = [for group in module.fargate_profile : group.fargate_profile_pod_execution_role_arn]\n</code></pre>"},{"location":"faq/#what-add-ons-are-available","title":"What add-ons are available?","text":"<p>The available EKS add-ons can be found here. You can also retrieve the available addons from the API using:</p> <pre><code>aws eks describe-addon-versions --query 'addons[*].addonName'\n</code></pre>"},{"location":"faq/#what-configuration-values-are-available-for-an-add-on","title":"What configuration values are available for an add-on?","text":"<p>You can retrieve the configuration value schema for a given addon using the following command:</p> <pre><code>aws eks describe-addon-configuration --addon-name &lt;value&gt; --addon-version &lt;value&gt; --query 'configurationSchema' --output text | jq\n</code></pre> <p>For example:</p> <pre><code>aws eks describe-addon-configuration --addon-name coredns --addon-version v1.11.1-eksbuild.8 --query 'configurationSchema' --output text | jq\n</code></pre> <p>Returns (at the time of writing):</p> <pre><code>{\n  \"$ref\": \"#/definitions/Coredns\",\n  \"$schema\": \"http://json-schema.org/draft-06/schema#\",\n  \"definitions\": {\n    \"Coredns\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"affinity\": {\n          \"default\": {\n            \"affinity\": {\n              \"nodeAffinity\": {\n                \"requiredDuringSchedulingIgnoredDuringExecution\": {\n                  \"nodeSelectorTerms\": [\n                    {\n                      \"matchExpressions\": [\n                        {\n                          \"key\": \"kubernetes.io/os\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"linux\"\n                          ]\n                        },\n                        {\n                          \"key\": \"kubernetes.io/arch\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"amd64\",\n                            \"arm64\"\n                          ]\n                        }\n                      ]\n                    }\n                  ]\n                }\n              },\n              \"podAntiAffinity\": {\n                \"preferredDuringSchedulingIgnoredDuringExecution\": [\n                  {\n                    \"podAffinityTerm\": {\n                      \"labelSelector\": {\n                        \"matchExpressions\": [\n                          {\n                            \"key\": \"k8s-app\",\n                            \"operator\": \"In\",\n                            \"values\": [\n                              \"kube-dns\"\n                            ]\n                          }\n                        ]\n                      },\n                      \"topologyKey\": \"kubernetes.io/hostname\"\n                    },\n                    \"weight\": 100\n                  }\n                ]\n              }\n            }\n          },\n          \"description\": \"Affinity of the coredns pods\",\n          \"type\": [\n            \"object\",\n            \"null\"\n          ]\n        },\n        \"computeType\": {\n          \"type\": \"string\"\n        },\n        \"corefile\": {\n          \"description\": \"Entire corefile contents to use with installation\",\n          \"type\": \"string\"\n        },\n        \"nodeSelector\": {\n          \"additionalProperties\": {\n            \"type\": \"string\"\n          },\n          \"type\": \"object\"\n        },\n        \"podAnnotations\": {\n          \"properties\": {},\n          \"title\": \"The podAnnotations Schema\",\n          \"type\": \"object\"\n        },\n        \"podDisruptionBudget\": {\n          \"description\": \"podDisruptionBudget configurations\",\n          \"enabled\": {\n            \"default\": true,\n            \"description\": \"the option to enable managed PDB\",\n            \"type\": \"boolean\"\n          },\n          \"maxUnavailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"default\": 1,\n            \"description\": \"minAvailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"minAvailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"description\": \"maxUnavailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"type\": \"object\"\n        },\n        \"podLabels\": {\n          \"properties\": {},\n          \"title\": \"The podLabels Schema\",\n          \"type\": \"object\"\n        },\n        \"replicaCount\": {\n          \"type\": \"integer\"\n        },\n        \"resources\": {\n          \"$ref\": \"#/definitions/Resources\"\n        },\n        \"tolerations\": {\n          \"default\": [\n            {\n              \"key\": \"CriticalAddonsOnly\",\n              \"operator\": \"Exists\"\n            },\n            {\n              \"effect\": \"NoSchedule\",\n              \"key\": \"node-role.kubernetes.io/control-plane\"\n            }\n          ],\n          \"description\": \"Tolerations of the coredns pod\",\n          \"items\": {\n            \"type\": \"object\"\n          },\n          \"type\": \"array\"\n        },\n        \"topologySpreadConstraints\": {\n          \"description\": \"The coredns pod topology spread constraints\",\n          \"type\": \"array\"\n        }\n      },\n      \"title\": \"Coredns\",\n      \"type\": \"object\"\n    },\n    \"Limits\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"cpu\": {\n          \"type\": \"string\"\n        },\n        \"memory\": {\n          \"type\": \"string\"\n        }\n      },\n      \"title\": \"Limits\",\n      \"type\": \"object\"\n    },\n    \"Resources\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"limits\": {\n          \"$ref\": \"#/definitions/Limits\"\n        },\n        \"requests\": {\n          \"$ref\": \"#/definitions/Limits\"\n        }\n      },\n      \"title\": \"Resources\",\n      \"type\": \"object\"\n    }\n  }\n}\n</code></pre> <p>[!NOTE] The available configuration values will vary between add-on versions, typically more configuration values will be added in later versions as functionality is enabled by EKS.</p>"},{"location":"local/","title":"Local Development","text":""},{"location":"local/#documentation-site","title":"Documentation Site","text":"<p>In order to run the documentation site locally, you will need to have the following installed locally:</p> <ul> <li>Python 3.x</li> <li>mkdocs</li> <li>The following pip packages for mkdocs (i.e. - <code>pip install ...</code>)<ul> <li><code>mkdocs-material</code></li> <li><code>mkdocs-include-markdown-plugin</code></li> <li><code>mkdocs-awesome-pages-plugin</code></li> </ul> </li> </ul> <p>To run the documentation site locally, run the following command from the root of the repository:</p> <pre><code>mkdocs serve\n</code></pre> <p>Opening the documentation at the link posted in the terminal output (i.e. - http://127.0.0.1:8000/terraform-aws-eks/)</p>"},{"location":"network_connectivity/","title":"Network Connectivity","text":""},{"location":"network_connectivity/#cluster-endpoint","title":"Cluster Endpoint","text":""},{"location":"network_connectivity/#public-endpoint-w-restricted-cidrs","title":"Public Endpoint w/ Restricted CIDRs","text":"<p>When restricting the clusters public endpoint to only the CIDRs specified by users, it is recommended that you also enable the private endpoint, or ensure that the CIDR blocks that you specify include the addresses that nodes and Fargate pods (if you use them) access the public endpoint from.</p> <p>Please refer to the AWS documentation for further information</p>"},{"location":"network_connectivity/#security-groups","title":"Security Groups","text":"<ul> <li>Cluster Security Group</li> <li>This module by default creates a cluster security group (\"additional\" security group when viewed from the console) in addition to the default security group created by the AWS EKS service. This \"additional\" security group allows users to customize inbound and outbound rules via the module as they see fit<ul> <li>The default inbound/outbound rules provided by the module are derived from the AWS minimum recommendations in addition to NTP and HTTPS public internet egress rules (without, these show up in VPC flow logs as rejects - they are used for clock sync and downloading necessary packages/updates)</li> <li>The minimum inbound/outbound rules are provided for cluster and node creation to succeed without errors, but users will most likely need to add the necessary port and protocol for node-to-node communication (this is user specific based on how nodes are configured to communicate across the cluster)</li> <li>Users have the ability to opt out of the security group creation and instead provide their own externally created security group if so desired</li> <li>The security group that is created is designed to handle the bare minimum communication necessary between the control plane and the nodes, as well as any external egress to allow the cluster to successfully launch without error</li> </ul> </li> <li>Users also have the option to supply additional, externally created security groups to the cluster as well via the <code>cluster_additional_security_group_ids</code> variable</li> <li> <p>Lastly, users are able to opt in to attaching the primary security group automatically created by the EKS service by setting <code>attach_cluster_primary_security_group</code> = <code>true</code> from the root module for the respective node group (or set it within the node group defaults). This security group is not managed by the module; it is created by the EKS service. It permits all traffic within the domain of the security group as well as all egress traffic to the internet.</p> </li> <li> <p>Node Group Security Group(s)</p> </li> <li>Users have the option to assign their own externally created security group(s) to the node group via the <code>vpc_security_group_ids</code> variable</li> </ul> <p>See the example snippet below which adds additional security group rules to the cluster security group as well as the shared node security group (for node-to-node access). Users can use this extensibility to open up network access as they see fit using the security groups provided by the module:</p> <p><pre><code>  ...\n  # Extend cluster security group rules\n  cluster_security_group_additional_rules = {\n    egress_nodes_ephemeral_ports_tcp = {\n      description                = \"To node 1025-65535\"\n      protocol                   = \"tcp\"\n      from_port                  = 1025\n      to_port                    = 65535\n      type                       = \"egress\"\n      source_node_security_group = true\n    }\n  }\n\n  # Extend node-to-node security group rules\n  node_security_group_additional_rules = {\n    ingress_self_all = {\n      description = \"Node to node all ports/protocols\"\n      protocol    = \"-1\"\n      from_port   = 0\n      to_port     = 0\n      type        = \"ingress\"\n      self        = true\n    }\n    egress_all = {\n      description      = \"Node all egress\"\n      protocol         = \"-1\"\n      from_port        = 0\n      to_port          = 0\n      type             = \"egress\"\n      cidr_blocks      = [\"0.0.0.0/0\"]\n      ipv6_cidr_blocks = [\"::/0\"]\n    }\n  }\n  ...\n</code></pre> The security groups created by this module are depicted in the image shown below along with their default inbound/outbound rules:</p> <p> </p>"},{"location":"user_data/","title":"User Data &amp; Bootstrapping","text":"<p>Users can see the various methods of using and providing user data through the user data examples as well more detailed information on the design and possible configurations via the user data module itself</p>"},{"location":"user_data/#summary","title":"Summary","text":"<ul> <li>AWS EKS Managed Node Groups</li> <li>By default, any supplied user data is pre-pended to the user data supplied by the EKS Managed Node Group service</li> <li>If users supply an <code>ami_id</code>, the service no longers supplies user data to bootstrap nodes; users can enable <code>enable_bootstrap_user_data</code> and use the module provided user data template, or provide their own user data template</li> <li>AMI types of <code>BOTTLEROCKET_*</code>, user data must be in TOML format</li> <li>AMI types of <code>WINDOWS_*</code>, user data must be in powershell/PS1 script format</li> <li>Self Managed Node Groups</li> <li><code>AL2_x86_64</code> AMI type (default) -&gt; the user data template (bash/shell script) provided by the module is used as the default; users are able to provide their own user data template</li> <li><code>BOTTLEROCKET_*</code> AMI types -&gt; the user data template (TOML file) provided by the module is used as the default; users are able to provide their own user data template</li> <li><code>WINDOWS_*</code> AMI types -&gt; the user data template (powershell/PS1 script) provided by the module is used as the default; users are able to provide their own user data template</li> </ul> <p>The templates provided by the module can be found under the templates directory</p>"},{"location":"user_data/#eks-managed-node-group","title":"EKS Managed Node Group","text":"<p>When using an EKS managed node group, users have 2 primary routes for interacting with the bootstrap user data:</p> <ol> <li> <p>If a value for <code>ami_id</code> is not provided, users can supply additional user data that is pre-pended before the EKS Managed Node Group bootstrap user data. You can read more about this process from the AWS supplied documentation</p> </li> <li> <p>Users can use the following variables to facilitate this process:</p> <pre><code>pre_bootstrap_user_data = \"...\"\n</code></pre> </li> <li> <p>If a custom AMI is used, then per the AWS documentation, users will need to supply the necessary user data to bootstrap and register nodes with the cluster when launched. There are two routes to facilitate this bootstrapping process:</p> </li> <li>If the AMI used is a derivative of the AWS EKS Optimized AMI , users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:<ul> <li>Users can use the following variables to facilitate this process:    <pre><code>enable_bootstrap_user_data = true # to opt in to using the module supplied bootstrap user data template\npre_bootstrap_user_data    = \"...\"\nbootstrap_extra_args       = \"...\"\npost_bootstrap_user_data   = \"...\"\n</code></pre></li> </ul> </li> <li>If the AMI is NOT an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the <code>templatefile()</code> for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.<ul> <li>Users can use the following variables to facilitate this process:    <pre><code>user_data_template_path  = \"./your/user_data.sh\" # user supplied bootstrap user data template\npre_bootstrap_user_data  = \"...\"\nbootstrap_extra_args     = \"...\"\npost_bootstrap_user_data = \"...\"\n</code></pre></li> </ul> </li> </ol> \u2139\ufe0f When using bottlerocket, the supplied user data (TOML format) is merged in with the values supplied by EKS. Therefore, <code>pre_bootstrap_user_data</code> and <code>post_bootstrap_user_data</code> are not valid since the bottlerocket OS handles when various settings are applied. If you wish to supply additional configuration settings when using bottlerocket, supply them via the <code>bootstrap_extra_args</code> variable. For the <code>AL2_*</code> AMI types, <code>bootstrap_extra_args</code> are settings that will be supplied to the AWS EKS Optimized AMI bootstrap script such as kubelet extra args, etc. See the bottlerocket GitHub repository documentation for more details on what settings can be supplied via the <code>bootstrap_extra_args</code> variable."},{"location":"user_data/#self-managed-node-group","title":"Self Managed Node Group","text":"<p>Self managed node groups require users to provide the necessary bootstrap user data. Users can elect to use the user data template provided by the module for their respective AMI type or provide their own user data template for rendering by the module.</p> <ul> <li>If the AMI used is a derivative of the AWS EKS Optimized AMI , users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:</li> <li>Users can use the following variables to facilitate this process:     <pre><code>enable_bootstrap_user_data = true # to opt in to using the module supplied bootstrap user data template\npre_bootstrap_user_data    = \"...\"\nbootstrap_extra_args       = \"...\"\npost_bootstrap_user_data   = \"...\"\n</code></pre></li> <li>If the AMI is NOT an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the <code>templatefile()</code> for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.<ul> <li>Users can use the following variables to facilitate this process:   <pre><code>user_data_template_path  = \"./your/user_data.sh\" # user supplied bootstrap user data template\npre_bootstrap_user_data  = \"...\"\nbootstrap_extra_args     = \"...\"\npost_bootstrap_user_data = \"...\"\n</code></pre></li> </ul> </li> </ul>"},{"location":"user_data/#logic-diagram","title":"Logic Diagram","text":"<p>The rough flow of logic that is encapsulated within the <code>_user_data</code> module can be represented by the following diagram to better highlight the various manners in which user data can be populated.</p> <p> </p>"}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Terraform AWS EKS module","text":"<p>Moar content coming soon!</p>"},{"location":"UPGRADE-17.0/","title":"How to handle the terraform-aws-eks module upgrade","text":""},{"location":"UPGRADE-17.0/#upgrade-module-to-v1700-for-managed-node-groups","title":"Upgrade module to v17.0.0 for Managed Node Groups","text":"<p>In this release, we now decided to remove random_pet resources in Managed Node Groups (MNG). Those were used to recreate MNG if something changed. But they were causing a lot of issues. To upgrade the module without recreating your MNG, you will need to explicitly reuse their previous name and set them in your MNG <code>name</code> argument.</p> <ol> <li>Run <code>terraform apply</code> with the module version v16.2.0</li> <li>Get your worker group names</li> </ol> <pre><code>~ terraform state show 'module.eks.module.node_groups.aws_eks_node_group.workers[\"example\"]' | grep node_group_name\nnode_group_name = \"test-eks-mwIwsvui-example-sincere-squid\"\n</code></pre> <ol> <li>Upgrade your module and configure your node groups to use existing names</li> </ol> <pre><code>module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"17.0.0\"\n\n  cluster_name    = \"test-eks-mwIwsvui\"\n  cluster_version = \"1.20\"\n  # ...\n\n  node_groups = {\n    example = {\n      name = \"test-eks-mwIwsvui-example-sincere-squid\"\n\n      # ...\n    }\n  }\n  # ...\n}\n</code></pre> <ol> <li>Run <code>terraform plan</code>, you should see that only <code>random_pets</code> will be destroyed</li> </ol> <pre><code>Terraform will perform the following actions:\n\n  # module.eks.module.node_groups.random_pet.node_groups[\"example\"] will be destroyed\n  - resource \"random_pet\" \"node_groups\" {\n      - id        = \"sincere-squid\" -&gt; null\n      - keepers   = {\n          - \"ami_type\"                  = \"AL2_x86_64\"\n          - \"capacity_type\"             = \"SPOT\"\n          - \"disk_size\"                 = \"50\"\n          - \"iam_role_arn\"              = \"arn:aws:iam::123456789123:role/test-eks-mwIwsvui20210527220853611600000009\"\n          - \"instance_types\"            = \"t3.large\"\n          - \"key_name\"                  = \"\"\n          - \"node_group_name\"           = \"test-eks-mwIwsvui-example\"\n          - \"source_security_group_ids\" = \"\"\n          - \"subnet_ids\"                = \"subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx|subnet-xxxxxxxxxxxx\"\n        } -&gt; null\n      - length    = 2 -&gt; null\n      - separator = \"-\" -&gt; null\n    }\n\nPlan: 0 to add, 0 to change, 1 to destroy.\n</code></pre> <ol> <li>If everything sounds good to you, run <code>terraform apply</code></li> </ol> <p>After the first apply, we recommend you to create a new node group and let the module use the <code>node_group_name_prefix</code> (by removing the <code>name</code> argument) to generate names and avoid collision during node groups re-creation if needed, because the lifecycle is <code>create_before_destroy = true</code>.</p>"},{"location":"UPGRADE-18.0/","title":"Upgrade from v17.x to v18.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p> <p>Note: please see https://github.com/terraform-aws-modules/terraform-aws-eks/issues/1744 where users have shared the steps/changes that have worked for their configurations to upgrade. Due to the numerous configuration possibilities, it is difficult to capture specific steps that will work for all; this has proven to be a useful thread to share collective information from the broader community regarding v18.x upgrades.</p> <p>For most users, adding the following to your v17.x configuration will preserve the state of your cluster control plane when upgrading to v18.x:</p> <pre><code>prefix_separator                   = \"\"\niam_role_name                      = $CLUSTER_NAME\ncluster_security_group_name        = $CLUSTER_NAME\ncluster_security_group_description = \"EKS cluster security group.\"\n</code></pre> <p>This configuration assumes that <code>create_iam_role</code> is set to <code>true</code>, which is the default value.</p> <p>As the location of the Terraform state of the IAM role has been changed from 17.x to 18.x, you'll also have to move the state before running <code>terraform apply</code> by calling:</p> <pre><code>terraform state mv 'module.eks.aws_iam_role.cluster[0]' 'module.eks.aws_iam_role.this[0]'\n</code></pre> <p>See more information here</p>"},{"location":"UPGRADE-18.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>Launch configuration support has been removed and only launch template is supported going forward. AWS is no longer adding new features back into launch configuration and their docs state <code>We strongly recommend that you do not use launch configurations. They do not provide full functionality for Amazon EC2 Auto Scaling or Amazon EC2. We provide information about launch configurations for customers who have not yet migrated from launch configurations to launch templates.</code></li> <li>Support for managing aws-auth configmap has been removed. This change also removes the dependency on the Kubernetes Terraform provider, the local dependency on aws-iam-authenticator for users, as well as the reliance on the forked http provider to wait and poll on cluster creation. To aid users in this change, an output variable <code>aws_auth_configmap_yaml</code> has been provided which renders the aws-auth configmap necessary to support at least the IAM roles used by the module (additional mapRoles/mapUsers definitions to be provided by users)</li> <li>Support for managing kubeconfig and its associated <code>local_file</code> resources have been removed; users are able to use the awscli provided <code>aws eks update-kubeconfig --name &lt;cluster_name&gt;</code> to update their local kubeconfig as necessary</li> <li>The terminology used in the module has been modified to reflect that used by the AWS documentation.</li> <li>AWS EKS Managed Node Group, <code>eks_managed_node_groups</code>, was previously referred to as simply node group, <code>node_groups</code></li> <li>Self Managed Node Group Group, <code>self_managed_node_groups</code>, was previously referred to as worker group, <code>worker_groups</code></li> <li>AWS Fargate Profile, <code>fargate_profiles</code>, remains unchanged in terms of naming and terminology</li> <li>The three different node group types supported by AWS and the module have been refactored into standalone sub-modules that are both used by the root <code>eks</code> module as well as available for individual, standalone consumption if desired.</li> <li>The previous <code>node_groups</code> sub-module is now named <code>eks-managed-node-group</code> and provisions a single AWS EKS Managed Node Group per sub-module definition (previous version utilized <code>for_each</code> to create 0 or more node groups)<ul> <li>Additional changes for the <code>eks-managed-node-group</code> sub-module over the previous <code>node_groups</code> module include:</li> <li>Variable name changes defined in section <code>Variable and output changes</code> below</li> <li>Support for nearly full control of the IAM role created, or provide the ARN of an existing IAM role, has been added</li> <li>Support for nearly full control of the security group created, or provide the ID of an existing security group, has been added</li> <li>User data has been revamped and all user data logic moved to the <code>_user_data</code> internal sub-module; the local <code>userdata.sh.tpl</code> has been removed entirely</li> </ul> </li> <li>The previous <code>fargate</code> sub-module is now named <code>fargate-profile</code> and provisions a single AWS EKS Fargate Profile per sub-module definition (previous version utilized <code>for_each</code> to create 0 or more profiles)<ul> <li>Additional changes for the <code>fargate-profile</code> sub-module over the previous <code>fargate</code> module include:</li> <li>Variable name changes defined in section <code>Variable and output changes</code> below</li> <li>Support for nearly full control of the IAM role created, or provide the ARN of an existing IAM role, has been added</li> <li>Similar to the <code>eks_managed_node_group_defaults</code> and <code>self_managed_node_group_defaults</code>, a <code>fargate_profile_defaults</code> has been provided to allow users to control the default configurations for the Fargate profiles created</li> </ul> </li> <li>A sub-module for <code>self-managed-node-group</code> has been created and provisions a single self managed node group (autoscaling group) per sub-module definition<ul> <li>Additional changes for the <code>self-managed-node-group</code> sub-module over the previous <code>node_groups</code> variable include:</li> <li>The underlying autoscaling group and launch template have been updated to more closely match that of the <code>terraform-aws-autoscaling</code> module and the features it offers</li> <li>The previous iteration used a count over a list of node group definitions which was prone to disruptive updates; this is now replaced with a map/for_each to align with that of the EKS managed node group and Fargate profile behaviors/style</li> </ul> </li> <li>The user data configuration supported across the module has been completely revamped. A new <code>_user_data</code> internal sub-module has been created to consolidate all user data configuration in one location which provides better support for testability (via the <code>tests/user-data</code> example). The new sub-module supports nearly all possible combinations including the ability to allow users to provide their own user data template which will be rendered by the module. See the <code>tests/user-data</code> example project for the full plethora of example configuration possibilities and more details on the logic of the design can be found in the <code>modules/_user_data</code> directory.</li> <li>Resource name changes may cause issues with existing resources. For example, security groups and IAM roles cannot be renamed, they must be recreated. Recreation of these resources may also trigger a recreation of the cluster. To use the legacy (&lt; 18.x) resource naming convention, set <code>prefix_separator</code> to \"\".</li> <li>Security group usage has been overhauled to provide only the bare minimum network connectivity required to launch a bare bones cluster. See the security group documentation section for more details. Users upgrading to v18.x will want to review the rules they have in place today versus the rules provisioned by the v18.x module and ensure to make any necessary adjustments for their specific workload.</li> </ul>"},{"location":"UPGRADE-18.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-18.0/#added","title":"Added","text":"<ul> <li>Support for AWS EKS Addons has been added</li> <li>Support for AWS EKS Cluster Identity Provider Configuration has been added</li> <li>AWS Terraform provider minimum required version has been updated to 3.64 to support the changes made and additional resources supported</li> <li>An example <code>user_data</code> project has been added to aid in demonstrating, testing, and validating the various methods of configuring user data with the <code>_user_data</code> sub-module as well as the root <code>eks</code> module</li> <li>Template for rendering the aws-auth configmap output - <code>aws_auth_cm.tpl</code></li> <li>Template for Bottlerocket OS user data bootstrapping - <code>bottlerocket_user_data.tpl</code></li> </ul>"},{"location":"UPGRADE-18.0/#modified","title":"Modified","text":"<ul> <li>The previous <code>fargate</code> example has been renamed to <code>fargate_profile</code></li> <li>The previous <code>irsa</code> and <code>instance_refresh</code> examples have been merged into one example <code>irsa_autoscale_refresh</code></li> <li>The previous <code>managed_node_groups</code> example has been renamed to <code>self_managed_node_group</code></li> <li>The previously hardcoded EKS OIDC root CA thumbprint value and variable has been replaced with a <code>tls_certificate</code> data source that refers to the cluster OIDC issuer url. Thumbprint values should remain unchanged however</li> <li>Individual cluster security group resources have been replaced with a single security group resource that takes a map of rules as input. The default ingress/egress rules have had their scope reduced in order to provide the bare minimum of access to permit successful cluster creation and allow users to opt in to any additional network access as needed for a better security posture. This means the <code>0.0.0.0/0</code> egress rule has been removed, instead TCP/443 and TCP/10250 egress rules to the node group security group are used instead</li> <li>The Linux/bash user data template has been updated to include the bare minimum necessary for bootstrapping AWS EKS Optimized AMI derivative nodes with provisions for providing additional user data and configurations; was named <code>userdata.sh.tpl</code> and is now named <code>linux_user_data.tpl</code></li> <li>The Windows user data template has been renamed from <code>userdata_windows.tpl</code> to <code>windows_user_data.tpl</code></li> </ul>"},{"location":"UPGRADE-18.0/#removed","title":"Removed","text":"<ul> <li>Miscellaneous documents on how to configure Kubernetes cluster internals have been removed. Documentation related to how to configure the AWS EKS Cluster and its supported infrastructure resources provided by the module are supported, while cluster internal configuration is out of scope for this project</li> <li>The previous <code>bottlerocket</code> example has been removed in favor of demonstrating the use and configuration of Bottlerocket nodes via the respective <code>eks_managed_node_group</code> and <code>self_managed_node_group</code> examples</li> <li>The previous <code>launch_template</code> and <code>launch_templates_with_managed_node_groups</code> examples have been removed; only launch templates are now supported (default) and launch configuration support has been removed</li> <li>The previous <code>secrets_encryption</code> example has been removed; the functionality has been demonstrated in several of the new examples rendering this standalone example redundant</li> <li>The additional, custom IAM role policy for the cluster role has been removed. The permissions are either now provided in the attached managed AWS permission policies used or are no longer required</li> <li>The <code>kubeconfig.tpl</code> template; kubeconfig management is no longer supported under this module</li> <li>The HTTP Terraform provider (forked copy) dependency has been removed</li> </ul>"},{"location":"UPGRADE-18.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> <ul> <li><code>cluster_create_timeout</code>, <code>cluster_update_timeout</code>, and <code>cluster_delete_timeout</code> have been replaced with <code>cluster_timeouts</code></li> <li><code>kubeconfig_name</code></li> <li><code>kubeconfig_output_path</code></li> <li><code>kubeconfig_file_permission</code></li> <li><code>kubeconfig_api_version</code></li> <li><code>kubeconfig_aws_authenticator_command</code></li> <li><code>kubeconfig_aws_authenticator_command_args</code></li> <li><code>kubeconfig_aws_authenticator_additional_args</code></li> <li><code>kubeconfig_aws_authenticator_env_variables</code></li> <li><code>write_kubeconfig</code></li> <li><code>default_platform</code></li> <li><code>manage_aws_auth</code></li> <li><code>aws_auth_additional_labels</code></li> <li><code>map_accounts</code></li> <li><code>map_roles</code></li> <li><code>map_users</code></li> <li><code>fargate_subnets</code></li> <li><code>worker_groups_launch_template</code></li> <li><code>worker_security_group_id</code></li> <li><code>worker_ami_name_filter</code></li> <li><code>worker_ami_name_filter_windows</code></li> <li><code>worker_ami_owner_id</code></li> <li><code>worker_ami_owner_id_windows</code></li> <li><code>worker_additional_security_group_ids</code></li> <li><code>worker_sg_ingress_from_port</code></li> <li><code>workers_additional_policies</code></li> <li><code>worker_create_security_group</code></li> <li><code>worker_create_initial_lifecycle_hooks</code></li> <li><code>worker_create_cluster_primary_security_group_rules</code></li> <li><code>cluster_create_endpoint_private_access_sg_rule</code></li> <li><code>cluster_endpoint_private_access_cidrs</code></li> <li><code>cluster_endpoint_private_access_sg</code></li> <li><code>manage_worker_iam_resources</code></li> <li><code>workers_role_name</code></li> <li><code>attach_worker_cni_policy</code></li> <li><code>eks_oidc_root_ca_thumbprint</code></li> <li><code>create_fargate_pod_execution_role</code></li> <li><code>fargate_pod_execution_role_name</code></li> <li><code>cluster_egress_cidrs</code></li> <li><code>workers_egress_cidrs</code></li> <li><code>wait_for_cluster_timeout</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>default_iam_role_arn</code></li> <li><code>workers_group_defaults</code></li> <li><code>worker_security_group_id</code></li> <li><code>node_groups_defaults</code></li> <li><code>node_groups</code></li> <li><code>ebs_optimized_not_supported</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>create_eks</code> and <code>create_fargate_pod_execution_role</code> have been replaced with simply <code>create</code></li> </ul> </li> <li> <p>Renamed variables:</p> <ul> <li><code>create_eks</code> -&gt; <code>create</code></li> <li><code>subnets</code> -&gt; <code>subnet_ids</code></li> <li><code>cluster_create_security_group</code> -&gt; <code>create_cluster_security_group</code></li> <li><code>cluster_log_retention_in_days</code> -&gt; <code>cloudwatch_log_group_retention_in_days</code></li> <li><code>cluster_log_kms_key_id</code> -&gt; <code>cloudwatch_log_group_kms_key_id</code></li> <li><code>manage_cluster_iam_resources</code> -&gt; <code>create_iam_role</code></li> <li><code>cluster_iam_role_name</code> -&gt; <code>iam_role_name</code></li> <li><code>permissions_boundary</code> -&gt; <code>iam_role_permissions_boundary</code></li> <li><code>iam_path</code> -&gt; <code>iam_role_path</code></li> <li><code>pre_userdata</code> -&gt; <code>pre_bootstrap_user_data</code></li> <li><code>additional_userdata</code> -&gt; <code>post_bootstrap_user_data</code></li> <li><code>worker_groups</code> -&gt; <code>self_managed_node_groups</code></li> <li><code>workers_group_defaults</code> -&gt; <code>self_managed_node_group_defaults</code></li> <li><code>node_groups</code> -&gt; <code>eks_managed_node_groups</code></li> <li><code>node_groups_defaults</code> -&gt; <code>eks_managed_node_group_defaults</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>create_eks</code> -&gt; <code>create</code></li> <li><code>worker_additional_security_group_ids</code> -&gt; <code>vpc_security_group_ids</code></li> <li>Fargate profile sub-module</li> <li><code>fargate_pod_execution_role_name</code> -&gt; <code>name</code></li> <li><code>create_fargate_pod_execution_role</code> -&gt; <code>create_iam_role</code></li> <li><code>subnets</code> -&gt; <code>subnet_ids</code></li> <li><code>iam_path</code> -&gt; <code>iam_role_path</code></li> <li><code>permissions_boundary</code> -&gt; <code>iam_role_permissions_boundary</code></li> </ul> </li> <li> <p>Added variables:</p> <ul> <li><code>cluster_additional_security_group_ids</code> added to allow users to add additional security groups to the cluster as needed</li> <li><code>cluster_security_group_name</code></li> <li><code>cluster_security_group_use_name_prefix</code> added to allow users to use either the name as specified or default to using the name specified as a prefix</li> <li><code>cluster_security_group_description</code></li> <li><code>cluster_security_group_additional_rules</code></li> <li><code>cluster_security_group_tags</code></li> <li><code>create_cloudwatch_log_group</code> added in place of the logic that checked if any cluster log types were enabled to allow users to opt in as they see fit</li> <li><code>create_node_security_group</code> added to create single security group that connects node groups and cluster in central location</li> <li><code>node_security_group_id</code></li> <li><code>node_security_group_name</code></li> <li><code>node_security_group_use_name_prefix</code></li> <li><code>node_security_group_description</code></li> <li><code>node_security_group_additional_rules</code></li> <li><code>node_security_group_tags</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li><code>cluster_addons</code></li> <li><code>cluster_identity_providers</code></li> <li><code>fargate_profile_defaults</code></li> <li><code>prefix_separator</code> added to support legacy behavior of not having a prefix separator</li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>platform</code></li> <li><code>enable_bootstrap_user_data</code></li> <li><code>pre_bootstrap_user_data</code></li> <li><code>post_bootstrap_user_data</code></li> <li><code>bootstrap_extra_args</code></li> <li><code>user_data_template_path</code></li> <li><code>create_launch_template</code></li> <li><code>launch_template_name</code></li> <li><code>launch_template_use_name_prefix</code></li> <li><code>description</code></li> <li><code>ebs_optimized</code></li> <li><code>ami_id</code></li> <li><code>key_name</code></li> <li><code>launch_template_default_version</code></li> <li><code>update_launch_template_default_version</code></li> <li><code>disable_api_termination</code></li> <li><code>kernel_id</code></li> <li><code>ram_disk_id</code></li> <li><code>block_device_mappings</code></li> <li><code>capacity_reservation_specification</code></li> <li><code>cpu_options</code></li> <li><code>credit_specification</code></li> <li><code>elastic_gpu_specifications</code></li> <li><code>elastic_inference_accelerator</code></li> <li><code>enclave_options</code></li> <li><code>instance_market_options</code></li> <li><code>license_specifications</code></li> <li><code>metadata_options</code></li> <li><code>enable_monitoring</code></li> <li><code>network_interfaces</code></li> <li><code>placement</code></li> <li><code>min_size</code></li> <li><code>max_size</code></li> <li><code>desired_size</code></li> <li><code>use_name_prefix</code></li> <li><code>ami_type</code></li> <li><code>ami_release_version</code></li> <li><code>capacity_type</code></li> <li><code>disk_size</code></li> <li><code>force_update_version</code></li> <li><code>instance_types</code></li> <li><code>labels</code></li> <li><code>cluster_version</code></li> <li><code>launch_template_version</code></li> <li><code>remote_access</code></li> <li><code>taints</code></li> <li><code>update_config</code></li> <li><code>timeouts</code></li> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>vpc_id</code></li> <li><code>security_group_rules</code></li> <li><code>cluster_security_group_id</code></li> <li><code>security_group_tags</code></li> <li><code>create_iam_role</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_name</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_path</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_permissions_boundary</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>iam_role_arn</code> (for if <code>create_iam_role</code> is <code>false</code> to bring your own externally created role)</li> <li><code>iam_role_name</code></li> <li><code>iam_role_use_name_prefix</code></li> <li><code>iam_role_description</code></li> <li><code>iam_role_additional_policies</code></li> <li><code>iam_role_tags</code></li> <li><code>selectors</code></li> <li><code>timeouts</code></li> </ul> </li> <li> <p>Removed outputs:</p> <ul> <li><code>cluster_version</code></li> <li><code>kubeconfig</code></li> <li><code>kubeconfig_filename</code></li> <li><code>workers_asg_arns</code></li> <li><code>workers_asg_names</code></li> <li><code>workers_user_data</code></li> <li><code>workers_default_ami_id</code></li> <li><code>workers_default_ami_id_windows</code></li> <li><code>workers_launch_template_ids</code></li> <li><code>workers_launch_template_arns</code></li> <li><code>workers_launch_template_latest_versions</code></li> <li><code>worker_security_group_id</code></li> <li><code>worker_iam_instance_profile_arns</code></li> <li><code>worker_iam_instance_profile_names</code></li> <li><code>worker_iam_role_name</code></li> <li><code>worker_iam_role_arn</code></li> <li><code>fargate_profile_ids</code></li> <li><code>fargate_profile_arns</code></li> <li><code>fargate_iam_role_name</code></li> <li><code>fargate_iam_role_arn</code></li> <li><code>node_groups</code></li> <li><code>security_group_rule_cluster_https_worker_ingress</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>node_groups</code></li> <li><code>aws_auth_roles</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>aws_auth_roles</code></li> </ul> </li> <li> <p>Renamed outputs:</p> <ul> <li><code>config_map_aws_auth</code> -&gt; <code>aws_auth_configmap_yaml</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>fargate_profile_ids</code> -&gt; <code>fargate_profile_id</code></li> <li><code>fargate_profile_arns</code> -&gt; <code>fargate_profile_arn</code></li> </ul> </li> <li> <p>Added outputs:</p> <ul> <li><code>cluster_platform_version</code></li> <li><code>cluster_status</code></li> <li><code>cluster_security_group_arn</code></li> <li><code>cluster_security_group_id</code></li> <li><code>node_security_group_arn</code></li> <li><code>node_security_group_id</code></li> <li><code>cluster_iam_role_unique_id</code></li> <li><code>cluster_addons</code></li> <li><code>cluster_identity_providers</code></li> <li><code>fargate_profiles</code></li> <li><code>eks_managed_node_groups</code></li> <li><code>self_managed_node_groups</code></li> <li>EKS Managed Node Group sub-module (was <code>node_groups</code>)</li> <li><code>launch_template_id</code></li> <li><code>launch_template_arn</code></li> <li><code>launch_template_latest_version</code></li> <li><code>node_group_arn</code></li> <li><code>node_group_id</code></li> <li><code>node_group_resources</code></li> <li><code>node_group_status</code></li> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> <li><code>iam_role_name</code></li> <li><code>iam_role_arn</code></li> <li><code>iam_role_unique_id</code></li> <li>Fargate profile sub-module (was <code>fargate</code>)</li> <li><code>iam_role_unique_id</code></li> <li><code>fargate_profile_status</code></li> </ul> </li> </ol>"},{"location":"UPGRADE-18.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-18.0/#before-17x-example","title":"Before 17.x Example","text":"<pre><code>module \"eks\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~&gt; 17.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id  = module.vpc.vpc_id\n  subnets = module.vpc.private_subnets\n\n  # Managed Node Groups\n  node_groups_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  node_groups = {\n    node_group = {\n      min_capacity     = 1\n      max_capacity     = 10\n      desired_capacity = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      k8s_labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      additional_tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Worker groups\n  worker_additional_security_group_ids = [aws_security_group.additional.id]\n\n  worker_groups_launch_template = [\n    {\n      name                    = \"worker-group\"\n      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n      spot_instance_pools     = 4\n      asg_max_size            = 5\n      asg_desired_capacity    = 2\n      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n      public_ip               = true\n    },\n  ]\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#after-18x-example","title":"After 18.x Example","text":"<pre><code>module \"cluster_after\" {\n  source  = \"terraform-aws-modules/eks/aws\"\n  version = \"~&gt; 18.0\"\n\n  cluster_name                    = local.name\n  cluster_version                 = local.cluster_version\n  cluster_endpoint_private_access = true\n  cluster_endpoint_public_access  = true\n\n  vpc_id     = module.vpc.vpc_id\n  subnet_ids = module.vpc.private_subnets\n\n  eks_managed_node_group_defaults = {\n    ami_type  = \"AL2_x86_64\"\n    disk_size = 50\n  }\n\n  eks_managed_node_groups = {\n    node_group = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n\n      update_config = {\n        max_unavailable_percentage = 50\n      }\n\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = [\n        {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      ]\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n  }\n\n  self_managed_node_groups = {\n    worker_group = {\n      name = \"worker-group\"\n\n      min_size      = 1\n      max_size      = 5\n      desired_size  = 2\n      instance_type = \"m4.large\"\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      block_device_mappings = {\n        xvda = {\n          device_name = \"/dev/xvda\"\n          ebs = {\n            delete_on_termination = true\n            encrypted             = false\n            volume_size           = 100\n            volume_type           = \"gp2\"\n          }\n\n        }\n      }\n\n      use_mixed_instances_policy = true\n      mixed_instances_policy = {\n        instances_distribution = {\n          spot_instance_pools = 4\n        }\n\n        override = [\n          { instance_type = \"m5.large\" },\n          { instance_type = \"m5a.large\" },\n          { instance_type = \"m5d.large\" },\n          { instance_type = \"m5ad.large\" },\n        ]\n      }\n    }\n  }\n\n  # Fargate\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  tags = {\n    Environment = \"test\"\n    GithubRepo  = \"terraform-aws-eks\"\n    GithubOrg   = \"terraform-aws-modules\"\n  }\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#diff-of-before-after","title":"Diff of before &lt;&gt; after","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 17.0\"\n+  version = \"~&gt; 18.0\"\n\n   cluster_name                    = local.name\n   cluster_version                 = local.cluster_version\n   cluster_endpoint_private_access = true\n   cluster_endpoint_public_access  = true\n\n   vpc_id  = module.vpc.vpc_id\n-  subnets = module.vpc.private_subnets\n+  subnet_ids = module.vpc.private_subnets\n\n-  # Managed Node Groups\n-  node_groups_defaults = {\n+  eks_managed_node_group_defaults = {\n     ami_type  = \"AL2_x86_64\"\n     disk_size = 50\n   }\n\n-  node_groups = {\n+  eks_managed_node_groups = {\n     node_group = {\n-      min_capacity     = 1\n-      max_capacity     = 10\n-      desired_capacity = 1\n+      min_size     = 1\n+      max_size     = 10\n+      desired_size = 1\n\n       instance_types = [\"t3.large\"]\n       capacity_type  = \"SPOT\"\n\n       update_config = {\n         max_unavailable_percentage = 50\n       }\n\n-      k8s_labels = {\n+      labels = {\n         Environment = \"test\"\n         GithubRepo  = \"terraform-aws-eks\"\n         GithubOrg   = \"terraform-aws-modules\"\n       }\n\n       taints = [\n         {\n           key    = \"dedicated\"\n           value  = \"gpuGroup\"\n           effect = \"NO_SCHEDULE\"\n         }\n       ]\n\n-      additional_tags = {\n+      tags = {\n         ExtraTag = \"example\"\n       }\n     }\n   }\n\n-  # Worker groups\n-  worker_additional_security_group_ids = [aws_security_group.additional.id]\n-\n-  worker_groups_launch_template = [\n-    {\n-      name                    = \"worker-group\"\n-      override_instance_types = [\"m5.large\", \"m5a.large\", \"m5d.large\", \"m5ad.large\"]\n-      spot_instance_pools     = 4\n-      asg_max_size            = 5\n-      asg_desired_capacity    = 2\n-      kubelet_extra_args      = \"--node-labels=node.kubernetes.io/lifecycle=spot\"\n-      public_ip               = true\n-    },\n-  ]\n+  self_managed_node_group_defaults = {\n+    vpc_security_group_ids = [aws_security_group.additional.id]\n+  }\n+\n+  self_managed_node_groups = {\n+    worker_group = {\n+      name = \"worker-group\"\n+\n+      min_size      = 1\n+      max_size      = 5\n+      desired_size  = 2\n+      instance_type = \"m4.large\"\n+\n+      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n+\n+      block_device_mappings = {\n+        xvda = {\n+          device_name = \"/dev/xvda\"\n+          ebs = {\n+            delete_on_termination = true\n+            encrypted             = false\n+            volume_size           = 100\n+            volume_type           = \"gp2\"\n+          }\n+\n+        }\n+      }\n+\n+      use_mixed_instances_policy = true\n+      mixed_instances_policy = {\n+        instances_distribution = {\n+          spot_instance_pools = 4\n+        }\n+\n+        override = [\n+          { instance_type = \"m5.large\" },\n+          { instance_type = \"m5a.large\" },\n+          { instance_type = \"m5d.large\" },\n+          { instance_type = \"m5ad.large\" },\n+        ]\n+      }\n+    }\n+  }\n\n   # Fargate\n   fargate_profiles = {\n     default = {\n       name = \"default\"\n       selectors = [\n         {\n           namespace = \"kube-system\"\n           labels = {\n             k8s-app = \"kube-dns\"\n           }\n         },\n         {\n           namespace = \"default\"\n         }\n       ]\n\n       tags = {\n         Owner = \"test\"\n       }\n\n       timeouts = {\n         create = \"20m\"\n         delete = \"20m\"\n       }\n     }\n   }\n\n   tags = {\n     Environment = \"test\"\n     GithubRepo  = \"terraform-aws-eks\"\n     GithubOrg   = \"terraform-aws-modules\"\n   }\n }\n</code></pre>"},{"location":"UPGRADE-18.0/#attaching-an-iam-role-policy-to-a-fargate-profile","title":"Attaching an IAM role policy to a Fargate profile","text":""},{"location":"UPGRADE-18.0/#before-17x","title":"Before 17.x","text":"<pre><code>resource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre>"},{"location":"UPGRADE-18.0/#after-18x","title":"After 18.x","text":"<pre><code># Attach the policy to an \"example\" Fargate profile\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  role       = module.eks.fargate_profiles[\"example\"].iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre> <p>Or:</p> <pre><code># Attach the policy to all Fargate profiles\nresource \"aws_iam_role_policy_attachment\" \"default\" {\n  for_each = module.eks.fargate_profiles\n\n  role       = each.value.iam_role_name\n  policy_arn = aws_iam_policy.default.arn\n}\n</code></pre>"},{"location":"UPGRADE-19.0/","title":"Upgrade from v18.x to v19.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p>"},{"location":"UPGRADE-19.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>The <code>cluster_id</code> output used to output the name of the cluster. This is due to the fact that the cluster name is a unique constraint and therefore its set as the unique identifier within Terraform's state map. However, starting with local EKS clusters created on Outposts, there is now an attribute returned from the <code>aws eks create-cluster</code> API named <code>id</code>. The <code>cluster_id</code> has been updated to return this value which means that for current, standard EKS clusters created in the AWS cloud, no value will be returned (at the time of this writing) for <code>cluster_id</code> and only local EKS clusters on Outposts will return a value that looks like a UUID/GUID. Users should switch all instances of <code>cluster_id</code> to use <code>cluster_name</code> before upgrading to v19. Reference</li> <li>Minimum supported version of Terraform AWS provider updated to v4.45 to support the latest features provided via the resources utilized.</li> <li>Minimum supported version of Terraform updated to v1.0</li> <li>Individual security group created per EKS managed node group or self-managed node group has been removed. This configuration went mostly unused and would often cause confusion (\"Why is there an empty security group attached to my nodes?\"). This functionality can easily be replicated by user's providing one or more externally created security groups to attach to nodes launched from the node group.</li> <li>Previously, <code>var.iam_role_additional_policies</code> (one for each of the following: cluster IAM role, EKS managed node group IAM role, self-managed node group IAM role, and Fargate Profile IAM role) accepted a list of strings. This worked well for policies that already existed but failed for policies being created at the same time as the cluster due to the well-known issue of unknown values used in a <code>for_each</code> loop. To rectify this issue in <code>v19.x</code>, two changes were made:</li> <li><code>var.iam_role_additional_policies</code> was changed from type <code>list(string)</code> to type <code>map(string)</code> -&gt; this is a breaking change. More information on managing this change can be found below, under <code>Terraform State Moves</code></li> <li>The logic used in the root module for this variable was changed to replace the use of <code>try()</code> with <code>lookup()</code>. More details on why can be found here</li> <li>The cluster name has been removed from the Karpenter module event rule names. Due to the use of long cluster names appending to the provided naming scheme, the cluster name has moved to a <code>ClusterName</code> tag and the event rule name is now a prefix. This guarantees that users can have multiple instances of Karpenter with their respective event rules/SQS queue without name collisions, while also still being able to identify which queues and event rules belong to which cluster.</li> <li>The new variable <code>node_security_group_enable_recommended_rules</code> is set to true by default and may conflict with any custom ingress/egress rules. Please ensure that any duplicates from the <code>node_security_group_additional_rules</code> are removed before upgrading, or set <code>node_security_group_enable_recommended_rules</code> to false. Reference</li> </ul>"},{"location":"UPGRADE-19.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-19.0/#added","title":"Added","text":"<ul> <li>Support for setting <code>preserve</code> as well as <code>most_recent</code> on addons.</li> <li><code>preserve</code> indicates if you want to preserve the created resources when deleting the EKS add-on</li> <li><code>most_recent</code> indicates if you want to use the most recent revision of the add-on or the default version (default)</li> <li>Support for setting default node security group rules for common access patterns required:</li> <li>Egress all for <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Ingress from cluster security group for 8443/TCP and 9443/TCP for common applications such as ALB Ingress Controller, Karpenter, OPA Gatekeeper, etc. These are commonly used as webhook ports for validating and mutating webhooks</li> </ul>"},{"location":"UPGRADE-19.0/#modified","title":"Modified","text":"<ul> <li><code>cluster_security_group_additional_rules</code> and <code>node_security_group_additional_rules</code> have been modified to use <code>lookup()</code> instead of <code>try()</code> to avoid the well-known issue of unknown values within a <code>for_each</code> loop</li> <li>Default cluster security group rules have removed egress rules for TCP/443 and TCP/10250 to node groups since the cluster primary security group includes a default rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Default node security group rules have removed egress rules have been removed since the default security group settings have egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li><code>block_device_mappings</code> previously required a map of maps but has since changed to an array of maps. Users can remove the outer key for each block device mapping and replace the outermost map <code>{}</code> with an array <code>[]</code>. There are no state changes required for this change.</li> <li><code>create_kms_key</code> previously defaulted to <code>false</code> and now defaults to <code>true</code>. Clusters created with this module now default to enabling secret encryption by default with a customer-managed KMS key created by this module</li> <li><code>cluster_encryption_config</code> previously used a type of <code>list(any)</code> and now uses a type of <code>any</code> -&gt; users can simply remove the outer <code>[</code>...<code>]</code> brackets on <code>v19.x</code></li> <li><code>cluster_encryption_config</code> previously defaulted to <code>[]</code> and now defaults to <code>{resources = [\"secrets\"]}</code> to encrypt secrets by default</li> <li><code>cluster_endpoint_public_access</code> previously defaulted to <code>true</code> and now defaults to <code>false</code>. Clusters created with this module now default to private-only access to the cluster endpoint</li> <li><code>cluster_endpoint_private_access</code> previously defaulted to <code>false</code> and now defaults to <code>true</code></li> <li>The addon configuration now sets <code>\"OVERWRITE\"</code> as the default value for <code>resolve_conflicts</code> to ease add-on upgrade management. Users can opt out of this by instead setting <code>\"NONE\"</code> as the value for <code>resolve_conflicts</code></li> <li>The <code>kms</code> module used has been updated from <code>v1.0.2</code> to <code>v1.1.0</code> - no material changes other than updated to latest</li> <li>The default value for EKS managed node group <code>update_config</code> has been updated to the recommended <code>{ max_unavailable_percentage = 33 }</code></li> <li>The default value for the self-managed node group <code>instance_refresh</code> has been updated to the recommended:     <pre><code>{\n  strategy = \"Rolling\"\n  preferences = {\n    min_healthy_percentage = 66\n  }\n}\n</code></pre></li> </ul>"},{"location":"UPGRADE-19.0/#removed","title":"Removed","text":"<ul> <li>Remove all references of <code>aws_default_tags</code> to avoid update conflicts; this is the responsibility of the provider and should be handled at the provider level</li> <li>https://github.com/terraform-aws-modules/terraform-aws-eks/issues?q=is%3Aissue+default_tags+is%3Aclosed</li> <li>https://github.com/terraform-aws-modules/terraform-aws-eks/pulls?q=is%3Apr+default_tags+is%3Aclosed</li> </ul>"},{"location":"UPGRADE-19.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> </li> <li> <p><code>node_security_group_ntp_ipv4_cidr_block</code> - default security group settings have an egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></p> </li> <li><code>node_security_group_ntp_ipv6_cidr_block</code> - default security group settings have an egress rule for ALL to <code>0.0.0.0/0</code>/<code>::/0</code></li> <li>Self-managed node groups:<ul> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>security_group_rules</code></li> <li><code>security_group_tags</code></li> <li><code>cluster_security_group_id</code></li> <li><code>vpc_id</code></li> </ul> </li> <li> <p>EKS managed node groups:</p> <ul> <li><code>create_security_group</code></li> <li><code>security_group_name</code></li> <li><code>security_group_use_name_prefix</code></li> <li><code>security_group_description</code></li> <li><code>security_group_rules</code></li> <li><code>security_group_tags</code></li> <li><code>cluster_security_group_id</code></li> <li><code>vpc_id</code></li> </ul> </li> <li> <p>Renamed variables:</p> </li> <li> <p>N/A</p> </li> <li> <p>Added variables:</p> </li> <li> <p><code>provision_on_outpost</code>for Outposts support</p> </li> <li><code>outpost_config</code> for Outposts support</li> <li><code>cluster_addons_timeouts</code> for setting a common set of timeouts for all addons (unless a specific value is provided within the addon configuration)</li> <li><code>service_ipv6_cidr</code> for setting the IPv6 CIDR block for the Kubernetes service addresses</li> <li> <p><code>node_security_group_enable_recommended_rules</code> for enabling recommended node security group rules for common access patterns</p> </li> <li> <p>Self-managed node groups:</p> <ul> <li><code>launch_template_id</code> for use when using an existing/externally created launch template (Ref: https://github.com/terraform-aws-modules/terraform-aws-autoscaling/pull/204)</li> <li><code>maintenance_options</code></li> <li><code>private_dns_name_options</code></li> <li><code>instance_requirements</code></li> <li><code>context</code></li> <li><code>default_instance_warmup</code></li> <li><code>force_delete_warm_pool</code></li> </ul> </li> <li>EKS managed node groups:<ul> <li><code>use_custom_launch_template</code> was added to better clarify how users can switch between a custom launch template or the default launch template provided by the EKS managed node group. Previously, to achieve this same functionality of using the default launch template, users needed to set <code>create_launch_template = false</code> and <code>launch_template_name = \"\"</code> which is not very intuitive.</li> <li><code>launch_template_id</code> for use when using an existing/externally created launch template (Ref: https://github.com/terraform-aws-modules/terraform-aws-autoscaling/pull/204)</li> <li><code>maintenance_options</code></li> <li><code>private_dns_name_options</code>  -</li> </ul> </li> <li> <p>Removed outputs:</p> </li> <li> <p>Self-managed node groups:</p> <ul> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> </ul> </li> <li> <p>EKS managed node groups:</p> <ul> <li><code>security_group_arn</code></li> <li><code>security_group_id</code></li> </ul> </li> <li> <p>Renamed outputs:</p> </li> <li> <p><code>cluster_id</code> is not renamed but the value it returns is now different. For standard EKS clusters created in the AWS cloud, the value returned at the time of this writing is <code>null</code>/empty. For local EKS clusters created on Outposts, the value returned will look like a UUID/GUID. Users should switch all instances of <code>cluster_id</code> to use <code>cluster_name</code> before upgrading to v19. Reference</p> </li> <li> <p>Added outputs:</p> </li> <li> <p><code>cluster_name</code> - The <code>cluster_id</code> currently set by the AWS provider is actually the cluster name, but in the future, this will change and there will be a distinction between the <code>cluster_name</code> and <code>cluster_id</code>. Reference</p> </li> </ol>"},{"location":"UPGRADE-19.0/#upgrade-migrations","title":"Upgrade Migrations","text":"<ol> <li>Before upgrading your module definition to <code>v19.x</code>, please see below for both EKS managed node group(s) and self-managed node groups and remove the node group(s) security group prior to upgrading.</li> </ol>"},{"location":"UPGRADE-19.0/#self-managed-node-groups","title":"Self-Managed Node Groups","text":"<p>Self-managed node groups on <code>v18.x</code> by default create a security group that does not specify any rules. In <code>v19.x</code>, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version <code>v18.x</code> of your module definition, remove this security group from your node groups by setting <code>create_security_group = false</code>.</p> <ul> <li>If you are currently utilizing this security group, it is recommended to create an additional security group that matches the rules/settings of the security group created by the node group, and specify that security group ID in <code>vpc_security_group_ids</code>. Once this is in place, you can proceed with the original security group removal.</li> <li>For most users, the security group is not used and can be safely removed. However, deployed instances will have the security group attached to nodes and require the security group to be disassociated before the security group can be deleted. Because instances are deployed via autoscaling groups, we cannot simply remove the security group from the code and have those changes reflected on the instances. Instead, we have to update the code and then trigger the autoscaling groups to cycle the instances deployed so that new instances are provisioned without the security group attached. You can utilize the <code>instance_refresh</code> parameter of Autoscaling groups to force nodes to re-deploy when removing the security group since changes to launch templates automatically trigger an instance refresh. An example configuration is provided below.</li> <li>Add the following to either/or <code>self_managed_node_group_defaults</code> or the individual self-managed node group definitions:     <pre><code>create_security_group = false\ninstance_refresh = {\n  strategy = \"Rolling\"\n  preferences = {\n    min_healthy_percentage = 66\n  }\n}\n</code></pre></li> <li>It is recommended to use the <code>aws-node-termination-handler</code> while performing this update. Please refer to the <code>irsa-autoscale-refresh</code> example for usage. This will ensure that pods are safely evicted in a controlled manner to avoid service disruptions.</li> <li>Once the necessary configurations are in place, you can apply the changes which will:</li> <li>Create a new launch template (version) without the self-managed node group security group</li> <li>Replace instances based on the <code>instance_refresh</code> configuration settings</li> <li>New instances will launch without the self-managed node group security group, and prior instances will be terminated</li> <li>Once the self-managed node group has cycled, the security group will be deleted</li> </ul>"},{"location":"UPGRADE-19.0/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"<p>EKS managed node groups on <code>v18.x</code> by default create a security group that does not specify any rules. In <code>v19.x</code>, this security group has been removed due to the predominant lack of usage (most users rely on the shared node security group). While still using version <code>v18.x</code> of your module definition, remove this security group from your node groups by setting <code>create_security_group = false</code>.</p> <ul> <li>If you are currently utilizing this security group, it is recommended to create an additional security group that matches the rules/settings of the security group created by the node group, and specify that security group ID in <code>vpc_security_group_ids</code>. Once this is in place, you can proceed with the original security group removal.</li> <li>EKS managed node groups rollout changes using a rolling update strategy that can be influenced through <code>update_config</code>. No additional changes are required for removing the security group created by node groups (unlike self-managed node groups which should utilize the <code>instance_refresh</code> setting of Autoscaling groups).</li> <li>Once <code>create_security_group = false</code> has been set, you can apply the changes which will:</li> <li>Create a new launch template (version) without the EKS managed node group security group</li> <li>Replace instances based on the <code>update_config</code> configuration settings</li> <li>New instances will launch without the EKS managed node group security group, and prior instances will be terminated</li> <li> <p>Once the EKS managed node group has cycled, the security group will be deleted</p> </li> <li> <p>Once the node group security group(s) have been removed, you can update your module definition to specify the <code>v19.x</code> version of the module</p> </li> <li>Run <code>terraform init -upgrade=true</code> to update your configuration and pull in the v19 changes</li> <li>Using the documentation provided above, update your module definition to reflect the changes in the module from <code>v18.x</code> to <code>v19.x</code>. You can utilize <code>terraform plan</code> as you go to help highlight any changes that you wish to make. See below for <code>terraform state mv ...</code> commands related to the use of <code>iam_role_additional_policies</code>. If you are not providing any values to these variables, you can skip this section.</li> <li>Once you are satisfied with the changes and the <code>terraform plan</code> output, you can apply the changes to sync your infrastructure with the updated module definition (or vice versa).</li> </ul>"},{"location":"UPGRADE-19.0/#diff-of-before-v18x-vs-after-v19x","title":"Diff of Before (v18.x) vs After (v19.x)","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 18.0\"\n+  version = \"~&gt; 19.0\"\n\n  cluster_name                    = local.name\n+ cluster_endpoint_public_access  = true\n- cluster_endpoint_private_access = true # now the default\n\n  cluster_addons = {\n-   resolve_conflicts = \"OVERWRITE\" # now the default\n+   preserve          = true\n+   most_recent       = true\n\n+   timeouts = {\n+     create = \"25m\"\n+     delete = \"10m\"\n    }\n    kube-proxy = {}\n    vpc-cni = {\n-     resolve_conflicts = \"OVERWRITE\" # now the default\n    }\n  }\n\n  # Encryption key\n  create_kms_key = true\n- cluster_encryption_config = [{\n-   resources = [\"secrets\"]\n- }]\n+ cluster_encryption_config = {\n+   resources = [\"secrets\"]\n+ }\n  kms_key_deletion_window_in_days = 7\n  enable_kms_key_rotation         = true\n\n- iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+ iam_role_additional_policies = {\n+   additional = aws_iam_policy.additional.arn\n+ }\n\n  vpc_id                   = module.vpc.vpc_id\n  subnet_ids               = module.vpc.private_subnets\n  control_plane_subnet_ids = module.vpc.intra_subnets\n\n  # Extend node-to-node security group rules\n- node_security_group_ntp_ipv4_cidr_block = [\"169.254.169.123/32\"] # now the default\n  node_security_group_additional_rules = {\n-    ingress_self_ephemeral = {\n-      description = \"Node to node ephemeral ports\"\n-      protocol    = \"tcp\"\n-      from_port   = 0\n-      to_port     = 0\n-      type        = \"ingress\"\n-      self        = true\n-    }\n-    egress_all = {\n-      description      = \"Node all egress\"\n-      protocol         = \"-1\"\n-      from_port        = 0\n-      to_port          = 0\n-      type             = \"egress\"\n-      cidr_blocks      = [\"0.0.0.0/0\"]\n-      ipv6_cidr_blocks = [\"::/0\"]\n-    }\n  }\n\n  # Self-Managed Node Group(s)\n  self_managed_node_group_defaults = {\n    vpc_security_group_ids = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  self_managed_node_groups = {\n    spot = {\n      instance_type = \"m5.large\"\n      instance_market_options = {\n        market_type = \"spot\"\n      }\n\n      pre_bootstrap_user_data = &lt;&lt;-EOT\n        echo \"foo\"\n        export FOO=bar\n      EOT\n\n      bootstrap_extra_args = \"--kubelet-extra-args '--node-labels=node.kubernetes.io/lifecycle=spot'\"\n\n      post_bootstrap_user_data = &lt;&lt;-EOT\n        cd /tmp\n        sudo yum install -y https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/linux_amd64/amazon-ssm-agent.rpm\n        sudo systemctl enable amazon-ssm-agent\n        sudo systemctl start amazon-ssm-agent\n      EOT\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n    }\n  }\n\n  # EKS Managed Node Group(s)\n  eks_managed_node_group_defaults = {\n    ami_type       = \"AL2_x86_64\"\n    instance_types = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n\n    attach_cluster_primary_security_group = true\n    vpc_security_group_ids                = [aws_security_group.additional.id]\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  eks_managed_node_groups = {\n    blue = {}\n    green = {\n      min_size     = 1\n      max_size     = 10\n      desired_size = 1\n\n      instance_types = [\"t3.large\"]\n      capacity_type  = \"SPOT\"\n      labels = {\n        Environment = \"test\"\n        GithubRepo  = \"terraform-aws-eks\"\n        GithubOrg   = \"terraform-aws-modules\"\n      }\n\n      taints = {\n        dedicated = {\n          key    = \"dedicated\"\n          value  = \"gpuGroup\"\n          effect = \"NO_SCHEDULE\"\n        }\n      }\n\n      update_config = {\n        max_unavailable_percentage = 33 # or set `max_unavailable`\n      }\n\n-     create_security_group          = true\n-     security_group_name            = \"eks-managed-node-group-complete-example\"\n-     security_group_use_name_prefix = false\n-     security_group_description     = \"EKS managed node group complete example security group\"\n-     security_group_rules = {}\n-     security_group_tags = {}\n\n      tags = {\n        ExtraTag = \"example\"\n      }\n    }\n  }\n\n  # Fargate Profile(s)\n  fargate_profile_defaults = {\n-   iam_role_additional_policies = [aws_iam_policy.additional.arn]\n+   iam_role_additional_policies = {\n+     additional = aws_iam_policy.additional.arn\n+   }\n  }\n\n  fargate_profiles = {\n    default = {\n      name = \"default\"\n      selectors = [\n        {\n          namespace = \"kube-system\"\n          labels = {\n            k8s-app = \"kube-dns\"\n          }\n        },\n        {\n          namespace = \"default\"\n        }\n      ]\n\n      tags = {\n        Owner = \"test\"\n      }\n\n      timeouts = {\n        create = \"20m\"\n        delete = \"20m\"\n      }\n    }\n  }\n\n  # OIDC Identity provider\n  cluster_identity_providers = {\n    cognito = {\n      client_id      = \"702vqsrjicklgb7c5b7b50i1gc\"\n      issuer_url     = \"https://cognito-idp.us-west-2.amazonaws.com/us-west-2_re1u6bpRA\"\n      username_claim = \"email\"\n      groups_claim   = \"cognito:groups\"\n      groups_prefix  = \"gid:\"\n    }\n  }\n\n  # aws-auth configmap\n  manage_aws_auth_configmap = true\n\n  aws_auth_node_iam_role_arns_non_windows = [\n    module.eks_managed_node_group.iam_role_arn,\n    module.self_managed_node_group.iam_role_arn,\n  ]\n  aws_auth_fargate_profile_pod_execution_role_arns = [\n    module.fargate_profile.fargate_profile_pod_execution_role_arn\n  ]\n\n  aws_auth_roles = [\n    {\n      rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n      username = \"role1\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_users = [\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user1\"\n      username = \"user1\"\n      groups   = [\"system:masters\"]\n    },\n    {\n      userarn  = \"arn:aws:iam::66666666666:user/user2\"\n      username = \"user2\"\n      groups   = [\"system:masters\"]\n    },\n  ]\n\n  aws_auth_accounts = [\n    \"777777777777\",\n    \"888888888888\",\n  ]\n\n  tags = local.tags\n}\n</code></pre>"},{"location":"UPGRADE-19.0/#terraform-state-moves","title":"Terraform State Moves","text":"<p>The following Terraform state move commands are optional but recommended if you are providing additional IAM policies that are to be attached to IAM roles created by this module (cluster IAM role, node group IAM role, Fargate profile IAM role). Because the resources affected are <code>aws_iam_role_policy_attachment</code>, in theory, you could get away with simply applying the configuration and letting Terraform detach and re-attach the policies. However, during this brief period of update, you could experience permission failures as the policy is detached and re-attached, and therefore the state move route is recommended.</p> <p>Where <code>\"&lt;POLICY_ARN&gt;\"</code> is specified, this should be replaced with the full ARN of the policy, and <code>\"&lt;POLICY_MAP_KEY&gt;\"</code> should be replaced with the key used in the <code>iam_role_additional_policies</code> map for the associated policy. For example, if you have the following<code>v19.x</code> configuration:</p> <pre><code>  ...\n  # This is demonstrating the cluster IAM role additional policies\n  iam_role_additional_policies = {\n    additional = aws_iam_policy.additional.arn\n  }\n  ...\n</code></pre> <p>The associated state move command would look similar to (albeit with your correct policy ARN):</p> <pre><code>terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"arn:aws:iam::111111111111:policy/ex-complete-additional\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"additional\"]'\n</code></pre> <p>If you are not providing any additional IAM policies, no actions are required.</p>"},{"location":"UPGRADE-19.0/#cluster-iam-role","title":"Cluster IAM Role","text":"<p>Repeat for each policy provided in <code>iam_role_additional_policies</code>:</p> <pre><code>terraform state mv 'module.eks.aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#eks-managed-node-group-iam-role","title":"EKS Managed Node Group IAM Role","text":"<p>Where <code>\"&lt;NODE_GROUP_KEY&gt;\"</code> is the key used in the <code>eks_managed_node_groups</code> map for the associated node group. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>eks_managed_node_group_defaults</code> or the individual node group definitions:</p> <pre><code>terraform state mv 'module.eks.module.eks_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.eks_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#self-managed-node-group-iam-role","title":"Self-Managed Node Group IAM Role","text":"<p>Where <code>\"&lt;NODE_GROUP_KEY&gt;\"</code> is the key used in the <code>self_managed_node_groups</code> map for the associated node group. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>self_managed_node_group_defaults</code> or the individual node group definitions:</p> <pre><code>terraform state mv 'module.eks.module.self_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.self_managed_node_group[\"&lt;NODE_GROUP_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-19.0/#fargate-profile-iam-role","title":"Fargate Profile IAM Role","text":"<p>Where <code>\"&lt;FARGATE_PROFILE_KEY&gt;\"</code> is the key used in the <code>fargate_profiles</code> map for the associated profile. Repeat for each policy provided in <code>iam_role_additional_policies</code> in either/or <code>fargate_profile_defaults</code> or the individual profile definitions:</p> <pre><code>terraform state mv 'module.eks.module.fargate_profile[\"&lt;FARGATE_PROFILE_KEY&gt;\"].aws_iam_role_policy_attachment.this[\"&lt;POLICY_ARN&gt;\"]' 'module.eks.module.fargate_profile[\"&lt;FARGATE_PROFILE_KEY&gt;\"].aws_iam_role_policy_attachment.additional[\"&lt;POLICY_MAP_KEY&gt;\"]'\n</code></pre>"},{"location":"UPGRADE-20.0/","title":"Upgrade from v19.x to v20.x","text":"<p>Please consult the <code>examples</code> directory for reference example configurations. If you find a bug, please open an issue with supporting configuration to reproduce.</p>"},{"location":"UPGRADE-20.0/#list-of-backwards-incompatible-changes","title":"List of backwards incompatible changes","text":"<ul> <li>Minium supported AWS provider version increased to <code>v5.34</code></li> <li>Minimum supported Terraform version increased to <code>v1.3</code> to support Terraform state <code>moved</code> blocks as well as other advanced features</li> <li>The <code>resolve_conflicts</code> argument within the <code>cluster_addons</code> configuration has been replaced with <code>resolve_conflicts_on_create</code> and <code>resolve_conflicts_on_update</code> now that <code>resolve_conflicts</code> is deprecated</li> <li>The default/fallback value for the <code>preserve</code> argument of <code>cluster_addons</code>is now set to <code>true</code>. This has shown to be useful for users deprovisioning clusters while avoiding the situation where the CNI is deleted too early and causes resources to be left orphaned resulting in conflicts.</li> <li>The Karpenter sub-module's use of the <code>irsa</code> naming convention has been removed, along with an update to the Karpenter controller IAM policy to align with Karpenter's <code>v1beta1</code>/<code>v0.32</code> changes. Instead of referring to the role as <code>irsa</code> or <code>pod_identity</code>, its simply just an IAM role used by the Karpenter controller and there is support for use with either IRSA and/or Pod Identity (default) at this time</li> <li>The <code>aws-auth</code> ConfigMap resources have been moved to a standalone sub-module. This removes the Kubernetes provider requirement from the main module and allows for the <code>aws-auth</code> ConfigMap to be managed independently of the main module. This sub-module will be removed entirely in the next major release.</li> <li>Support for cluster access management has been added with the default authentication mode set as <code>API_AND_CONFIG_MAP</code>. This is a one way change if applied; if you wish to use <code>CONFIG_MAP</code>, you will need to set <code>authentication_mode = \"CONFIG_MAP\"</code> explicitly when upgrading.</li> <li>Karpenter EventBridge rule key <code>spot_interrupt</code> updated to correct mis-spelling (was <code>spot_interupt</code>). This will cause the rule to be replaced</li> </ul>"},{"location":"UPGRADE-20.0/#upcoming-changes-planned-in-v210","title":"\u26a0\ufe0f Upcoming Changes Planned in v21.0 \u26a0\ufe0f","text":"<p>To give users advanced notice and provide some future direction for this module, these are the following changes we will be looking to make in the next major release of this module:</p> <ol> <li>The <code>aws-auth</code> sub-module will be removed entirely from the project. Since this sub-module is captured in the v20.x releases, users can continue using it even after the module moves forward with the next major version. The long term strategy and direction is cluster access entry and to rely only on the AWS Terraform provider.</li> <li>The default value for <code>authentication_mode</code> will change to <code>API</code>. Aligning with point 1 above, this is a one way change, but users are free to specify the value of their choosing in place of this default (when the change is made). This module will proceed with an EKS API first strategy.</li> <li>The launch template and autoscaling group usage contained within the EKS managed node group and self-managed node group sub-modules *might be replaced with the <code>terraform-aws-autoscaling</code> module. At minimum, it makes sense to replace most of functionality in the self-managed node group module with this external module, but its not yet clear if there is any benefit of using it in the EKS managed node group sub-module. The interface that users interact with will stay the same, the changes will be internal to the implementation and we will do everything we can to keep the disruption to a minimum.</li> <li>The <code>platform</code> variable will be replaced and instead <code>ami_type</code> will become the standard across both self-managed node group(s) and EKS managed node group(s). As EKS expands its portfolio of supported operating systems, the <code>ami_type</code> is better suited to associate the correct user data format to the respective OS. The <code>platform</code> variable is a legacy artifact of self-managed node groups but not as descriptive as the <code>ami_type</code>, and therefore it will be removed in favor of <code>ami_type</code>.</li> </ol>"},{"location":"UPGRADE-20.0/#additional-changes","title":"Additional changes","text":""},{"location":"UPGRADE-20.0/#added","title":"Added","text":"<ul> <li>A module tag has been added to the cluster control plane</li> <li>Support for cluster access entries. The <code>bootstrap_cluster_creator_admin_permissions</code> setting on the control plane has been hardcoded to <code>false</code> since this operation is a one time operation only at cluster creation per the EKS API. Instead, users can enable/disable <code>enable_cluster_creator_admin_permissions</code> at any time to achieve the same functionality. This takes the identity that Terraform is using to make API calls and maps it into a cluster admin via an access entry. For users on existing clusters, you will need to remove the default cluster administrator that was created by EKS prior to the cluster access entry APIs - see the section <code>Removing the default cluster administrator</code> for more details.</li> <li>Support for specifying the CloudWatch log group class (standard or infrequent access)</li> <li>Native support for Windows based managed node groups similar to AL2 and Bottlerocket</li> <li>Self-managed node groups now support <code>instance_maintenance_policy</code> and have added <code>max_healthy_percentage</code>, <code>scale_in_protected_instances</code>, and <code>standby_instances</code> arguments to the <code>instance_refresh.preferences</code> block</li> </ul>"},{"location":"UPGRADE-20.0/#modified","title":"Modified","text":"<ul> <li>For <code>sts:AssumeRole</code> permissions by services, the use of dynamically looking up the DNS suffix has been replaced with the static value of <code>amazonaws.com</code>. This does not appear to change by partition and instead requires users to set this manually for non-commercial regions.</li> <li>The default value for <code>kms_key_enable_default_policy</code> has changed from <code>false</code> to <code>true</code> to align with the default behavior of the <code>aws_kms_key</code> resource</li> <li>The Karpenter default value for <code>create_instance_profile</code> has changed from <code>true</code> to <code>false</code> to align with the changes in Karpenter v0.32</li> <li>The Karpenter variable <code>create_instance_profile</code> default value has changed from <code>true</code> to <code>false</code>. Starting with Karpenter <code>v0.32.0</code>, Karpenter accepts an IAM role and creates the EC2 instance profile used by the nodes</li> </ul>"},{"location":"UPGRADE-20.0/#removed","title":"Removed","text":"<ul> <li>The <code>complete</code> example has been removed due to its redundancy with the other examples</li> <li>References to the IRSA sub-module in the IAM repository have been removed. Once https://github.com/clowdhaus/terraform-aws-eks-pod-identity has been updated and moved into the organization, the documentation here will be updated to mention the new module.</li> </ul>"},{"location":"UPGRADE-20.0/#variable-and-output-changes","title":"Variable and output changes","text":"<ol> <li> <p>Removed variables:</p> </li> <li> <p><code>cluster_iam_role_dns_suffix</code> - replaced with a static string of <code>amazonaws.com</code></p> </li> <li><code>manage_aws_auth_configmap</code></li> <li><code>create_aws_auth_configmap</code></li> <li><code>aws_auth_node_iam_role_arns_non_windows</code></li> <li><code>aws_auth_node_iam_role_arns_windows</code></li> <li><code>aws_auth_fargate_profile_pod_execution_role_arn</code></li> <li><code>aws_auth_roles</code></li> <li><code>aws_auth_users</code></li> <li> <p><code>aws_auth_accounts</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>irsa_tag_key</code></li> <li><code>irsa_tag_values</code></li> <li><code>irsa_subnet_account_id</code></li> <li><code>enable_karpenter_instance_profile_creation</code></li> </ul> </li> <li> <p>Renamed variables:</p> </li> <li> <p>Karpenter</p> <ul> <li><code>create_irsa</code> -&gt; <code>create_iam_role</code></li> <li><code>irsa_name</code> -&gt; <code>iam_role_name</code></li> <li><code>irsa_use_name_prefix</code> -&gt; <code>iam_role_name_prefix</code></li> <li><code>irsa_path</code> -&gt; <code>iam_role_path</code></li> <li><code>irsa_description</code> -&gt; <code>iam_role_description</code></li> <li><code>irsa_max_session_duration</code> -&gt; <code>iam_role_max_session_duration</code></li> <li><code>irsa_permissions_boundary_arn</code> -&gt; <code>iam_role_permissions_boundary_arn</code></li> <li><code>irsa_tags</code> -&gt; <code>iam_role_tags</code></li> <li><code>policies</code> -&gt; <code>iam_role_policies</code></li> <li><code>irsa_policy_name</code> -&gt; <code>iam_policy_name</code></li> <li><code>irsa_ssm_parameter_arns</code> -&gt; <code>ami_id_ssm_parameter_arns</code></li> <li><code>create_iam_role</code> -&gt; <code>create_node_iam_role</code></li> <li><code>iam_role_additional_policies</code> -&gt; <code>node_iam_role_additional_policies</code></li> <li><code>policies</code> -&gt; <code>iam_role_policies</code></li> <li><code>iam_role_arn</code> -&gt; <code>node_iam_role_arn</code></li> <li><code>iam_role_name</code> -&gt; <code>node_iam_role_name</code></li> <li><code>iam_role_name_prefix</code> -&gt; <code>node_iam_role_name_prefix</code></li> <li><code>iam_role_path</code> -&gt; <code>node_iam_role_path</code></li> <li><code>iam_role_description</code> -&gt; <code>node_iam_role_description</code></li> <li><code>iam_role_max_session_duration</code> -&gt; <code>node_iam_role_max_session_duration</code></li> <li><code>iam_role_permissions_boundary_arn</code> -&gt; <code>node_iam_role_permissions_boundary_arn</code></li> <li><code>iam_role_attach_cni_policy</code> -&gt; <code>node_iam_role_attach_cni_policy</code></li> <li><code>iam_role_additional_policies</code> -&gt; <code>node_iam_role_additional_policies</code></li> <li><code>iam_role_tags</code> -&gt; <code>node_iam_role_tags</code></li> </ul> </li> <li> <p>Added variables:</p> </li> <li> <p><code>create_access_entry</code></p> </li> <li><code>enable_cluster_creator_admin_permissions</code></li> <li><code>authentication_mode</code></li> <li><code>access_entries</code></li> <li> <p><code>cloudwatch_log_group_class</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>iam_policy_name</code></li> <li><code>iam_policy_use_name_prefix</code></li> <li><code>iam_policy_description</code></li> <li><code>iam_policy_path</code></li> <li><code>enable_irsa</code></li> <li><code>create_access_entry</code></li> <li><code>access_entry_type</code></li> </ul> </li> <li> <p>Self-managed node group</p> <ul> <li><code>instance_maintenance_policy</code></li> <li><code>create_access_entry</code></li> <li><code>iam_role_arn</code></li> </ul> </li> <li> <p>Removed outputs:</p> </li> <li> <p><code>aws_auth_configmap_yaml</code></p> </li> <li> <p>Renamed outputs:</p> </li> <li> <p>Karpenter</p> <ul> <li><code>irsa_name</code> -&gt; <code>iam_role_name</code></li> <li><code>irsa_arn</code> -&gt; <code>iam_role_arn</code></li> <li><code>irsa_unique_id</code> -&gt; <code>iam_role_unique_id</code></li> <li><code>role_name</code> -&gt; <code>node_iam_role_name</code></li> <li><code>role_arn</code> -&gt; <code>node_iam_role_arn</code></li> <li><code>role_unique_id</code> -&gt; <code>node_iam_role_unique_id</code></li> </ul> </li> <li> <p>Added outputs:</p> </li> <li> <p><code>access_entries</code></p> </li> <li> <p>Karpenter</p> <ul> <li><code>node_access_entry_arn</code></li> </ul> </li> <li> <p>Self-managed node group</p> <ul> <li><code>access_entry_arn</code></li> </ul> </li> </ol>"},{"location":"UPGRADE-20.0/#upgrade-migrations","title":"Upgrade Migrations","text":""},{"location":"UPGRADE-20.0/#diff-of-before-v1921-vs-after-v200","title":"Diff of Before (v19.21) vs After (v20.0)","text":"<pre><code> module \"eks\" {\n   source  = \"terraform-aws-modules/eks/aws\"\n-  version = \"~&gt; 19.21\"\n+  version = \"~&gt; 20.0\"\n\n# If you want to maintain the current default behavior of v19.x\n+  kms_key_enable_default_policy = false\n\n-   manage_aws_auth_configmap = true\n\n-   aws_auth_roles = [\n-     {\n-       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n-       username = \"role1\"\n-       groups   = [\"custom-role-group\"]\n-     },\n-   ]\n\n-   aws_auth_users = [\n-     {\n-       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n-       username = \"user1\"\n-       groups   = [\"custom-users-group\"]\n-     },\n-   ]\n}\n\n+ module \"eks_aws_auth\" {\n+   source  = \"terraform-aws-modules/eks/aws//modules/aws-auth\"\n+   version = \"~&gt; 20.0\"\n\n+   manage_aws_auth_configmap = true\n\n+   aws_auth_roles = [\n+     {\n+       rolearn  = \"arn:aws:iam::66666666666:role/role1\"\n+       username = \"role1\"\n+       groups   = [\"custom-role-group\"]\n+     },\n+   ]\n\n+   aws_auth_users = [\n+     {\n+       userarn  = \"arn:aws:iam::66666666666:user/user1\"\n+       username = \"user1\"\n+       groups   = [\"custom-users-group\"]\n+     },\n+   ]\n+ }\n</code></pre>"},{"location":"UPGRADE-20.0/#karpenter-diff-of-before-v1921-vs-after-v200","title":"Karpenter Diff of Before (v19.21) vs After (v20.0)","text":"<pre><code> module \"eks_karpenter\" {\n   source  = \"terraform-aws-modules/eks/aws//modules/karpenter\"\n-  version = \"~&gt; 19.21\"\n+  version = \"~&gt; 20.0\"\n\n# If you wish to maintain the current default behavior of v19.x\n+  enable_irsa             = true\n+  create_instance_profile = true\n\n# To avoid any resource re-creation\n+  iam_role_name          = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_role_description   = \"Karpenter IAM role for service account\"\n+  iam_policy_name        = \"KarpenterIRSA-${module.eks.cluster_name}\"\n+  iam_policy_description = \"Karpenter IAM role for service account\"\n}\n</code></pre>"},{"location":"UPGRADE-20.0/#terraform-state-moves","title":"Terraform State Moves","text":""},{"location":"UPGRADE-20.0/#authentication-mode-changes","title":"\u26a0\ufe0f Authentication Mode Changes \u26a0\ufe0f","text":"<p>Changing the <code>authentication_mode</code> is a one-way decision. See announcement blog for further details:</p> <p>Switching authentication modes on an existing cluster is a one-way operation. You can switch from CONFIG_MAP to API_AND_CONFIG_MAP. You can then switch from API_AND_CONFIG_MAP to API. You cannot revert these operations in the opposite direction. Meaning you cannot switch back to CONFIG_MAP or API_AND_CONFIG_MAP from API. And you cannot switch back to CONFIG_MAP from API_AND_CONFIG_MAP.</p> <p>[!IMPORTANT] If migrating to cluster access entries and you will NOT have any entries that remain in the <code>aws-auth</code> ConfigMap, you do not need to remove the configmap from the statefile. You can simply follow the migration guide and once access entries have been created, you can let Terraform remove/delete the <code>aws-auth</code> ConfigMap.</p> <p>If you WILL have entries that remain in the <code>aws-auth</code> ConfigMap, then you will need to remove the ConfigMap resources from the statefile to avoid any disruptions. When you add the new <code>aws-auth</code> sub-module and apply the changes, the sub-module will upsert the ConfigMap on the cluster. Provided the necessary entries are defined in that sub-module's definition, it will \"re-adopt\" the ConfigMap under Terraform's control.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-config_map","title":"authentication_mode = \"CONFIG_MAP\"","text":"<p>If using <code>authentication_mode = \"CONFIG_MAP\"</code>, before making any changes, you will first need to remove the configmap from the statefile to avoid any disruptions:</p> <pre><code>terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n</code></pre> <p>Once the configmap has been removed from the statefile, you can add the new <code>aws-auth</code> sub-module and copy the relevant definitions from the EKS module over to the new <code>aws-auth</code> sub-module definition (see before after diff above).</p> <p>[!CAUTION] You will need to add entries to the <code>aws-auth</code> sub-module for any IAM roles used by node groups and/or Fargate profiles - the module no longer handles this in the background on behalf of users.</p> <p>When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-api_and_config_map","title":"authentication_mode = \"API_AND_CONFIG_MAP\"","text":"<p>When using <code>authentication_mode = \"API_AND_CONFIG_MAP\"</code> and there are entries that will remain in the configmap (entries that cannot be replaced by cluster access entry), you will first need to update the <code>authentication_mode</code> on the cluster to <code>\"API_AND_CONFIG_MAP\"</code>. To help make this upgrade process easier, a copy of the changes defined in the <code>v20.0.0</code> PR have been captured here but with the <code>aws-auth</code> components still provided in the module. This means you get the equivalent of the <code>v20.0.0</code> module, but it still includes support for the <code>aws-auth</code> configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the <code>authentication_mode</code> has been updated to <code>\"API_AND_CONFIG_MAP\"</code>. Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.</p> <p>Once the <code>authentication_mode</code> has been updated, next you will need to remove the configmap from the statefile to avoid any disruptions:</p> <p>[!NOTE] This is only required if there are entries that will remain in the <code>aws-auth</code> ConfigMap after migrating. Otherwise, you can skip this step and let Terraform destroy the ConfigMap.</p> <pre><code>terraform state rm 'module.eks.kubernetes_config_map_v1_data.aws_auth[0]'\nterraform state rm 'module.eks.kubernetes_config_map.aws_auth[0]' # include if Terraform created the original configmap\n</code></pre>"},{"location":"UPGRADE-20.0/#i-terraform-17-users","title":"\u2139\ufe0f Terraform 1.7+ users","text":"<p>If you are using Terraform <code>v1.7+</code>, you can utilize the <code>remove</code> to facilitate both the removal of the configmap through code. You can create a fork/clone of the provided migration module and add the <code>remove</code> blocks and apply those changes before proceeding. We do not want to force users onto the bleeding edge with this module, so we have not included <code>remove</code> support at this time.</p> <p>Once the configmap has been removed from the statefile, you can add the new <code>aws-auth</code> sub-module and copy the relevant definitions from the EKS module over to the new <code>aws-auth</code> sub-module definition (see before after diff above). When you apply the changes with the new sub-module, the configmap in the cluster will get updated with the contents provided in the sub-module definition, so please be sure all of the necessary entries are added before applying the changes. In the before/example above - the configmap would remove any entries for roles used by node groups and/or Fargate Profiles, but maintain the custom entries for users and roles passed into the module definition.</p>"},{"location":"UPGRADE-20.0/#authentication_mode-api","title":"authentication_mode = \"API\"","text":"<p>In order to switch to <code>API</code> only using cluster access entry, you first need to update the <code>authentication_mode</code> on the cluster to <code>API_AND_CONFIG_MAP</code> without modifying the <code>aws-auth</code> configmap. To help make this upgrade process easier, a copy of the changes defined in the <code>v20.0.0</code> PR have been captured here but with the <code>aws-auth</code> components still provided in the module. This means you get the equivalent of the <code>v20.0.0</code> module, but it still includes support for the <code>aws-auth</code> configmap. You can follow the provided README on that interim migration module for the order of execution and return here once the <code>authentication_mode</code> has been updated to <code>\"API_AND_CONFIG_MAP\"</code>. Note - EKS automatically adds access entries for the roles used by EKS managed node groups and Fargate profiles; users do not need to do anything additional for these roles.</p> <p>Once the <code>authentication_mode</code> has been updated, you can update the <code>authentication_mode</code> on the cluster to <code>API</code> and remove the <code>aws-auth</code> configmap components.</p>"},{"location":"compute_resources/","title":"Compute Resources","text":""},{"location":"compute_resources/#table-of-contents","title":"Table of Contents","text":"<ul> <li>EKS Managed Node Groups</li> <li>Self Managed Node Groups</li> <li>Fargate Profiles</li> <li>Default Configurations</li> </ul> <p>\u2139\ufe0f Only the pertinent attributes are shown below for brevity</p>"},{"location":"compute_resources/#eks-managed-node-groups","title":"EKS Managed Node Groups","text":"<p>Refer to the EKS Managed Node Group documentation documentation for service related details.</p> <ol> <li>The module creates a custom launch template by default to ensure settings such as tags are propagated to instances. Please note that many of the customization options listed here are only available when a custom launch template is created. To use the default template provided by the AWS EKS managed node group service, disable the launch template creation by setting <code>use_custom_launch_template</code> to <code>false</code>:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    default = {\n      use_custom_launch_template = false\n    }\n  }\n</code></pre> <ol> <li>Native support for Bottlerocket OS is provided by providing the respective AMI type:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_default = {\n      use_custom_launch_template = false\n\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre> <ol> <li>Bottlerocket OS is supported in a similar manner. However, note that the user data for Bottlerocket OS uses the TOML format:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_prepend_userdata = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      bootstrap_extra_args = &lt;&lt;-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n      EOT\n    }\n  }\n</code></pre> <ol> <li>When using a custom AMI, the AWS EKS Managed Node Group service will NOT inject the necessary bootstrap script into the supplied user data. Users can elect to provide their own user data to bootstrap and connect or opt in to use the module provided user data:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    custom_ami = {\n      ami_id = \"ami-0caf35bc73450c396\"\n\n      # By default, EKS managed node groups will not append bootstrap script;\n      # this adds it back in using the default template provided by the module\n      # Note: this assumes the AMI provided is an EKS optimized AMI derivative\n      enable_bootstrap_user_data = true\n\n      pre_bootstrap_user_data = &lt;&lt;-EOT\n        export FOO=bar\n      EOT\n\n      # Because we have full control over the user data supplied, we can also run additional\n      # scripts/configuration changes after the bootstrap script has been run\n      post_bootstrap_user_data = &lt;&lt;-EOT\n        echo \"you are free little kubelet!\"\n      EOT\n    }\n  }\n</code></pre> <ol> <li>There is similar support for Bottlerocket OS:</li> </ol> <pre><code>  eks_managed_node_groups = {\n    bottlerocket_custom_ami = {\n      ami_id   = \"ami-0ff61e0bcfc81dc94\"\n      ami_type = \"BOTTLEROCKET_x86_64\"\n\n      # use module user data template to bootstrap\n      enable_bootstrap_user_data = true\n      # this will get added to the template\n      bootstrap_extra_args = &lt;&lt;-EOT\n        # extra args added\n        [settings.kernel]\n        lockdown = \"integrity\"\n\n        [settings.kubernetes.node-labels]\n        \"label1\" = \"foo\"\n        \"label2\" = \"bar\"\n\n        [settings.kubernetes.node-taints]\n        \"dedicated\" = \"experimental:PreferNoSchedule\"\n        \"special\" = \"true:NoSchedule\"\n      EOT\n    }\n  }\n</code></pre> <p>See the <code>examples/eks-managed-node-group/</code> example for a working example of various configurations.</p>"},{"location":"compute_resources/#self-managed-node-groups","title":"Self Managed Node Groups","text":"<p>Refer to the Self Managed Node Group documentation documentation for service related details.</p> <ol> <li>The <code>self-managed-node-group</code> uses the latest AWS EKS Optimized AMI (Linux) for the given Kubernetes version by default:</li> </ol> <pre><code>  cluster_version = \"1.31\"\n\n  # This self managed node group will use the latest AWS EKS Optimized AMI for Kubernetes 1.27\n  self_managed_node_groups = {\n    default = {}\n  }\n</code></pre> <ol> <li>To use Bottlerocket, specify the <code>ami_type</code> as one of the respective <code>\"BOTTLEROCKET_*\" types</code> and supply a Bottlerocket OS AMI:</li> </ol> <pre><code>  cluster_version = \"1.31\"\n\n  self_managed_node_groups = {\n    bottlerocket = {\n      ami_id   = data.aws_ami.bottlerocket_ami.id\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre> <p>See the <code>examples/self-managed-node-group/</code> example for a working example of various configurations.</p>"},{"location":"compute_resources/#fargate-profiles","title":"Fargate Profiles","text":"<p>Fargate profiles are straightforward to use and therefore no further details are provided here. See the <code>tests/fargate-profile/</code> tests for a working example of various configurations.</p>"},{"location":"compute_resources/#default-configurations","title":"Default Configurations","text":"<p>Each type of compute resource (EKS managed node group, self managed node group, or Fargate profile) provides the option for users to specify a default configuration. These default configurations can be overridden from within the compute resource's individual definition. The order of precedence for configurations (from highest to least precedence):</p> <ul> <li>Compute resource individual configuration</li> <li>Compute resource family default configuration (<code>eks_managed_node_group_defaults</code>, <code>self_managed_node_group_defaults</code>, <code>fargate_profile_defaults</code>)<ul> <li>Module default configuration (see <code>variables.tf</code> and <code>node_groups.tf</code>)</li> </ul> </li> </ul> <p>For example, the following creates 4 AWS EKS Managed Node Groups:</p> <pre><code>  eks_managed_node_group_defaults = {\n    ami_type               = \"AL2_x86_64\"\n    disk_size              = 50\n    instance_types         = [\"m6i.large\", \"m5.large\", \"m5n.large\", \"m5zn.large\"]\n  }\n\n  eks_managed_node_groups = {\n    # Uses module default configurations overridden by configuration above\n    default = {}\n\n    # This further overrides the instance types used\n    compute = {\n      instance_types = [\"c5.large\", \"c6i.large\", \"c6d.large\"]\n    }\n\n    # This further overrides the instance types and disk size used\n    persistent = {\n      disk_size = 1024\n      instance_types = [\"r5.xlarge\", \"r6i.xlarge\", \"r5b.xlarge\"]\n    }\n\n    # This overrides the OS used\n    bottlerocket = {\n      ami_type = \"BOTTLEROCKET_x86_64\"\n    }\n  }\n</code></pre>"},{"location":"faq/","title":"Frequently Asked Questions","text":"<ul> <li>Setting <code>disk_size</code> or <code>remote_access</code> does not make any changes</li> <li>I received an error: <code>expect exactly one securityGroup tagged with kubernetes.io/cluster/&lt;NAME&gt; ...</code></li> <li>Why are nodes not being registered?</li> <li>Why are there no changes when a node group's <code>desired_size</code> is modified?</li> <li>How do I access compute resource attributes?</li> <li>What add-ons are available?</li> <li>What configuration values are available for an add-on?</li> </ul>"},{"location":"faq/#setting-disk_size-or-remote_access-does-not-make-any-changes","title":"Setting <code>disk_size</code> or <code>remote_access</code> does not make any changes","text":"<p><code>disk_size</code>, and <code>remote_access</code> can only be set when using the EKS managed node group default launch template. This module defaults to providing a custom launch template to allow for custom security groups, tag propagation, etc. If you wish to forgo the custom launch template route, you can set <code>use_custom_launch_template = false</code> and then you can set <code>disk_size</code> and <code>remote_access</code>.</p>"},{"location":"faq/#i-received-an-error-expect-exactly-one-securitygroup-tagged-with-kubernetesioclustername","title":"I received an error: <code>expect exactly one securityGroup tagged with kubernetes.io/cluster/&lt;NAME&gt; ...</code>","text":"<p>By default, EKS creates a cluster primary security group that is created outside of the module and the EKS service adds the tag <code>{ \"kubernetes.io/cluster/&lt;CLUSTER_NAME&gt;\" = \"owned\" }</code>. This on its own does not cause any conflicts for addons such as the AWS Load Balancer Controller until users decide to attach both the cluster primary security group and the shared node security group created by the module (by setting <code>attach_cluster_primary_security_group = true</code>). The issue is not with having multiple security groups in your account with this tag key:value combination, but having multiple security groups with this tag key:value combination attached to nodes in the same cluster. There are a few ways to resolve this depending on your use case/intentions:</p> <p>\u26a0\ufe0f <code>&lt;CLUSTER_NAME&gt;</code> below needs to be replaced with the name of your cluster</p> <ol> <li>If you want to use the cluster primary security group, you can disable the creation of the shared node security group with:</li> </ol> <pre><code>  create_node_security_group            = false # default is true\n  attach_cluster_primary_security_group = true # default is false\n</code></pre> <ol> <li>By not attaching the cluster primary security group. The cluster primary security group has quite broad access and the module has instead provided a security group with the minimum amount of access to launch an empty EKS cluster successfully and users are encouraged to open up access when necessary to support their workload.</li> </ol> <pre><code>  attach_cluster_primary_security_group = false # this is the default for the module\n</code></pre> <p>In theory, if you are attaching the cluster primary security group, you shouldn't need to use the shared node security group created by the module. However, this is left up to users to decide for their requirements and use case.</p> <p>If you choose to use Custom Networking, make sure to only attach the security groups matching your choice above in your ENIConfig resources. This will ensure you avoid redundant tags.</p>"},{"location":"faq/#why-are-nodes-not-being-registered","title":"Why are nodes not being registered?","text":"<p>Nodes not being able to register with the EKS control plane is generally due to networking mis-configurations.</p> <ol> <li>At least one of the cluster endpoints (public or private) must be enabled.</li> </ol> <p>If you require a public endpoint, setting up both (public and private) and restricting the public endpoint via setting <code>cluster_endpoint_public_access_cidrs</code> is recommended. More info regarding communication with an endpoint is available here.</p> <ol> <li> <p>Nodes need to be able to contact the EKS cluster endpoint. By default, the module only creates a public endpoint. To access the endpoint, the nodes need outgoing internet access:</p> </li> <li> <p>Nodes in private subnets: via a NAT gateway or instance along with the appropriate routing rules</p> </li> <li>Nodes in public subnets: ensure that nodes are launched with public IPs (enable through either the module here or your subnet setting defaults)</li> </ol> <p>Important: If you apply only the public endpoint and configure the <code>cluster_endpoint_public_access_cidrs</code> to restrict access, know that EKS nodes will also use the public endpoint and you must allow access to the endpoint. If not, then your nodes will fail to work correctly.</p> <ol> <li> <p>The private endpoint can also be enabled by setting <code>cluster_endpoint_private_access = true</code>. Ensure that VPC DNS resolution and hostnames are also enabled for your VPC when the private endpoint is enabled.</p> </li> <li> <p>Nodes need to be able to connect to other AWS services to function (download container images, make API calls to assume roles, etc.). If for some reason you cannot enable public internet access for nodes you can add VPC endpoints to the relevant services: EC2 API, ECR API, ECR DKR and S3.</p> </li> </ol>"},{"location":"faq/#why-are-there-no-changes-when-a-node-groups-desired_size-is-modified","title":"Why are there no changes when a node group's <code>desired_size</code> is modified?","text":"<p>The module is configured to ignore this value. Unfortunately, Terraform does not support variables within the <code>lifecycle</code> block. The setting is ignored to allow autoscaling via controllers such as cluster autoscaler or Karpenter to work properly and without interference by Terraform. Changing the desired count must be handled outside of Terraform once the node group is created.</p>"},{"location":"faq/#how-do-i-access-compute-resource-attributes","title":"How do I access compute resource attributes?","text":"<p>Examples of accessing the attributes of the compute resource(s) created by the root module are shown below. Note - the assumption is that your cluster module definition is named <code>eks</code> as in <code>module \"eks\" { ... }</code>:</p> <ul> <li>EKS Managed Node Group attributes</li> </ul> <pre><code>eks_managed_role_arns = [for group in module.eks_managed_node_group : group.iam_role_arn]\n</code></pre> <ul> <li>Self Managed Node Group attributes</li> </ul> <pre><code>self_managed_role_arns = [for group in module.self_managed_node_group : group.iam_role_arn]\n</code></pre> <ul> <li>Fargate Profile attributes</li> </ul> <pre><code>fargate_profile_pod_execution_role_arns = [for group in module.fargate_profile : group.fargate_profile_pod_execution_role_arn]\n</code></pre>"},{"location":"faq/#what-add-ons-are-available","title":"What add-ons are available?","text":"<p>The available EKS add-ons can be found here. You can also retrieve the available addons from the API using:</p> <pre><code>aws eks describe-addon-versions --query 'addons[*].addonName'\n</code></pre>"},{"location":"faq/#what-configuration-values-are-available-for-an-add-on","title":"What configuration values are available for an add-on?","text":"<p>You can retrieve the configuration value schema for a given addon using the following command:</p> <pre><code>aws eks describe-addon-configuration --addon-name &lt;value&gt; --addon-version &lt;value&gt; --query 'configurationSchema' --output text | jq\n</code></pre> <p>For example:</p> <pre><code>aws eks describe-addon-configuration --addon-name coredns --addon-version v1.11.1-eksbuild.8 --query 'configurationSchema' --output text | jq\n</code></pre> <p>Returns (at the time of writing):</p> <pre><code>{\n  \"$ref\": \"#/definitions/Coredns\",\n  \"$schema\": \"http://json-schema.org/draft-06/schema#\",\n  \"definitions\": {\n    \"Coredns\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"affinity\": {\n          \"default\": {\n            \"affinity\": {\n              \"nodeAffinity\": {\n                \"requiredDuringSchedulingIgnoredDuringExecution\": {\n                  \"nodeSelectorTerms\": [\n                    {\n                      \"matchExpressions\": [\n                        {\n                          \"key\": \"kubernetes.io/os\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"linux\"\n                          ]\n                        },\n                        {\n                          \"key\": \"kubernetes.io/arch\",\n                          \"operator\": \"In\",\n                          \"values\": [\n                            \"amd64\",\n                            \"arm64\"\n                          ]\n                        }\n                      ]\n                    }\n                  ]\n                }\n              },\n              \"podAntiAffinity\": {\n                \"preferredDuringSchedulingIgnoredDuringExecution\": [\n                  {\n                    \"podAffinityTerm\": {\n                      \"labelSelector\": {\n                        \"matchExpressions\": [\n                          {\n                            \"key\": \"k8s-app\",\n                            \"operator\": \"In\",\n                            \"values\": [\n                              \"kube-dns\"\n                            ]\n                          }\n                        ]\n                      },\n                      \"topologyKey\": \"kubernetes.io/hostname\"\n                    },\n                    \"weight\": 100\n                  }\n                ]\n              }\n            }\n          },\n          \"description\": \"Affinity of the coredns pods\",\n          \"type\": [\n            \"object\",\n            \"null\"\n          ]\n        },\n        \"computeType\": {\n          \"type\": \"string\"\n        },\n        \"corefile\": {\n          \"description\": \"Entire corefile contents to use with installation\",\n          \"type\": \"string\"\n        },\n        \"nodeSelector\": {\n          \"additionalProperties\": {\n            \"type\": \"string\"\n          },\n          \"type\": \"object\"\n        },\n        \"podAnnotations\": {\n          \"properties\": {},\n          \"title\": \"The podAnnotations Schema\",\n          \"type\": \"object\"\n        },\n        \"podDisruptionBudget\": {\n          \"description\": \"podDisruptionBudget configurations\",\n          \"enabled\": {\n            \"default\": true,\n            \"description\": \"the option to enable managed PDB\",\n            \"type\": \"boolean\"\n          },\n          \"maxUnavailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"default\": 1,\n            \"description\": \"minAvailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"minAvailable\": {\n            \"anyOf\": [\n              {\n                \"pattern\": \".*%$\",\n                \"type\": \"string\"\n              },\n              {\n                \"type\": \"integer\"\n              }\n            ],\n            \"description\": \"maxUnavailable value for managed PDB, can be either string or integer; if it's string, should end with %\"\n          },\n          \"type\": \"object\"\n        },\n        \"podLabels\": {\n          \"properties\": {},\n          \"title\": \"The podLabels Schema\",\n          \"type\": \"object\"\n        },\n        \"replicaCount\": {\n          \"type\": \"integer\"\n        },\n        \"resources\": {\n          \"$ref\": \"#/definitions/Resources\"\n        },\n        \"tolerations\": {\n          \"default\": [\n            {\n              \"key\": \"CriticalAddonsOnly\",\n              \"operator\": \"Exists\"\n            },\n            {\n              \"effect\": \"NoSchedule\",\n              \"key\": \"node-role.kubernetes.io/control-plane\"\n            }\n          ],\n          \"description\": \"Tolerations of the coredns pod\",\n          \"items\": {\n            \"type\": \"object\"\n          },\n          \"type\": \"array\"\n        },\n        \"topologySpreadConstraints\": {\n          \"description\": \"The coredns pod topology spread constraints\",\n          \"type\": \"array\"\n        }\n      },\n      \"title\": \"Coredns\",\n      \"type\": \"object\"\n    },\n    \"Limits\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"cpu\": {\n          \"type\": \"string\"\n        },\n        \"memory\": {\n          \"type\": \"string\"\n        }\n      },\n      \"title\": \"Limits\",\n      \"type\": \"object\"\n    },\n    \"Resources\": {\n      \"additionalProperties\": false,\n      \"properties\": {\n        \"limits\": {\n          \"$ref\": \"#/definitions/Limits\"\n        },\n        \"requests\": {\n          \"$ref\": \"#/definitions/Limits\"\n        }\n      },\n      \"title\": \"Resources\",\n      \"type\": \"object\"\n    }\n  }\n}\n</code></pre> <p>[!NOTE] The available configuration values will vary between add-on versions, typically more configuration values will be added in later versions as functionality is enabled by EKS.</p>"},{"location":"local/","title":"Local Development","text":""},{"location":"local/#documentation-site","title":"Documentation Site","text":"<p>In order to run the documentation site locally, you will need to have the following installed locally:</p> <ul> <li>Python 3.x</li> <li>mkdocs</li> <li>The following pip packages for mkdocs (i.e. - <code>pip install ...</code>)<ul> <li><code>mkdocs-material</code></li> <li><code>mkdocs-include-markdown-plugin</code></li> <li><code>mkdocs-awesome-pages-plugin</code></li> </ul> </li> </ul> <p>To run the documentation site locally, run the following command from the root of the repository:</p> <pre><code>mkdocs serve\n</code></pre> <p>Opening the documentation at the link posted in the terminal output (i.e. - http://127.0.0.1:8000/terraform-aws-eks/)</p>"},{"location":"network_connectivity/","title":"Network Connectivity","text":""},{"location":"network_connectivity/#cluster-endpoint","title":"Cluster Endpoint","text":""},{"location":"network_connectivity/#public-endpoint-w-restricted-cidrs","title":"Public Endpoint w/ Restricted CIDRs","text":"<p>When restricting the clusters public endpoint to only the CIDRs specified by users, it is recommended that you also enable the private endpoint, or ensure that the CIDR blocks that you specify include the addresses that nodes and Fargate pods (if you use them) access the public endpoint from.</p> <p>Please refer to the AWS documentation for further information</p>"},{"location":"network_connectivity/#security-groups","title":"Security Groups","text":"<ul> <li>Cluster Security Group</li> <li>This module by default creates a cluster security group (\"additional\" security group when viewed from the console) in addition to the default security group created by the AWS EKS service. This \"additional\" security group allows users to customize inbound and outbound rules via the module as they see fit<ul> <li>The default inbound/outbound rules provided by the module are derived from the AWS minimum recommendations in addition to NTP and HTTPS public internet egress rules (without, these show up in VPC flow logs as rejects - they are used for clock sync and downloading necessary packages/updates)</li> <li>The minimum inbound/outbound rules are provided for cluster and node creation to succeed without errors, but users will most likely need to add the necessary port and protocol for node-to-node communication (this is user specific based on how nodes are configured to communicate across the cluster)</li> <li>Users have the ability to opt out of the security group creation and instead provide their own externally created security group if so desired</li> <li>The security group that is created is designed to handle the bare minimum communication necessary between the control plane and the nodes, as well as any external egress to allow the cluster to successfully launch without error</li> </ul> </li> <li>Users also have the option to supply additional, externally created security groups to the cluster as well via the <code>cluster_additional_security_group_ids</code> variable</li> <li> <p>Lastly, users are able to opt in to attaching the primary security group automatically created by the EKS service by setting <code>attach_cluster_primary_security_group</code> = <code>true</code> from the root module for the respective node group (or set it within the node group defaults). This security group is not managed by the module; it is created by the EKS service. It permits all traffic within the domain of the security group as well as all egress traffic to the internet.</p> </li> <li> <p>Node Group Security Group(s)</p> </li> <li>Users have the option to assign their own externally created security group(s) to the node group via the <code>vpc_security_group_ids</code> variable</li> </ul> <p>See the example snippet below which adds additional security group rules to the cluster security group as well as the shared node security group (for node-to-node access). Users can use this extensibility to open up network access as they see fit using the security groups provided by the module:</p> <p><pre><code>  ...\n  # Extend cluster security group rules\n  cluster_security_group_additional_rules = {\n    egress_nodes_ephemeral_ports_tcp = {\n      description                = \"To node 1025-65535\"\n      protocol                   = \"tcp\"\n      from_port                  = 1025\n      to_port                    = 65535\n      type                       = \"egress\"\n      source_node_security_group = true\n    }\n  }\n\n  # Extend node-to-node security group rules\n  node_security_group_additional_rules = {\n    ingress_self_all = {\n      description = \"Node to node all ports/protocols\"\n      protocol    = \"-1\"\n      from_port   = 0\n      to_port     = 0\n      type        = \"ingress\"\n      self        = true\n    }\n    egress_all = {\n      description      = \"Node all egress\"\n      protocol         = \"-1\"\n      from_port        = 0\n      to_port          = 0\n      type             = \"egress\"\n      cidr_blocks      = [\"0.0.0.0/0\"]\n      ipv6_cidr_blocks = [\"::/0\"]\n    }\n  }\n  ...\n</code></pre> The security groups created by this module are depicted in the image shown below along with their default inbound/outbound rules:</p> <p> </p>"},{"location":"user_data/","title":"User Data &amp; Bootstrapping","text":"<p>Users can see the various methods of using and providing user data through the user data tests as well more detailed information on the design and possible configurations via the user data module itself</p>"},{"location":"user_data/#summary","title":"Summary","text":"<ul> <li>AWS EKS Managed Node Groups</li> <li>By default, any supplied user data is pre-pended to the user data supplied by the EKS Managed Node Group service</li> <li>If users supply an <code>ami_id</code>, the service no longers supplies user data to bootstrap nodes; users can enable <code>enable_bootstrap_user_data</code> and use the module provided user data template, or provide their own user data template</li> <li>AMI types of <code>BOTTLEROCKET_*</code>, user data must be in TOML format</li> <li>AMI types of <code>WINDOWS_*</code>, user data must be in powershell/PS1 script format</li> <li>Self Managed Node Groups</li> <li><code>AL2_x86_64</code> AMI type (default) -&gt; the user data template (bash/shell script) provided by the module is used as the default; users are able to provide their own user data template</li> <li><code>BOTTLEROCKET_*</code> AMI types -&gt; the user data template (TOML file) provided by the module is used as the default; users are able to provide their own user data template</li> <li><code>WINDOWS_*</code> AMI types -&gt; the user data template (powershell/PS1 script) provided by the module is used as the default; users are able to provide their own user data template</li> </ul> <p>The templates provided by the module can be found under the templates directory</p>"},{"location":"user_data/#eks-managed-node-group","title":"EKS Managed Node Group","text":"<p>When using an EKS managed node group, users have 2 primary routes for interacting with the bootstrap user data:</p> <ol> <li> <p>If a value for <code>ami_id</code> is not provided, users can supply additional user data that is pre-pended before the EKS Managed Node Group bootstrap user data. You can read more about this process from the AWS supplied documentation</p> </li> <li> <p>Users can use the following variables to facilitate this process:</p> <pre><code>pre_bootstrap_user_data = \"...\"\n</code></pre> </li> <li> <p>If a custom AMI is used, then per the AWS documentation, users will need to supply the necessary user data to bootstrap and register nodes with the cluster when launched. There are two routes to facilitate this bootstrapping process:</p> </li> <li>If the AMI used is a derivative of the AWS EKS Optimized AMI , users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:<ul> <li>Users can use the following variables to facilitate this process:    <pre><code>enable_bootstrap_user_data = true # to opt in to using the module supplied bootstrap user data template\npre_bootstrap_user_data    = \"...\"\nbootstrap_extra_args       = \"...\"\npost_bootstrap_user_data   = \"...\"\n</code></pre></li> </ul> </li> <li>If the AMI is NOT an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the <code>templatefile()</code> for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.<ul> <li>Users can use the following variables to facilitate this process:    <pre><code>user_data_template_path  = \"./your/user_data.sh\" # user supplied bootstrap user data template\npre_bootstrap_user_data  = \"...\"\nbootstrap_extra_args     = \"...\"\npost_bootstrap_user_data = \"...\"\n</code></pre></li> </ul> </li> </ol> \u2139\ufe0f When using bottlerocket, the supplied user data (TOML format) is merged in with the values supplied by EKS. Therefore, <code>pre_bootstrap_user_data</code> and <code>post_bootstrap_user_data</code> are not valid since the bottlerocket OS handles when various settings are applied. If you wish to supply additional configuration settings when using bottlerocket, supply them via the <code>bootstrap_extra_args</code> variable. For the <code>AL2_*</code> AMI types, <code>bootstrap_extra_args</code> are settings that will be supplied to the AWS EKS Optimized AMI bootstrap script such as kubelet extra args, etc. See the bottlerocket GitHub repository documentation for more details on what settings can be supplied via the <code>bootstrap_extra_args</code> variable."},{"location":"user_data/#self-managed-node-group","title":"Self Managed Node Group","text":"<p>Self managed node groups require users to provide the necessary bootstrap user data. Users can elect to use the user data template provided by the module for their respective AMI type or provide their own user data template for rendering by the module.</p> <ul> <li>If the AMI used is a derivative of the AWS EKS Optimized AMI , users can opt in to using a template provided by the module that provides the minimum necessary configuration to bootstrap the node when launched:</li> <li>Users can use the following variables to facilitate this process:     <pre><code>enable_bootstrap_user_data = true # to opt in to using the module supplied bootstrap user data template\npre_bootstrap_user_data    = \"...\"\nbootstrap_extra_args       = \"...\"\npost_bootstrap_user_data   = \"...\"\n</code></pre></li> <li>If the AMI is NOT an AWS EKS Optimized AMI derivative, or if users wish to have more control over the user data that is supplied to the node when launched, users have the ability to supply their own user data template that will be rendered instead of the module supplied template. Note - only the variables that are supplied to the <code>templatefile()</code> for the respective AMI type are available for use in the supplied template, otherwise users will need to pre-render/pre-populate the template before supplying the final template to the module for rendering as user data.<ul> <li>Users can use the following variables to facilitate this process:   <pre><code>user_data_template_path  = \"./your/user_data.sh\" # user supplied bootstrap user data template\npre_bootstrap_user_data  = \"...\"\nbootstrap_extra_args     = \"...\"\npost_bootstrap_user_data = \"...\"\n</code></pre></li> </ul> </li> </ul>"},{"location":"user_data/#logic-diagram","title":"Logic Diagram","text":"<p>The rough flow of logic that is encapsulated within the <code>_user_data</code> module can be represented by the following diagram to better highlight the various manners in which user data can be populated.</p> <p> </p>"}]}
\ No newline at end of file
diff --git a/sitemap.xml b/sitemap.xml
index fe8723e7..91344e9d 100644
--- a/sitemap.xml
+++ b/sitemap.xml
@@ -2,42 +2,42 @@
 <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/UPGRADE-17.0/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/UPGRADE-18.0/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/UPGRADE-19.0/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/UPGRADE-20.0/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/compute_resources/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/faq/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/local/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/network_connectivity/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
     <url>
          <loc>https://terraform-aws-modules/terraform-aws-eks/user_data/</loc>
-         <lastmod>2024-10-12</lastmod>
+         <lastmod>2024-10-22</lastmod>
     </url>
 </urlset>
\ No newline at end of file
diff --git a/sitemap.xml.gz b/sitemap.xml.gz
index 60adf026213ac5df5f2fc6a46969f1dafa362604..1c7c42b1294265322fa9edbe4355d09798b10478 100644
GIT binary patch
delta 237
zcmV<J022S10+|8_ABzYG0P7Z!2OWPb3QdabP-xn<lx8+Uz9b^H<mk?E`|m3!p_GoL
zVBaF$<K4S&5X#pdZIhfKp!Kd`St?lq&iLB8eZjWx&wR<&<s$DxgFs0%J1p3NsCy7%
z7={!rL907Veb@^$huEOFi9)6;RxXlQW5D%xsl4%~98)mH5QLz2KD1m9$Xgg+_YL4L
z6^<x!F;4vq(MZu=Dyi;xCb?3PyB$G|l5?OjzU_K~DggRE7?{CR-s#iy5OGE~(_=WG
n!G~jIymMg4o-KV&Pt_xYs@A0cj&eETw_Sb#Uib;t<^=!%3nX!4

delta 237
zcmV<J022S10+|8_ABzYG0K5s22OWQG3QdabP-xn<lx8+Uz9b@+<mk?E`|m3!p_GoL
zVBaF$<K4S&5GvLmwoT3u(0Es}ER`$)r+s7GzGU0?XTD_XYEkr|MW7^_9hU4s)IA6>
z3`2^Bz^V>YANB(EAvP#(qLAr|Rf{Cn7;wE^s%X8g#uSV(1R<!M50<L|x%C)L-va(p
z;fSIT<J8X(g%s^oF7rE{NuK4AyB$GIF6Tg_z3qB}Isp1UXqdrM-l@~{5OGGe(_=WG
n!G~k5y>p<+oDF?WPt_xYx>2P5j&eETw_SY!sD{2)<^=!%d4h4?

diff --git a/user_data/index.html b/user_data/index.html
index c0976973..54b2c183 100644
--- a/user_data/index.html
+++ b/user_data/index.html
@@ -386,7 +386,7 @@
 
 
 <h1 id="user-data-bootstrapping">User Data &amp; Bootstrapping<a class="headerlink" href="#user-data-bootstrapping" title="Permanent link">&para;</a></h1>
-<p>Users can see the various methods of using and providing user data through the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/examples/user_data">user data examples</a> as well more detailed information on the design and possible configurations via the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/modules/_user_data">user data module itself</a></p>
+<p>Users can see the various methods of using and providing user data through the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/tests/user-data">user data tests</a> as well more detailed information on the design and possible configurations via the <a href="https://github.com/terraform-aws-modules/terraform-aws-eks/tree/master/modules/_user_data">user data module itself</a></p>
 <h2 id="summary">Summary<a class="headerlink" href="#summary" title="Permanent link">&para;</a></h2>
 <ul>
 <li>AWS EKS Managed Node Groups</li>