AWS eks: Restructure EKS AI/ML cluster setup guide; add ALB load balancing step
Summary
Rewrites the intro and infrastructure list for the EKS AI/ML cluster setup guide, splitting GPU node provisioning into a separate GPU NodePool page, adding an ALB-backed `alb` IngressClass for Grafana and inference endpoints, and reformatting the step table and CLI/Terraform links.
Security assessment
The change is editorial restructuring of an AI/ML cluster setup guide. The only security-adjacent mention is an ALB IngressClass fronting Grafana and the inference endpoint, but it is described as infrastructure setup, not as a security control, hardening step, or fix for a specific weakness. No vulnerability, CVE, credential, or IAM boundary change is referenced.
Evidence
+ * **Load balancing** — An `alb` `IngressClass` backed by an AWS Application Load Balancer, which fronts traffic for Grafana and, later, the inference endpoint.
Diff
diff --git a/eks/latest/userguide/ml-cluster-setup.md b/eks/latest/userguide/ml-cluster-setup.md index aa3e1e8..e71700c 100644 --- a//eks/latest/userguide/ml-cluster-setup.md +++ b//eks/latest/userguide/ml-cluster-setup.md @@ -19 +19,3 @@ To contribute to this user guide, choose the **Edit this page on GitHub** link t -This section guides you through creating an Amazon EKS cluster ready to run inference workloads, including the compute with GPUs, monitoring stack, and Amazon S3 storage for model weights, along with the necessary AWS IAM permissions. +This section sets up the supporting infrastructure that AI/ML training and inference workloads need on Amazon EKS. This includes the EKS cluster itself, load balancing for in-cluster endpoints, a monitoring stack, Amazon S3 storage for model weights, and the AWS IAM permissions that connect them. + +You provision GPU compute afterward, in [Create a GPU NodePool for AI/ML inference](./ml-inference-gpu-nodepool.html). Setting up this infrastructure first means that monitoring and storage are already in place when GPU nodes join the cluster, so the GPU components start reporting metrics as soon as the nodes are ready. @@ -25 +27,3 @@ The setup creates the following infrastructure: - * **EKS cluster with GPU-enabled nodes** — A Karpenter-managed NodePool that dynamically provisions G-family GPU instances using Spot capacity with On-Demand fallback. + * **EKS cluster** — An EKS cluster with the cluster-level components that GPU workloads depend on, including EKS Auto Mode or Karpenter for node auto-scaling and the NVIDIA device plugin. + + * **Load balancing** — An `alb` `IngressClass` backed by an AWS Application Load Balancer, which fronts traffic for Grafana and, later, the inference endpoint. @@ -36 +40 @@ The setup creates the following infrastructure: -The guide provides two paths for setting up your cluster. Choose one and follow it consistently through all steps. +The guide provides two paths for setting up your cluster, which differ in how much of the node lifecycle you manage yourself. Choose one and follow it consistently through all steps, including the GPU NodePool you create later. @@ -40 +44 @@ The guide provides two paths for setting up your cluster. Choose one and follow - * **Self-managed Karpenter** — You install and configure each component explicitly: Karpenter via `eksctl`, automatic node repair through its feature gate, the EKS node monitoring agent as an EKS add-on, and the NVIDIA device plugin via Helm. You also create a custom `EC2NodeClass` that uses the EKS-optimized NVIDIA AL2023 AMIs and configures SOCI. + * **Self-managed Karpenter** — You install and configure each component explicitly: Karpenter by using `eksctl`, automatic node repair through its feature gate, the EKS node monitoring agent as an EKS add-on, and the NVIDIA device plugin by using Helm. You also create a custom `EC2NodeClass` that uses the EKS-optimized NVIDIA AL2023 AMIs and configures SOCI. @@ -50,3 +54 @@ Step | Description -**Create dynamically provisioned GPU nodes** | Define a dynamic GPU NodePool that provisions G-family GPU instances as workloads are scheduled. -**Test with a sample pod** | Validate the setup end-to-end by running an `nvidia-smi` pod that triggers Karpenter to provision a GPU-enabled node. -**Add reserved capacity (optional)** | Attach an On-Demand Capacity Reservation (ODCR) to your NodeClass for reserved-first with Spot/On-Demand fallback. +**Set up load balancing** | Deploy an Application Load Balancer (ALB) to front traffic for Grafana and the inference endpoint. @@ -55,0 +58,2 @@ Step | Description +Once you finish setting up your cluster, the next step is GPU compute. For instructions on creating the GPU NodePool that dynamically provisions GPU instances, see [Create a GPU NodePool for AI/ML inference](./ml-inference-gpu-nodepool.html). + @@ -58 +62,6 @@ Step | Description -For step-by-step instructions using the AWS CLI, see [Set up Amazon EKS cluster for AI/ML workloads using CLIs](./ml-cluster-setup-cli.html). For step-by-step instructions using Terraform, see [Set up Amazon EKS cluster for AI/ML workloads using Terraform](./ml-cluster-setup-tf.html). + * For step-by-step instructions using the AWS CLI, see [Set up Amazon EKS cluster for AI/ML workloads using CLIs](./ml-cluster-setup-cli.html). + + * For step-by-step instructions using Terraform, see [Set up Amazon EKS cluster for AI/ML workloads using Terraform](./ml-cluster-setup-tf.html). + + +