AWS eks: Update GPU NodePool instance constraints and driver notes
Summary
Revises EKS Auto Mode and Karpenter NodePool examples to exclude the g7 instance family via instance-family NotIn instead of capping instance generation, and updates explanatory text about NVIDIA driver versions and AMI differences between the two paths.
Security assessment
The change adjusts Kubernetes NodePool scheduling constraints (instance-family NotIn g7) and clarifies NVIDIA driver/AMI compatibility for GPU instances. This is operational/compatibility guidance with no authentication, encryption, credential, or vulnerability content.
Evidence
+ - key: eks.amazonaws.com/instance-family
Diff
diff --git a/eks/latest/userguide/ml-cluster-setup-cli.md b/eks/latest/userguide/ml-cluster-setup-cli.md index 398a693d3..b1cee2ad8 100644 --- a//eks/latest/userguide/ml-cluster-setup-cli.md +++ b//eks/latest/userguide/ml-cluster-setup-cli.md @@ -509 +509 @@ Both paths now have an `alb` `IngressClass`. The Grafana and inference sections -Define a NodePool that dynamically provisions G-family GPU instances with a generation greater than 4 and less than 7, using Spot capacity with On-Demand as a fallback. The EKS Auto Mode and Karpenter paths both use the same NodePool API with the only difference being the NodeClass it points to. In EKS Auto Mode, the bundled `default` NodeClass already selects the right AMI and configures SOCI parallel pull, so the NodePool is the only object you create. In self-managed Karpenter, you also need a custom `EC2NodeClass` that pins the AMI and tunes SOCI. +Define a NodePool that dynamically provisions G-family GPU instances with a generation greater than 4, using Spot capacity with On-Demand as a fallback. The EKS Auto Mode and Karpenter paths both use the same NodePool API with the only difference being the NodeClass it points to. In EKS Auto Mode, the bundled `default` NodeClass already selects the right AMI and configures SOCI parallel pull, so the NodePool is the only object you create. In self-managed Karpenter, you also need a custom `EC2NodeClass` that pins the AMI and tunes SOCI. Because the two paths use different AMIs, the instance constraints differ between them, as noted in each path. @@ -511 +511 @@ Define a NodePool that dynamically provisions G-family GPU instances with a gene -###### Important +EKS Auto Mode @@ -513 +512,0 @@ Define a NodePool that dynamically provisions G-family GPU instances with a gene -The G7 EC2 instance type requires NVIDIA driver version 595 or later. The EKS-optimized accelerated AMIs currently include NVIDIA driver version 580, which does not support G7 instances. The NodePool in this step constrains the instance generation to less than 7 so that Karpenter does not select a G7 instance. To use G7 instances with Amazon EKS, you must build a custom AMI with NVIDIA driver version 595. For more information, see [Use EKS-optimized accelerated AMIs for GPU instances](./ml-eks-optimized-ami.html). @@ -515 +514 @@ The G7 EC2 instance type requires NVIDIA driver version 595 or later. The EKS-op -EKS Auto Mode +###### Important @@ -516,0 +516 @@ EKS Auto Mode +This path uses the EKS-optimized Bottlerocket NVIDIA AMI, which does not include a driver version that supports the `g7` instance family (`g7` requires NVIDIA driver version 595 or later). Therefore, the NodePool in this path excludes the `g7` family. Other generation-7 instances, such as `g7e`, run on NVIDIA driver 580 and are supported. If you need `g7` support, use the self-managed Karpenter path with the EKS-optimized AL2023 NVIDIA AMI. For more information, see [EKS-optimized NVIDIA AMIs](./ml-eks-optimized-ami.html#eks-amis-nvidia). @@ -549,3 +549,3 @@ In EKS Auto Mode, the bundled `default` NodeClass automatically selects the Bott - - key: eks.amazonaws.com/instance-generation - operator: Lt - values: ["7"] + - key: eks.amazonaws.com/instance-family + operator: NotIn + values: ["g7"] @@ -560 +560 @@ In EKS Auto Mode, the bundled `default` NodeClass automatically selects the Bott -This NodePool provisions G-family GPU instances with a generation greater than 4 and less than 7 ([G5](https://aws.amazon.com/ec2/instance-types/g5/), [G6e](https://aws.amazon.com/ec2/instance-types/g6e/), etc.). EKS Auto Mode chooses an instance type within these constraints. +This NodePool provisions G-family GPU instances with a generation greater than 4, excluding the `g7` family (for example, [G5](https://aws.amazon.com/ec2/instance-types/g5/), [G6e](https://aws.amazon.com/ec2/instance-types/g6e/), and G7e). EKS Auto Mode chooses an instance type within these constraints. @@ -647,3 +646,0 @@ Expected output: `READY True`. If `False`, run `kubectl describe ec2nodeclass gp - - key: karpenter.k8s.aws/instance-generation - operator: Lt - values: ["7"] @@ -659,0 +657,2 @@ The `amiFamily: al2023` label on the node template is what the NVIDIA device plu +Unlike the EKS Auto Mode path, this NodePool does not cap the instance generation, because the EKS-optimized AL2023 NVIDIA AMI includes a driver version that supports `g7` instances. The EKS AL2023 AMI currently uses NVIDIA driver 580 on all EC2 instances except `g7`, which uses NVIDIA driver 595. For more information, see [EKS-optimized NVIDIA AMIs](./ml-eks-optimized-ami.html#eks-amis-nvidia). + @@ -766 +765 @@ Expected NodeClaim output: -The instance type and AZ will vary. Any G-family instance with a generation greater than 4 and less than 7 is eligible. +The instance type and Availability Zone vary. Any G-family instance that satisfies your NodePool’s generation constraint is eligible: greater than 4 for both paths, with the `g7` family excluded on the EKS Auto Mode path. @@ -884,3 +883,3 @@ Update the NodePool to use the ODCR-backed NodeClass and include `reserved` as a - - key: eks.amazonaws.com/instance-generation - operator: Lt - values: ["7"] + - key: eks.amazonaws.com/instance-family + operator: NotIn + values: ["g7"] @@ -969,3 +967,0 @@ Update the NodePool to include `reserved` as a capacity type: - - key: karpenter.k8s.aws/instance-generation - operator: Lt - values: ["7"]