AWS eks medium security documentation change
Summary
Enhanced documentation for Pod networking configurations including separate subnets/security groups, IPv4 prefix sizing, and IPv6 egress controls. Added clarification about security group scope and Network Policy limitations.
Security assessment
The changes explicitly address security controls: 1) Added warnings about Network Policy limitations with IPv4 egress interfaces ('enableV4Egress' field) that could allow bypassing security policies. 2) Introduced documentation for podSecurityGroupSelectorTerms as a security control mechanism. 3) Clarified security group scope differences between Pod traffic within VPC vs external traffic.
Diff
diff --git a/eks/latest/userguide/create-node-class.md b/eks/latest/userguide/create-node-class.md index 6f6fa53d4..56c2f07ec 100644 --- a//eks/latest/userguide/create-node-class.md +++ b//eks/latest/userguide/create-node-class.md @@ -5 +5 @@ -Create a Node ClassCreate node class access entryNode Class SpecificationConsiderationsSubnet selection for PodsSecondary IP Mode for PodsDisable IPv4 egress from IPv6 pods in IPv6 clusters. +Create a Node ClassCreate node class access entryNode Class SpecificationConsiderationsSeparate subnets and security groups for PodsSecondary IP Mode for PodsDisable IPv4 egress from IPv6 pods in IPv6 clusters. @@ -244 +244 @@ For information about deploying CloudFormation stacks, see [Getting started with - * If there are multiple subnets _in the same AZ_ that match the `subnetSelectorTerms`, EKS Auto Mode creates Pods on each node distributed across the subnets in that AZ. EKS Auto Mode creates secondary network interfaces on each node in the other subnets in the same AZ. It chooses based on the number of available IP addresses in each subnet, to use the subnets more efficiently. However, you can’t specify which subnet EKS Auto Mode uses for each Pod; if you need Pods to run in specific subnets, use Subnet selection for Pods instead. + * If there are multiple subnets _in the same AZ_ that match the `subnetSelectorTerms`, EKS Auto Mode creates Pods on each node distributed across the subnets in that AZ. EKS Auto Mode creates secondary network interfaces on each node in the other subnets in the same AZ. It chooses based on the number of available IP addresses in each subnet, to use the subnets more efficiently. However, you can’t specify which subnet EKS Auto Mode uses for each Pod; if you need Pods to run in specific subnets, use Separate subnets and security groups for Pods instead. @@ -249 +249,19 @@ For information about deploying CloudFormation stacks, see [Getting started with -## Subnet selection for Pods +## Separate subnets and security groups for Pods + +The `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` fields enable advanced networking configurations by allowing Pods to use different subnets and security groups than their nodes. Both fields must be specified together. This separation provides enhanced control over network traffic routing and security policies. + +###### Note + +This feature is different from the [Security Groups for Pods](./security-groups-for-pods.html) (SGPP) feature used with the AWS VPC CNI for non-EKS Auto Mode compute. SGPP is not supported in EKS Auto Mode. Instead, use `podSecurityGroupSelectorTerms` in the `NodeClass` to apply separate security groups to Pod traffic. The security groups apply at the `NodeClass` level, meaning all Pods on nodes using that `NodeClass` share the same Pod security groups. + +### How it works + +When you configure `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms`: + + 1. The node’s primary ENI uses the subnets and security groups from `subnetSelectorTerms` and `securityGroupSelectorTerms`. Only the node’s own IP address is assigned to this interface. + + 2. EKS Auto Mode creates secondary ENIs in the subnets matching `podSubnetSelectorTerms`, with the security groups from `podSecurityGroupSelectorTerms` attached. Pod IP addresses are allocated from these secondary ENIs using /28 prefixes by default, with automatic fallback to secondary IPs (/32) when a contiguous prefix block is not available. If `ipv4PrefixSize` is set to `"32"` in `advancedNetworking`, only secondary IPs are used. + + 3. The security groups specified in `podSecurityGroupSelectorTerms` apply to Pod traffic within the VPC. For traffic destined outside the VPC, Pods use the node’s primary ENI (and its security groups) because source network address translation (SNAT) translates the Pod IP to the node IP. You can modify this behavior with the `snatPolicy` field in the `NodeClass`. + + @@ -251 +268,0 @@ For information about deploying CloudFormation stacks, see [Getting started with -The `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` fields enables advanced networking configurations by allowing Pods to run in different subnets than their nodes. This separation provides enhanced control over network traffic routing and security policies. Note that `podSecurityGroupSelectorTerms` are required with the `podSubnetSelectorTerms`. @@ -255 +272 @@ The `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` fields enables -Use `podSubnetSelectorTerms` when you need to: +Use `podSubnetSelectorTerms` and `podSecurityGroupSelectorTerms` when you need to: @@ -257 +274 @@ Use `podSubnetSelectorTerms` when you need to: - * Separate infrastructure traffic (node-to-node communication) from application traffic (Pod-to-Pod communication) + * Apply different security groups to control traffic for nodes and Pods separately. @@ -259 +276 @@ Use `podSubnetSelectorTerms` when you need to: - * Apply different network configurations to node subnets than Pod subnets. + * Separate infrastructure traffic (node-to-node communication) from application traffic (Pod-to-Pod communication). @@ -261 +278 @@ Use `podSubnetSelectorTerms` when you need to: - * Implement different security policies or routing rules for nodes and Pods. + * Apply different network configurations to node subnets than Pod subnets. @@ -278 +295 @@ Use `podSubnetSelectorTerms` when you need to: - # Subnets for EC2 instances (nodes) + # Subnets and security groups for EC2 instances (nodes) @@ -288 +305 @@ Use `podSubnetSelectorTerms` when you need to: - # Separate subnets for Pods + # Separate subnets and security groups for Pods @@ -298 +315,5 @@ Use `podSubnetSelectorTerms` when you need to: -### Considerations for subnet selectors for Pods +### Considerations for separate Pod subnets and security groups + + * **Security group scope** : The security groups from `podSecurityGroupSelectorTerms` are attached to the secondary ENIs and apply to Pod traffic within the VPC. When SNAT is enabled (the default `snatPolicy: Random`), traffic leaving the VPC is translated to the node’s primary ENI IP address, so the node’s security groups from `securityGroupSelectorTerms` apply to that traffic instead. If you set `snatPolicy: Disabled`, Pods use their own IP addresses for all traffic, and you must ensure that routing and security groups are configured accordingly. + + * **NodeClass-level granularity** : The Pod security groups apply to all Pods scheduled on nodes using the `NodeClass`. To apply different security groups to different workloads, create separate `NodeClass` and `NodePool` resources and use taints, tolerations, or node selectors to schedule workloads to the appropriate nodes. @@ -300 +321 @@ Use `podSubnetSelectorTerms` when you need to: - * **Reduced Pod density** : Fewer Pods can run on each node when using `podSubnetSelectorTerms`, because the primary network interface of the node is in the node subnet, and can’t be used for Pods in the Pod subnet. + * **Reduced Pod density** : Fewer Pods can run on each node because the primary network interface of the node is reserved for the node IP and can’t be used for Pods. @@ -302 +323 @@ Use `podSubnetSelectorTerms` when you need to: - * **Subnet selector limitations** : The standard `subnetSelectorTerms` and `securityGroupSelectorTerms` configurations don’t apply to Pod subnet selection. + * **Subnet selector limitations** : The standard `subnetSelectorTerms` and `securityGroupSelectorTerms` configurations don’t apply to Pod subnet or security group selection. @@ -308 +329 @@ Use `podSubnetSelectorTerms` when you need to: - * **Availability Zones** : Verify that you’ve created Pod subnets across multiple AZs. If you are using specific Pod subnet, it must be in the same AZ as the node subnet AZ. + * **Availability Zones** : Verify that you’ve created Pod subnets across multiple AZs. If you are using a specific Pod subnet, it must be in the same AZ as the node subnet AZ. @@ -315 +336 @@ Use `podSubnetSelectorTerms` when you need to: -The `ipv4PrefixSize` fields enables advanced networking configurations by allowing only allocating secondary IP addresses to nodes. This feature doesn’t allocate prefix (/28) to nodes and maintain only one secondary IP as MinimalIPTarget. +The `ipv4PrefixSize` field enables advanced networking configurations by allocating only secondary IP addresses to nodes. This feature doesn’t allocate prefixes (/28) to nodes and maintains only one secondary IP as MinimalIPTarget. @@ -321 +342 @@ Use `ipv4PrefixSize` when you need to: - * **Reduced IP utilization** : Only one IP addresses will be warmed up in every node. + * **Reduced IP utilization** : Only one IP address will be warmed up in every node. @@ -323 +344 @@ Use `ipv4PrefixSize` when you need to: - * **Lower pods churning rate** : Pods creation velocity is not a major concern. + * **Lower pod churning rate** : Pod creation velocity is not a major concern. @@ -325 +346 @@ Use `ipv4PrefixSize` when you need to: - * **No prefix fragmentation** : Prefix caused fragmentation is a major concern or blocker to use Auto mode. + * **No prefix fragmentation** : Prefix-caused fragmentation is a major concern or blocker to use Auto Mode. @@ -345 +366 @@ Use `ipv4PrefixSize` when you need to: - * **Reduced Pod creation velocity** : Since only one secondary IP is warmed up, the IPAM service need more time to provision IPs on more pods creation. + * **Reduced Pod creation velocity** : Since only one secondary IP is warmed up, the IPAM service needs more time to provision IPs when more pods are created. @@ -352 +373 @@ Use `ipv4PrefixSize` when you need to: -The `enableV4Egress` fields is true by default. For Auto Mode IPv6 clusters, the feature can be disabled and thus Auto Mode won’t create an egress only IPv4 interface for IPv6 pods. This is important because the IPv4 network interface will not be secured by the Network policy feature. Network policies will only be enforced on the Pod’s primary interface (i.e.,) eth0. +The `enableV4Egress` field is `true` by default. For Auto Mode IPv6 clusters, the feature can be disabled so that Auto Mode won’t create an egress-only IPv4 interface for IPv6 pods. This is important because the IPv4 egress interface is not subject to Network Policy enforcement. Network policies are only enforced on the Pod’s primary interface (eth0). @@ -360 +381 @@ Use `enableV4Egress` when you need to: - * **Use Network Policy** : Currently EKS network policy isn’t supporting dual stack. Disabling the v4Egress can protect pods' traffic being egressed from pods unexpectedly. + * **Use Network Policy** : Currently EKS Network Policy doesn’t support dual stack. Disabling `enableV4Egress` can prevent pod traffic from egressing over IPv4 unexpectedly.