AWS AWSEC2 documentation change
Summary
Added documentation for P6e-GB200 instances including network interface configuration details, bandwidth allocation rules, and GPU-specific traffic routing recommendations
Security assessment
The changes document new instance type capabilities and network configuration optimizations without mentioning security vulnerabilities, patches, or security controls. Focuses on performance characteristics and bandwidth allocation between EFA/ENA interfaces.
Diff
diff --git a/AWSEC2/latest/UserGuide/efa-acc-inst-types.md b/AWSEC2/latest/UserGuide/efa-acc-inst-types.md index 6b0ced06f..92c5e002e 100644 --- a//AWSEC2/latest/UserGuide/efa-acc-inst-types.md +++ b//AWSEC2/latest/UserGuide/efa-acc-inst-types.md @@ -5 +5 @@ -EFA configuration for a P5 instances +EFA configuration for a P5 instancesEFA configuration for a P6e-GB200 instances @@ -153 +153 @@ The following snippet shows an example with 3 network interfaces out of the poss -P5 instances have a total network bandwidth capacity of 3,200 Gbps, of which up to 800 Gbps can be utilized for IP network traffic. Because EFA and IP network traffic share the same underlying resources, bandwidth used by one will reduce the bandwidth that is available to the other. This means that you can distribute the network bandwidth between EFA traffic and IP traffic in any combination, as long as the total bandwidth does not exceed 3,200 Gbps and IP bandwidth does not exceed 800 Gbps. +P5 instances have a total network bandwidth capacity of 3,200 Gbps, of which up to 800 Gbps can be utilized for IP network traffic. Because EFA and IP network traffic share the same underlying resources, bandwidth used by one will reduce the bandwidth that is available to the other. This means that you can distribute the network bandwidth between EFA traffic and IP traffic in any combination, as long as the total bandwidth does not exceed 3,200 Gbps and IP bandwidth does not exceed 800 Gbps. For example, if you use 400 Gbps for IP bandwidth, you can achieve up to 2,800 Gbps of EFA bandwidth at the same time. @@ -157 +157 @@ P5 instances have a total network bandwidth capacity of 3,200 Gbps, of which up -This configuration provides up to 3200 Gbps of EFA networking bandwidth and up to 100 Gbps of IP networking bandwidth with one private IP address. This configuration also helps to avoid potential Linux IP issues, such as disallowed auto-assignment of public IP addresses and IP routing challenges (hostname to IP address mapping issues and source IP address mismatches), that can arise if an instance has multiple network interfaces. For example, if you use 400 Gbps or IP bandwidth, you can achieve up to 2,800 Gbps of EFA bandwidth at the same time. +This configuration provides up to 3200 Gbps of EFA networking bandwidth and up to 100 Gbps of IP networking bandwidth with one private IP address. This configuration also helps to avoid potential Linux IP issues, such as disallowed auto-assignment of public IP addresses and IP routing challenges (hostname to IP address mapping issues and source IP address mismatches), that can arise if an instance has multiple network interfaces. @@ -176 +176 @@ This configuration provides up to 3200 Gbps of EFA networking bandwidth and up t - * Specify one EFA (EFA with ENA) network interface and three EFA-only network interfaces **in each** of the following network card index subsets, and use device index 1: + * Specify one EFA (EFA with ENA) network interface and three EFA-only network interfaces **in each** of the following network card index subsets, and use device index 1 for all of them: @@ -236,0 +237,70 @@ The following example illustrates this configuration: +## EFA configuration for a P6e-GB200 instances + +P6e-GB200 instances can be configured with up to 17 network cards. The following image shows the physical network interface card (NIC) layout for P6e-GB200 instances, along with the mapping of network card indexes (NCIs). + + + +The primary NCI (index 0) supports up to 100 Gbps of ENA bandwidth. NCIs with the following indexes support EFA-only network interfaces and 400 Gbps EFA bandwidth: [1, 3, 5, 7, 9, 11, 13, 15]. NCIs with the following indexes support up to 200 Gbps ENA or EFA bandwidth: [2, 4, 6, 8, 10, 12, 14, 16]. + +The NCIs in the following groups share an underlying physical NIC on the host: + + * [1 and 2] + + * [3 and 4] + + * [5 and 6] + + * [7 and 8] + + * [9 and 10] + + * [11 and 12] + + * [13 and 14] + + * [15 and 16] + + + + +Each physical NIC supports up 400 Gbps of bandwidth. Because the NCIs in these groups share the same underlying physical NIC, bandwidth used by one will reduce the bandwidth that is available to the other. For example, if NCI 2 uses 200 Gbps of ENA bandwidth, NCI 1 can use up to 200 Gbps of EFA bandwidth at the same time. + +Each underlying GPU on the host can send traffic directly over the following pairs of NCIs: + + * [1 and 3] + + * [5 and 7] + + * [9 and 11] + + * [13 and 15] + + + + +Each GPU supports up to 400 Gbps of EFA bandwidth. Because the network cards in these groups share the same GPU, bandwidth used by one will reduce the bandwidth that is available to the other. For example, if NCI 1 uses 200 Gbps of EFA bandwidth, NCI 3 can use up to 200 Gbps of EFA bandwidth at the same time. Therefore, to achieve maximum EFA performance, we recommend that you do one of the following to achieve a total of 1,600 Gbps EFA bandwitdh: + + * Add an EFA-only network interface to only one NCI in each group to achieve 400 Gpbs per network interface (_4 EFA network interfaces x 400 Gbps_). + + * Add an EFA-only network interface to each NCI in each group to achieve 200 Gpbs per network interface (_8 EFA network interfaces x 200 Gbps_). + + + + +For example, the following configuration provides up to 1,600 Gbps of EFA bandwidth using a single EFA-only network interface in each NCI group, and up to 100 Gbps of ENA networking bandwidth using only the primary NCI (index 0). + + * For the primary NCI (network card index 0, device index 0), use an ENA network interface. + + * Add EFA-only network interfaces to the following: + + * NCI 1, device index 1 + + * NCI 5, device index 1 + + * NCI 9, device index 1 + + * NCI 13, device index 1 + + + +