AWS AmazonECS documentation change
Summary
Restructured documentation to introduce managed Neuron device allocation approach alongside manual specification, added instance selection guidance, and clarified launch type support limitations.
Security assessment
Changes focus on operational improvements for Neuron device allocation methods (managed vs manual) and instance selection. No security vulnerabilities, patches, or security features are mentioned. Updates clarify launch type support (Inf1 only on EC2) and operational constraints without security implications.
Diff
diff --git a/AmazonECS/latest/developerguide/ecs-inference.md b/AmazonECS/latest/developerguide/ecs-inference.md index eb4ce3d8b..a3b438106 100644 --- a//AmazonECS/latest/developerguide/ecs-inference.md +++ b//AmazonECS/latest/developerguide/ecs-inference.md @@ -7 +7 @@ -ConsiderationsUse the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMITask definition requirements +ConsiderationsManaged Neuron device allocationManual Neuron device specification @@ -11 +11 @@ ConsiderationsUse the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMITask de -You can register [Amazon EC2 Trn1](https://aws.amazon.com/ec2/instance-types/trn1/), [Amazon EC2 Trn2](https://aws.amazon.com/ec2/instance-types/trn2/), [Amazon EC2 Inf1](https://aws.amazon.com/ec2/instance-types/inf1/), and [Amazon EC2 Inf2](https://aws.amazon.com/ec2/instance-types/inf2/) instances to your clusters for machine learning workloads. +You can use [Amazon EC2 Trn1](https://aws.amazon.com/ec2/instance-types/trn1/), [Amazon EC2 Trn2](https://aws.amazon.com/ec2/instance-types/trn2/), [Amazon EC2 Inf1](https://aws.amazon.com/ec2/instance-types/inf1/) (Inf1 is only supported on EC2 launch type), and [Amazon EC2 Inf2](https://aws.amazon.com/ec2/instance-types/inf2/) instances with your clusters for machine learning workloads. @@ -13 +13 @@ You can register [Amazon EC2 Trn1](https://aws.amazon.com/ec2/instance-types/trn -Amazon EC2 Trn1 and Trn2 instances are powered by [AWS Trainium](https://aws.amazon.com/ai/machine-learning/trainium/) chips. These instances provide high performance and low cost training for machine learning in the cloud. You can train a machine learning inference model using a machine learning framework with AWS Neuron on a Trn1 or Trn2 instance. Then, you can run the model on a Inf1 instance, or an Inf2 instance to use the acceleration of the AWS Inferentia chips. +Amazon EC2 Trn1 and Trn2 instances are powered by [AWS Trainium](https://aws.amazon.com/ai/machine-learning/trainium/) chips. These instances provide high performance and low cost training for machine learning in the cloud. You can train a machine learning inference model using a machine learning framework with AWS Neuron on a Trn1 or Trn2 instance. Then, you can run the model on an Inf1 instance (Inf1 is only supported on EC2 launch type), or an Inf2 instance to use the acceleration of the AWS Inferentia chips. @@ -23 +23 @@ Before you begin deploying Neuron on Amazon ECS, consider the following: - * Your clusters can contain a mix of Trn1, Trn2, Inf1, Inf2 and other instances. + * Depending on the launch type, your clusters can contain a mix of Trn1, Trn2, Inf1, Inf2, and other instances. @@ -31 +31 @@ Applications that use other frameworks might not have improved performance on Tr - * Only one inference or inference-training task can run on each [AWS Trainium](https://aws.amazon.com/ai/machine-learning/trainium/) or [AWS Inferentia](https://aws.amazon.com/ai/machine-learning/inferentia/) chip. For Inf1, each chip has 4 NeuronCores. For Trn1, Trn2, and Inf2 each chip has 2 NeuronCores. You can run as many tasks as there are chips for each of your Trn1, Trn2, Inf1, and Inf2 instances. + * Amazon ECS supports two approaches for configuring Neuron device access: @@ -33 +33 @@ Applications that use other frameworks might not have improved performance on Tr - * When creating a service or running a standalone task, you can use instance type attributes when you configure task placement constraints. This ensures that the task is launched on the container instance that you specify. Doing so can help you optimize overall resource utilization and ensure that tasks for inference workloads are on your Trn1, Trn2, Inf1, and Inf2 instances. For more information, see [How Amazon ECS places tasks on container instances](./task-placement.html). + * **Managed Neuron device allocation** – Use the `resourceRequirements` parameter with type `NeuronDevice` in your container definition. Amazon ECS automatically discovers and assigns Neuron devices to your containers. Available on Managed Instances only. For more information, see Managed Neuron device allocation. @@ -35 +35 @@ Applications that use other frameworks might not have improved performance on Tr -In the following example, a task is run on an `Inf1.xlarge` instance on your `default` cluster. + * **Manual Neuron device specification** – Use the `linuxParameters.devices` parameter to explicitly specify Neuron device paths. Available on both EC2 launch type and Managed Instances. For more information, see Manual Neuron device specification. @@ -37,4 +37 @@ In the following example, a task is run on an `Inf1.xlarge` instance on your `de - aws ecs run-task \ - --cluster default \ - --task-definition ecs-inference-task-def \ - **--placement-constraints type=memberOf,expression="attribute:ecs.instance-type == Inf1.xlarge"** +###### Important @@ -42 +39 @@ In the following example, a task is run on an `Inf1.xlarge` instance on your `de - * Neuron resource requirements can't be defined in a task definition. Instead, you configure a container to use specific AWS Trainium or AWS Inferentia chips available on the host container instance. Do this by using the `linuxParameters` parameter and specifying the device details. For more information, see Task definition requirements. +Use only one approach consistently to avoid conflicts. @@ -47 +44 @@ In the following example, a task is run on an `Inf1.xlarge` instance on your `de -## Use the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI +## Managed Neuron device allocation @@ -49 +46 @@ In the following example, a task is run on an `Inf1.xlarge` instance on your `de -Amazon ECS provides an Amazon ECS optimized AMI that's based on Amazon Linux 2023 for AWS Trainium and AWS Inferentia workloads. It comes with the AWS Neuron drivers and runtime for Docker. This AMI makes running machine learning inference workloads easier on Amazon ECS. +With Managed Instances, you can use the `resourceRequirements` parameter in your container definition to request Neuron devices. Amazon ECS automatically discovers Neuron devices on the instance, assigns them to your task, and configures the container with access to all Neuron devices on the instance. Because the task requires exclusive access to all devices, only one Neuron task runs per instance. @@ -51 +48 @@ Amazon ECS provides an Amazon ECS optimized AMI that's based on Amazon Linux 202 -We recommend using the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI when launching your Amazon EC2 Trn1, Inf1, and Inf2 instances. +###### Note @@ -53 +50 @@ We recommend using the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI when -You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI using the AWS CLI with the following command. +`Inf1` instances are only supported on the EC2 launch type. To use Inf1 instances, see Manual Neuron device specification. @@ -54,0 +52 @@ You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI +### Neuron instance selection @@ -56 +54,29 @@ You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI - aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2023/neuron/recommended +To select Neuron-enabled instance types for your Managed Instances workloads, use the `instanceRequirements` object in the launch template of the capacity provider. You can use the following attributes to select Neuron-enabled instances: + + * `acceleratorManufacturers` – Use `amazon-web-services` to select instances with AWS accelerators (includes Inferentia and Trainium). + + * `acceleratorNames` – Use `inferentia2`, `trainium`, or `trainium2` to select specific accelerator chips. + + * `allowedInstanceTypes` – Use `inf*` and `trn*` to select Neuron instance types by name. + + + + +The following example uses `allowedInstanceTypes`: + + + { + "instanceRequirements": { + "allowedInstanceTypes": ["inf*", "trn*"] + } + } + +### Task definition + +To request Neuron devices in your task definition, add a `resourceRequirements` entry with type `NeuronDevice` and value `ALL`. This gives the container exclusive access to all Neuron devices on the instance. + +The following constraints apply: + + * At most one container definition can specify `NeuronDevice` in `resourceRequirements`. + + * You can't combine `resourceRequirements` with type `NeuronDevice` and `linuxParameters.devices` for Neuron devices in the same task definition. @@ -58 +83,0 @@ You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI -## Task definition requirements @@ -60 +84,0 @@ You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI -To deploy Neuron on Amazon ECS, your task definition must contain the container definition for a pre-built container serving the inference model for TensorFlow. It's provided by AWS Deep Learning Containers. This container contains the AWS Neuron runtime and the TensorFlow Serving application. At startup, this container fetches your model from Amazon S3, launches Neuron TensorFlow Serving with the saved model, and waits for prediction requests. In the following example, the container image has TensorFlow 1.15 and Ubuntu 18.04. A complete list of pre-built Deep Learning Containers optimized for Neuron is maintained on GitHub. For more information, see [Using AWS Neuron TensorFlow Serving](https://docs.aws.amazon.com/dlami/latest/devguide/tutorial-inferentia-tf-neuron-serving.html). @@ -63 +87 @@ To deploy Neuron on Amazon ECS, your task definition must contain the container - 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-neuron:1.15.4-neuron-py37-ubuntu18.04 +After your task starts, you can verify the Neuron device assignment by calling the `DescribeTasks` API operation. The response includes a `neuronDeviceIds` field on each container that shows the IDs of the assigned Neuron devices. You can also call the `DescribeContainerInstances` API operation to view `NEURON_DEVICES` in the `registeredResources` and `remainingResources` fields for the container instance. @@ -65 +89 @@ To deploy Neuron on Amazon ECS, your task definition must contain the container -Alternatively, you can build your own Neuron sidecar container image. For more information, see [Tutorial: Neuron TensorFlow Serving](https://github.com/aws-neuron/aws-neuron-sdk/blob/master/frameworks/tensorflow/tensorflow-neuron/tutorials/tutorials-tensorflow-utilizing-neuron-capabilities.rst) in the _AWS Deep Learning AMIs Developer Guide_. +For an example task definition, see [Example Neuron task definitions](./ecs-inference-task-def.html). @@ -67 +91,11 @@ Alternatively, you can build your own Neuron sidecar container image. For more i -The task definition must be specific to a single instance type. You must configure a container to use specific AWS Trainium or AWS Inferentia devices that are available on the host container instance. You can do so using the `linuxParameters` parameter. For a sample task definition, see [Specifying AWS Neuron machine learning in an Amazon ECS task definition](./ecs-inference-task-def.html). The following table details the chips that are specific to each instance type. +## Manual Neuron device specification + +With this approach, you manually specify AWS Trainium or AWS Inferentia device paths in your task definition using the `linuxParameters.devices` parameter. This approach works on both the EC2 launch type and Managed Instances. + +Only one inference or inference-training task can run on each [AWS Trainium](https://aws.amazon.com/ai/machine-learning/trainium/) or [AWS Inferentia](https://aws.amazon.com/ai/machine-learning/inferentia/) chip. You can run as many tasks as there are chips on the instance by assigning different devices to each task. + +For the EC2 launch type, you can use instance type attributes when you configure task placement constraints to ensure that the task is launched on the instance type you specify. For more information, see [How Amazon ECS places tasks on container instances](./task-placement.html). + +### Task definition requirements + +The task definition must be specific to a single instance type. You must configure a container to use specific AWS Trainium or AWS Inferentia devices that are available on the host container instance. You can do so using the `linuxParameters` parameter. The following table details the chips that are specific to each instance type. @@ -80 +114 @@ inf2.8xlarge | 32 | 64 | 1 | `/dev/neuron0` -inf2.24xlarge | 96 | 384 | 6 | `/dev/neuron0`, `/dev/neuron1`, `/dev/neuron2`, `/dev/neuron3`, `/dev/neuron4`, `/dev/neuron5`, +inf2.24xlarge | 96 | 384 | 6 | `/dev/neuron0`, `/dev/neuron1`, `/dev/neuron2`, `/dev/neuron3`, `/dev/neuron4`, `/dev/neuron5` @@ -82,0 +117,17 @@ inf2.48xlarge | 192 | 768 | 12 | `/dev/neuron0`, `/dev/neuron1`, `/dev/neuron2`, +For an example task definition, see [Example Neuron task definitions](./ecs-inference-task-def.html). + +### Managed Instances + +Managed Instances automatically use an AMI that includes the Neuron driver. No additional AMI configuration is required. + +### EC2 launch type + +Amazon ECS provides an Amazon ECS optimized AMI that's based on Amazon Linux 2023 for AWS Trainium and AWS Inferentia workloads. It comes with the AWS Neuron drivers and runtime for Docker. This AMI makes running machine learning inference workloads easier on Amazon ECS. + +We recommend using the Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI when launching your Amazon EC2 Trn1, Inf1, and Inf2 instances. + +You can retrieve the current Amazon ECS-optimized Amazon Linux 2023 (Neuron) AMI using the AWS CLI with the following command. + + + aws ssm get-parameters --names /aws/service/ecs/optimized-ami/amazon-linux-2023/neuron/recommended + @@ -91 +142 @@ Specifying video transcoding in a task definition -Specifying AWS Neuron machine learning in a task definition +Example Neuron task definitions