AWS sagemaker medium security documentation change
Summary
Restructured and expanded documentation for creating SageMaker HyperPod clusters with Amazon EKS orchestration, adding detailed sections for Quick setup, Custom setup, instance group configuration, storage options, and deployment methods.
Security assessment
Added documentation for Restricted Instance Groups (RIG) which enforce additional security restrictions for model customization. Mentions security group configuration requirements and deep health checks for instance recovery. However, there's no explicit mention of patching vulnerabilities or addressing specific security incidents.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-eks-operate-console-ui-create-cluster.md b/sagemaker/latest/dg/sagemaker-hyperpod-eks-operate-console-ui-create-cluster.md index 417d9b08d..324024358 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-eks-operate-console-ui-create-cluster.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-eks-operate-console-ui-create-cluster.md @@ -5 +5 @@ -# Creating a SageMaker HyperPod cluster +Create clusterQuick setupCustom setupDeploy resources @@ -7 +7,20 @@ -See the following instructions on creating a new SageMaker HyperPod cluster using the SageMaker HyperPod console UI. +# Creating a SageMaker HyperPod cluster with Amazon EKS orchestration + +The following tutorial demonstrates how to create a new SageMaker HyperPod cluster and set it up with Amazon EKS orchestration through the SageMaker AI console UI. + +###### In this topic: + + * Create cluster + + * Quick setup + + * Custom setup + + * Deploy resources + + + + +## Create cluster + +To navigate to the **SageMaker HyperPod Clusters** page and choose Amazon EKS orchestration, follow these steps. @@ -13 +32,26 @@ See the following instructions on creating a new SageMaker HyperPod cluster usin - 3. In the SageMaker HyperPod landing page, choose **Create HyperPod cluster**. + 3. On the **SageMaker HyperPod Clusters** page, choose **Create HyperPod cluster**. + + 4. On the **Create HyperPod cluster** drop-down, choose **Orchestrated by Amazon EKS**. + + 5. On the EKS cluster creation page, you will see two options, choose the option that best fits your needs. + + 1. **Quick setup** -To get started immediately with default settings, choose **Quick setup**. With this option, SageMaker AI will create new resources such as VPC, subnets, security groups, Amazon S3 bucket, IAM role, and FSx for Lustre in the process of creating your cluster. + + 2. **Custom setup** -To integrate with existing AWS resources or have specific networking, security, or storage requirements, choose **Custom setup**. With this option, you can choose to use the existing resources or create new ones, and you can customize the configuration that best fits your needs. + + + + +## Quick setup + +On the **Quick setup** section, follow these steps to create your HyperPod cluster with Amazon EKS orchestration. + +Specify a name for the new cluster. You can’t change the name after the cluster is created. + +To add an instance group, choose **Add group**. Each instance group can be configured differently, and you can create a heterogeneous cluster that consists of multiple instance groups with various instance types. To deploy a cluster, you must add at least one instance group. Follow these steps to add an instance group. + + 1. For **Instance group type** , choose **Standard** or **Restricted Instance Group (RIG)**. Typically, you will choose **Standard** , which provides a general purpose computing environment without additional security restrictions. **Restricted Instance Group (RIG)** is a specialized environment for foundational models customization such as Amazon Nova. For more information about setting up RIG for Amazon Nova model customization, see [Amazon Nova customization on Amazon SageMaker HyperPod](./nova-hp.html). + + 2. For **Name** , specify a name for the instance group. + + 3. For **Instance capacity** , choose either on-demand capacity or a training plan to reserve your compute resources. @@ -15 +59 @@ See the following instructions on creating a new SageMaker HyperPod cluster usin - 4. From the drop-down menu of **Create HyperPod cluster** , choose **Orchestrated by Amazon EKS**. + 4. For **Instance type** , choose the instance for the instance group. Ensure that you choose the instance type with sufficient quotas in your account, or request additional quotas by following at [SageMaker HyperPod quotas](./sagemaker-hyperpod-prerequisites.html#sagemaker-hyperpod-prerequisites-quotas). @@ -17 +61 @@ See the following instructions on creating a new SageMaker HyperPod cluster usin - 5. From the Amazon EKS cluster list, choose the EKS cluster with which you want to configure the new HyperPod cluster. + 5. For **Instance quantity** , specify an integer not exceeding the instance quota for cluster usage. For this tutorial, enter **1** for all three groups. @@ -19 +63,39 @@ See the following instructions on creating a new SageMaker HyperPod cluster usin - 1. If you need to create a new EKS cluster, choose **Create EKS cluster**. You can create it from the EKS cluster list page without having to open the Amazon EKS console. + 6. For **Target Availability Zone** , choose the Availability Zone where your instances will be provisioned. The Availability Zone should correspond to the location of your accelerated compute capacity. + + 7. For **Additional storage volume per instance (GB) - optional** , specify an integer between 1 and 16384 to set the size of an additional Elastic Block Store (EBS) volume in gigabytes (GB). The EBS volume is attached to each instance of the instance group. The default mount path for the additional EBS volume is `/opt/sagemaker`. After the cluster is successfully created, you can SSH into the cluster instances (nodes) and verify if the EBS volume is mounted correctly by running the `df -h` command. Attaching an additional EBS volume provides stable, off-instance, and independently persisting storage, as described in the [Amazon EBS volumes](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes.html) section in the _Amazon Elastic Block Store User Guide_. + + 8. For **Instance deep health checks** , choose your option. Deep health checks monitor instance health during creation and after software updates, automatically recovering faulty instances through reboots or replacements when enabled. + + 9. Choose **Add instance group**. + + + + +This section lists all the default settings for your cluster creation, including all the new AWS resources that will be created during the cluster creation process. Review the default settings. + +## Custom setup + +On the **Custom setup** section, follow these steps to create your first HyperPod cluster with Amazon EKS orchestration. + +Specify a name for the new cluster. You can’t change the name after the cluster is created. + +For **Instance recovery** , choose **Automatic -_recommended_** or **None**. + +Configure network settings within the cluster and in-and-out of the cluster. For orchestration of SageMaker HyperPod cluster with Amazon EKS, the VPC is automatically set to the one configured with the EKS cluster you selected. + + 1. For **VPC** , choose your own VPC if you already have one that gives SageMaker AI access to your VPC. To create a new VPC, follow the instructions at [Create a VPC](https://docs.aws.amazon.com/vpc/latest/userguide/create-vpc.html) in the _Amazon Virtual Private Cloud User Guide_. You can leave it as **None** to use the default SageMaker AI VPC. + + 2. For **VPC IPv4 CIDR block** , enter the starting IP of your VPC. + + 3. For **Availability Zones** , choose the Availability Zones (AZ) where HyperPod will create subnets for your cluster. Choose AZs that match the location of your accelerated compute capacity. + + 4. For **Security group(s)** , choose security groups that are either attached to the Amazon EKS cluster or whose inbound traffic is permitted by the security group associated with the Amazon EKS cluster. To create new security groups, go to the Amazon VPC console. + + + + +Follow these steps to create or select an Amazon EKS cluster to use as an orchestrator. + + 1. For **EKS cluster** , choose either create a new Amazon EKS cluster or use an existing one. + +If you need to create a new EKS cluster, you can create it from the EKS cluster section without having to open the Amazon EKS console. @@ -25 +107 @@ The VPC subnet you choose for HyperPod has to be private. - 2. After submitting a new EKS cluster creation request, wait until the EKS cluster becomes Active. +After submitting a new EKS cluster creation request, wait until the EKS cluster becomes `Active`. @@ -27 +109 @@ The VPC subnet you choose for HyperPod has to be private. - 3. Install the Helm chart as instructed in [Installing packages on the Amazon EKS cluster using Helm](./sagemaker-hyperpod-eks-install-packages-using-helm-chart.html). + 2. For **Kubernetes version** , choose a version from the drop-down menu. For more information about Kubernetes versions, see [Understand the Kubernetes version lifecycle on EKS](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html) from the _Amazon EKS User Guide_. @@ -29 +111,5 @@ The VPC subnet you choose for HyperPod has to be private. - 4. After the EKS cluster creation has completed, choose **Create HyperPod cluster** and then **Orchestrated by EKS** again. You should be able to find and select the new EKS cluster. To proceed, choose **Select**. + 3. For **Operators** , choose **Use default Helm charts and add-ons** or **Don't install operators**. The option defaults to **Use default Helm charts and add-ons** , which will be used to install operators on the EKS cluster. For more information about the default Helm charts and add-ons, see [`helm_chart`](https://github.com/aws/sagemaker-hyperpod-cli/tree/main/helm_chart/HyperPodHelmChart) from the GitHub repository. For more information, see [Installing packages on the Amazon EKS cluster using Helm](./sagemaker-hyperpod-eks-install-packages-using-helm-chart.html). + + 4. For **Enabled operators** , view the list of enabled operators. To edit the operators, uncheck the box at top and choose operators to enable for the EKS cluster. + +###### Note @@ -31 +117 @@ The VPC subnet you choose for HyperPod has to be private. - 6. On the **Configure a new HyperPod cluster** page, set up the basic information for the cluster such as name, options to enable the HyperPod cluster resiliency features, and tags. +To use HyperPod with EKS, you must install Helm charts and add-ons that enable operators on the EKS cluster. These components configure EKS as the control plane for HyperPod and provide the necessary setup for workload management and orchestration. @@ -33 +118,0 @@ The VPC subnet you choose for HyperPod has to be private. - 7. For **Cluster name** , specify a name for the new cluster. @@ -35 +119,0 @@ The VPC subnet you choose for HyperPod has to be private. - 8. For **Cluster resiliency - node recovery** , specify `Automatic` to enable automatic node recovery. SageMaker HyperPod replaces or reboots instances (nodes) when issues are found by the health-monitoring agent. @@ -37 +120,0 @@ The VPC subnet you choose for HyperPod has to be private. - 9. For **Tags** , add key and value pairs to the new cluster and manage the cluster as an AWS resource. To learn more, see [Tagging your AWS resources](https://docs.aws.amazon.com/tag-editor/latest/userguide/tagging.html). @@ -39 +122 @@ The VPC subnet you choose for HyperPod has to be private. - 10. In **Step 2: Specify networking** , configure network settings within the cluster and in-and-out of the cluster. For orchestration of SageMaker HyperPod cluster with Amazon EKS, the VPC is automatically set to the one configured with the EKS cluster you selected. +To add an instance group, choose **Add group**. Each instance group can be configured differently, and you can create a heterogeneous cluster that consists of multiple instance groups with various instance types. To deploy a cluster, you must add at least one instance group. Follow these steps to add an instance group. @@ -41 +124 @@ The VPC subnet you choose for HyperPod has to be private. - 11. In **Step 3: Configure instance groups** , choose **Create instance group**. Each instance group can be configured differently, and you can create a heterogeneous cluster that consists of multiple instance groups with various instance types. In the **Create an instance group** configuration pop-up window, fill the instance group configuration information. + 1. For **Instance group type** , choose **Standard** or **Restricted Instance Group (RIG)**. Typically, you will choose **Standard** , which provides a general purpose computing environment without additional security restrictions. **Restricted Instance Group (RIG)** is a specialized environment for foundational models customization such as Amazon Nova. For more information about setting up RIG for Amazon Nova model customization, see [Amazon Nova customization on Amazon SageMaker HyperPod](./nova-hp.html). @@ -43 +126 @@ The VPC subnet you choose for HyperPod has to be private. -On **Create an instance group** pop-up page, choose **Standard** to configure a new instance group following the UI guidance. + 2. For **Name** , specify a name for the instance group. @@ -45 +128 @@ On **Create an instance group** pop-up page, choose **Standard** to configure a - 1. For **Instance group name** , specify a name for the instance group. + 3. For **Instance capacity** , choose either on-demand capacity or a training plan to reserve your compute resources. @@ -47 +130 @@ On **Create an instance group** pop-up page, choose **Standard** to configure a - 2. For **Select instance type** , choose the instance for the instance group. + 4. For **Instance type** , choose the instance for the instance group. Ensure that you choose the instance type with sufficient quotas in your account, or request additional quotas by following at [SageMaker HyperPod quotas](./sagemaker-hyperpod-prerequisites.html#sagemaker-hyperpod-prerequisites-quotas). @@ -49 +132 @@ On **Create an instance group** pop-up page, choose **Standard** to configure a - 3. For **Quantity** , specify an integer not exceeding the instance quota for cluster usage. + 5. For **Instance quantity** , specify an integer not exceeding the instance quota for cluster usage. For this tutorial, enter **1** for all three groups. @@ -51 +134 @@ On **Create an instance group** pop-up page, choose **Standard** to configure a - 4. Prepare a lifecycle configuration script and upload to an Amazon S3 bucket, such as `s3://`amzn-s3-demo-bucket`/`Lifecycle-scripts`/`base-config`/`. + 6. For **Target Availability Zone** , choose the Availability Zone where your instances will be provisioned. The Availability Zone should correspond to the location of your accelerated compute capacity. @@ -53 +136 @@ On **Create an instance group** pop-up page, choose **Standard** to configure a -For a quick start, download the sample script [`on_create.sh`](https://github.com/aws-samples/awsome-distributed-training/blob/main/1.architectures/7.sagemaker-hyperpod-eks/LifecycleScripts/base-config/on_create.sh) from the AWSome Distributed Training GitHub repository, and upload it to the S3 bucket. You can also include additional setup instructions, a series of setup scripts, or commands to be executed during the HyperPod cluster provisioning stage. + 7. For **Additional storage volume per instance (GB) - optional** , specify an integer between 1 and 16384 to set the size of an additional Elastic Block Store (EBS) volume in gigabytes (GB). The EBS volume is attached to each instance of the instance group. The default mount path for the additional EBS volume is `/opt/sagemaker`. After the cluster is successfully created, you can SSH into the cluster instances (nodes) and verify if the EBS volume is mounted correctly by running the `df -h` command. Attaching an additional EBS volume provides stable, off-instance, and independently persisting storage, as described in the [Amazon EBS volumes](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes.html) section in the _Amazon Elastic Block Store User Guide_. @@ -55 +138 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 5. For **S3 bucket URI for lifecycle scripts** , enter the Amazon S3 path in which the lifecycle scripts are stored. + 8. For **Instance deep health checks** , choose your option. Deep health checks monitor instance health during creation and after software updates, automatically recovering faulty instances through reboots or replacements when enabled. To learn more, see [Deep health checks](./sagemaker-hyperpod-eks-resiliency-deep-health-checks.html) @@ -57 +140 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 6. For **Directory path to the entrypoint script in the base Amazon S3 path** , enter the file name of the lifecycle script under **Amazon S3 path to lifecycle script files**. If you use the provided sample script, enter `on_create.sh`. + 9. Choose **Add instance group**. @@ -59 +141,0 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 7. For IAM role, choose the IAM role you have created for SageMaker HyperPod resources, following the section [IAM role for SageMaker HyperPod](./sagemaker-hyperpod-prerequisites-iam.html#sagemaker-hyperpod-prerequisites-iam-role-for-hyperpod). @@ -61 +142,0 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 8. Under **Advanced configuration** , you can set up the following optional configurations. @@ -63 +143,0 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 1. (Optional) For **Threads per core** , specify `1` for disabling multi-threading and `2` for enabling multi-threading. To find which instance type supports multi-threading, see the reference table of [CPU cores and threads per CPU core per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/cpu-options-supported-instances-values.html) in the _Amazon EC2 User Guide_. @@ -65 +145 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 2. (Optional) For **Additional instance storage configs** , specify an integer between 1 and 16384 to set the size of an additional Elastic Block Store (EBS) volume in gigabytes (GB). The EBS volume is attached to each instance of the instance group. The default mount path for the additional EBS volume is `/opt/sagemaker`. After the cluster is successfully created, you can SSH into the cluster instances (nodes) and verify if the EBS volume is mounted correctly by running the `df -h` command. Attaching an additional EBS volume provides stable, off-instance, and independently persisting storage, as described in the [Amazon EBS volumes](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes.html) section in the _Amazon Elastic Block Store User Guide_. +You can choose to use the default lifecycle scripts or the custom lifecycle scripts, which will be stored in your Amazon S3 bucket. You can view the default lifecycle scripts in the [Awesome Distributed Training GitHub repository](https://github.com/aws-samples/awsome-distributed-training/tree/main/1.architectures/7.sagemaker-hyperpod-eks/LifecycleScripts). To learn more about the lifecycle scripts, see [Customizing SageMaker HyperPod clusters using lifecycle scripts](./sagemaker-hyperpod-lifecycle-best-practices-slurm.html). @@ -67 +147 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 9. For **Deep health check** , select the advanced health checks you want to run on the instances. To learn more, see [Deep health checks](./sagemaker-hyperpod-eks-resiliency-deep-health-checks.html). + 1. For **Lifecycle scripts** , choose to use default or custom lifecycle scripts. @@ -69 +149 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co - 12. In **Step 3: Configure instance groups** , choose **Create instance group**. Each instance group can be configured differently, and you can create a heterogeneous cluster that consists of multiple instance groups with various instance types. In the **Create an instance group** configuration pop-up window, fill the instance group configuration information. + 2. For **S3 bucket for lifecycle scripts** , choose to create a new bucket or use an existing bucket to store the lifecycle scripts. @@ -71 +150,0 @@ For a quick start, download the sample script [`on_create.sh`](https://github.co -On **Create an instance group** pop-up page, choose **Restricted Instance Group (RIG)** to configure a new restricted instance group following the UI guidance. RIG is only required when you want to create a cluster for Amazon Nova model customization. For more information, see [Amazon Nova customization on Amazon SageMaker HyperPod](./nova-hp.html). @@ -73 +151,0 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 1. For **Instance group name** , specify a name for the restricted instance group. @@ -75 +152,0 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 2. For **Select instance type** , choose the instance for the restricted instance group. @@ -77 +154 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 3. For **Quantity** , specify an integer not exceeding the instance quota for cluster usage. +Choose or create an IAM role that allows HyperPod to run and access necessary AWS resources on your behalf. For more information, see [IAM role for SageMaker HyperPod](./sagemaker-hyperpod-prerequisites-iam.html#sagemaker-hyperpod-prerequisites-iam-role-for-hyperpod). @@ -79 +156 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 4. For **Instance group IAM role** , choose the IAM role you have created for SageMaker HyperPod resources, following the section [IAM role for SageMaker HyperPod](./sagemaker-hyperpod-prerequisites-iam.html#sagemaker-hyperpod-prerequisites-iam-role-for-hyperpod). +Configure the FSx for Lustre file system to be provisioned on the HyperPod cluster. @@ -81 +158 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 5. Under **Environment Config - FSx for Lustre** , you can set up the following optional configurations. + 1. For **File system** , choose an existing FSx for Lustre file system, to create a new FSx for Lustre file system, or don't provision an FSx for Lustre file system. @@ -83 +160 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 1. For **Throughput per unit of storage** , choose the unit of storage you need. + 2. For **Throughput per unit of storage** , choose the throughput that will be available per TiB of provisioned storage. @@ -85 +162 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 2. For **Storage capacity** , enter the value you need. + 3. For **Storage capacity** , enter a capacity value in TB. @@ -87 +164 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 6. For **Cluster resiliency - deep health checks of accelerated computing instances - optional** , choose the options(s) based on your use case. To learn more, see [Deep health checks](./sagemaker-hyperpod-eks-resiliency-deep-health-checks.html). + 4. For **Data compression type** , choose **LZ4** to enable data compression. @@ -89 +166 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 7. For **Advanced configuration** : + 5. For **Lustre version** , view the value that's recommended for the new file systems. @@ -91 +167,0 @@ On **Create an instance group** pop-up page, choose **Restricted Instance Group - 1. In **Threads per core** , choose the number you need.