AWS sagemaker documentation change
Summary
Complete restructuring of documentation to add three installation methods (Console, AWS CLI, Helm) with detailed security configurations including IAM role setups, OIDC federation, TLS certificate management,and secure access controls for inference operations
Security assessment
The changes add security documentation for IAM role trust policies (including web identity federation), least-privilege permissions for S3 buckets, TLS certificate management via cert-manager, and secure model access controls. However, there is no evidence these changes address an existing vulnerability or security incident - they appear to be standard security best practice documentation for new features.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md b/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md index 81bd1ab8e..54b2db4a6 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md @@ -5 +5 @@ -PrerequisitesPrepare your environment for inference operator installationCreate the KEDA operator roleInstall the inference operatorVerify the inference operator is working(Optional) Set up user access through the JumpStart UI in SageMaker AI Studio Classic +Method 1: Install HyperPod Inference Add-on through SageMaker AI console (Recommended)Method 2: Installing the Inference Operator using the AWS CLIMethod 3: Helm chart installationVerify the inference operator is working(Optional) Set up user access through the JumpStart UI in SageMaker AI Studio Classic @@ -9 +9 @@ PrerequisitesPrepare your environment for inference operator installationCreate -This guide provides you with a comprehensive setup guide for enabling inference capabilities on Amazon SageMaker HyperPod clusters. The following steps help you set up the infrastructure, permissions, and operators required to support machine learning engineers in deploying and managing inference endpoints. +This guide shows you how to enable inference capabilities on Amazon SageMaker HyperPod clusters. You'll set up the infrastructure, permissions, and operators that machine learning engineers need to deploy and manage inference endpoints. @@ -15 +15,688 @@ To create a cluster with the inference operator pre-installed, see [Create an EK -## Prerequisites +You can install the inference operator using the SageMaker AI console for a streamlined experience, or use the AWS CLI for more control. This guide covers both installation methods. + +## Method 1: Install HyperPod Inference Add-on through SageMaker AI console (Recommended) + +The SageMaker AI console provides the most streamlined experience with two installation options: + + * **Quick Install:** Automatically creates all required resources with optimized defaults, including IAM roles, Amazon S3 buckets, and dependency add-ons. A new Studio domain will be created with required permissions to deploy a JumpStart model to the relevant cluster. This option is ideal for getting started quickly with minimal configuration decisions. + + * **Custom Install:** Provides flexibility to specify existing resources or customize configurations while maintaining the one-click experience. Customers can choose to reuse existing IAM roles, Amazon S3 buckets, or dependency add-ons based on their organizational requirements. + + + + +### Prerequisites + + * An existing HyperPod cluster with Amazon EKS orchestration + + * IAM permissions for Amazon EKS cluster administration + + * kubectl configured for cluster access + + + + +### Installation steps + + 1. Navigate to the SageMaker AI console and go to **HyperPod Clusters** → **Cluster Management**. + + 2. Select your cluster where you want to install the Inference Operator. + + 3. Navigate to the **Inference** tab. Select **Quick Install** for automated setup or **Custom Install** for configuration flexibility. + + 4. If choosing Custom Install, specify existing resources or customize settings as needed. + + 5. Click **Install** to begin the automated installation process. + + 6. Verify the installation status through the console, or by running the following commands: + + kubectl get pods -n hyperpod-inference-system + + aws eks describe-addon --cluster-name CLUSTER-NAME --addon-name amazon-sagemaker-hyperpod-inference --region REGION + + + + +After the add-on is successfully installed, you can deploy models using the model deployment documentation or navigate to Verify the inference operator is working. + +## Method 2: Installing the Inference Operator using the AWS CLI + +The AWS CLI installation method provides more control over the installation process and is suitable for automation and advanced configurations. + +### Prerequisites + +The inference operator enables deployment and management of machine learning inference endpoints on your Amazon EKS cluster. Before installation, ensure your cluster has the required security configurations and supporting infrastructure. Complete these steps to configure IAM roles, install the AWS Load Balancer Controller, set up Amazon S3 and Amazon FSx CSI drivers, and deploy KEDA and cert-manager: + + 1. Connect to your cluster and set up environment variables + + 2. Configure IAM roles for inference operator + + 3. Create the ALB Controller role + + 4. Create the KEDA operator role + + 5. Install the dependency EKS Add-Ons + + + + +###### Note + +Alternatively, you can use CloudFormation templates to automate the prerequisite setup. For more information, see Using CloudFormation templates to create the prerequisite stack. + +### Connect to your cluster and set up environment variables + +Before proceeding, verify that your AWS credentials are properly configured and have the necessary permissions. Run the following steps using an IAM principal with Administrator privileges and Cluster Admin access to an Amazon EKS cluster. Ensure you've created a HyperPod cluster with [Creating a SageMaker HyperPod cluster with Amazon EKS orchestration](./sagemaker-hyperpod-eks-operate-console-ui-create-cluster.html). Install helm, eksctl, and kubectl command line utilities. + +For Kubernetes administrative access to the Amazon EKS cluster, open the Amazon EKS console and select your cluster. In the **Access** tab, select **IAM Access Entries**. If no entry exists for your IAM principal, select **Create Access Entry**. Select the desired IAM principal and associate the `AmazonEKSClusterAdminPolicy` with it. + + 1. Configure kubectl to connect to the newly created HyperPod cluster orchestrated by Amazon EKS cluster. Specify the Region and HyperPod cluster name. + + export HYPERPOD_CLUSTER_NAME=<hyperpod-cluster-name> + export REGION=<region> + + # S3 bucket where tls certificates will be uploaded + export BUCKET_NAME="hyperpod-tls-<your-bucket-suffix>" # Bucket should have prefix: hyperpod-tls-* + + export EKS_CLUSTER_NAME=$(aws --region $REGION sagemaker describe-cluster --cluster-name $HYPERPOD_CLUSTER_NAME \ + --query 'Orchestrator.Eks.ClusterArn' --output text | \ + cut -d'/' -f2) + aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $REGION + +###### Note + +If using a custom bucket name that doesn't start with `hyperpod-tls-`, attach the following policy to your execution role: + + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "TLSBucketDeleteObjectsPermission", + "Effect": "Allow", + "Action": ["s3:DeleteObject"], + "Resource": ["arn:aws:s3:::${BUCKET_NAME}/*"], + "Condition": { + "StringEquals": { + "aws:ResourceAccount": "${aws:PrincipalAccount}" + } + } + }, + { + "Sid": "TLSBucketGetObjectAccess", + "Effect": "Allow", + "Action": ["s3:GetObject"], + "Resource": ["arn:aws:s3:::${BUCKET_NAME}/*"] + }, + { + "Sid": "TLSBucketPutObjectAccess", + "Effect": "Allow", + "Action": ["s3:PutObject", "s3:PutObjectTagging"], + "Resource": ["arn:aws:s3:::${BUCKET_NAME}/*"], + "Condition": { + "StringEquals": { + "aws:ResourceAccount": "${aws:PrincipalAccount}" + } + } + } + ] + } + + 2. Set default env variables. + + HYPERPOD_INFERENCE_ROLE_NAME="SageMakerHyperPodInference-$HYPERPOD_CLUSTER_NAME" + HYPERPOD_INFERENCE_NAMESPACE="hyperpod-inference-system" + + 3. Extract the Amazon EKS cluster name from the cluster ARN, update the local kubeconfig, and verify connectivity by listing all pods across namespaces. + + kubectl get pods --all-namespaces + + 4. (Optional) Install the NVIDIA device plugin to enable GPU support on the cluster. + + # Install nvidia device plugin + kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.14.5/nvidia-device-plugin.yml + # Verify that GPUs are visible to k8s + kubectl get nodes -o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia.com/gpu + + + + +### Configure IAM roles for inference operator + + 1. Gather essential AWS resource identifiers and ARNs required for configuring service integrations between Amazon EKS, SageMaker AI, and IAM components. + + %%bash -x + + export ACCOUNT_ID=$(aws --region $REGION sts get-caller-identity --query 'Account' --output text) + export OIDC_ID=$(aws --region $REGION eks describe-cluster --name $EKS_CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | cut -d '/' -f 5) + export EKS_CLUSTER_ROLE=$(aws eks --region $REGION describe-cluster --name $EKS_CLUSTER_NAME --query 'cluster.roleArn' --output text) + + 2. Associate an IAM OIDCidentity provider with your EKS cluster. + + eksctl utils associate-iam-oidc-provider --region=$REGION --cluster=$EKS_CLUSTER_NAME --approve + + 3. Create the trust policy required for the HyperPod inference operator IAM role. These policies enable secure cross-service communication between Amazon EKS, SageMaker AI, and other AWS services. + + %%bash -x + + # Create trust policy JSON + cat << EOF > trust-policy.json + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": [ + "sagemaker.amazonaws.com" + ] + }, + "Action": "sts:AssumeRole" + }, + { + "Effect": "Allow", + "Principal": { + "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/oidc.eks.${REGION}.amazonaws.com/id/${OIDC_ID}" + }, + "Action": "sts:AssumeRoleWithWebIdentity", + "Condition": { + "StringLike": {