AWS sagemaker medium security documentation change
Summary
Added optional user access setup through JumpStart UI with IAM and Kubernetes RBAC configurations, updated IAM policies to restrict S3 access to specific buckets, modified service account details, and adjusted Helm installation steps.
Security assessment
The change restricts S3 bucket permissions from wildcard ('*') to specific bucket ARNs using ${S3_CSI_BUCKET_NAME}, implementing principle of least privilege. Added Kubernetes RBAC roles and IAM policies configure granular access controls for data scientists, preventing excessive permissions. These changes directly address potential security risks of overprivileged access to S3 resources and cluster operations.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md b/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md index c5b02a0bf..41d88a5db 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-setup.md @@ -5 +5 @@ -PrerequisitesPrepare your environment for inference operator installationCreate the KEDA operator roleInstall the inference operatorVerify the inference operator is working +PrerequisitesPrepare your environment for inference operator installationCreate the KEDA operator roleInstall the inference operatorVerify the inference operator is working(Optional) Set up user access through the JumpStart UI in SageMaker AI Studio Classic @@ -13 +13,3 @@ This guide provides you with a comprehensive setup guide for enabling inference -Before proceeding, verify that your AWS credentials are properly configured and have the necessary permissions. Verify that you've created a HyperPod cluster with [Creating a SageMaker HyperPod cluster](./sagemaker-hyperpod-eks-operate-console-ui-create-cluster.html) . +Before proceeding, verify that your AWS credentials are properly configured and have the necessary permissions. The following steps need to be run by an IAM principal with Administrator privileges and Cluster Admin access to an EKS cluster. Verify that you've created a HyperPod cluster with [Creating a SageMaker HyperPod cluster](./sagemaker-hyperpod-eks-operate-console-ui-create-cluster.html) . Verify you have installed helm, eksctl, and kubectl command line utilities. + +For Kubernetes administrative access to the EKS cluster, go to the EKS console and select the cluster you are using. Look in the **Access** tab and select IAM Access Entries. If there isn't an entry for your IAM principal, select **Create Access Entry**. Then select the desired IAM principal and associate the `AmazonEKSClusterAdminPolicy` with it. @@ -39,2 +41,2 @@ Before proceeding, verify that your AWS credentials are properly configured and - HYPERPOD_INFERENCE_SA_NAME="hyperpod-inference-service-account" - HYPERPOD_INFERENCE_SA_NAMESPACE="kube-system" + HYPERPOD_INFERENCE_SA_NAME="hyperpod-inference-operator-controller" + HYPERPOD_INFERENCE_SA_NAMESPACE="hyperpod-inference-system" @@ -125 +127 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - "arn:aws:s3:::$BUCKET_NAME" + "arn:aws:s3:::$BUCKET_NAME", @@ -184 +185,0 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - "eks:AccessKubernetesApi", @@ -214 +215,3 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - "sagemaker:DescribeHubContent" + "sagemaker:DescribeHubContent", + "sagemaker:UpdateClusterInference", + "sagemaker:DescribeCluster" @@ -268,3 +270,0 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - # Create the IAM role - eksctl create iamserviceaccount --approve --role-only --name=$HYPERPOD_INFERENCE_SA_NAME --namespace=$HYPERPOD_INFERENCE_SA_NAMESPACE --cluster=$EKS_CLUSTER_NAME --attach-policy-arn=$policy_arn --role-name=$HYPERPOD_INFERENCE_ROLE_NAME --region=$REGION - @@ -341,0 +342,2 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h + export S3_CSI_BUCKET_NAME=“<bucketname_for_mounting_through_filesystem>” + @@ -347,12 +350 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - "Sid": "MountpointFullBucketAccess", - "Effect": "Allow", - "Action": [ - "s3:ListBucket" - ], - "Resource": [ - "arn:aws:s3:::*", - "arn:aws:s3:::*/*" - ] - }, - { - "Sid": "MountpointFullObjectAccess", + "Sid": "MountpointAccess", @@ -360,0 +353 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h + "s3:ListBucket", @@ -367,2 +360,2 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - "arn:aws:s3:::*", - "arn:aws:s3:::*/*" + "arn:aws:s3:::${S3_CSI_BUCKET_NAME}", + "arn:aws:s3:::${S3_CSI_BUCKET_NAME}/*" @@ -379 +372 @@ Complete the next critical preparation steps to ensure your Amazon EKS cluster h - cat <<EOF>> s3accesstrustpolicy.json + cat <<EOF> s3accesstrustpolicy.json @@ -622,0 +616 @@ Access the helm chart from [https://github.com/aws/sagemaker-hyperpod-cli/tree/m + helm dependencies update charts/inference-operator @@ -640 +634 @@ Access the helm chart from [https://github.com/aws/sagemaker-hyperpod-cli/tree/m - helm install hyperpod-inference-operator charts/inference-operator + helm install hyperpod-inference-operator charts/inference-operator \ @@ -743,0 +738,172 @@ Access the helm chart from [https://github.com/aws/sagemaker-hyperpod-cli/tree/m +## (Optional) Set up user access through the JumpStart UI in SageMaker AI Studio Classic + +For more background on setting up SageMaker HyperPod access for Studio Classic users and configuring fine-grained Kubernetes RBAC permissions for data scientist users, read [Setting up an Amazon EKS cluster in Studio](./sagemaker-hyperpod-studio-setup-eks.html) and [Setting up Kubernetes role-based access control](./sagemaker-hyperpod-eks-setup-rbac.html). + + 1. Identify the IAM role that Data Scientist users will use to manage and deploy models to SageMaker HyperPod from SageMaker AI Studio Classic. This is typically the User Profile Execution Role or Domain Execution Role for the Studio Classic user. + + %%bash -x + + export DATASCIENTIST_ROLE_NAME="<Execution Role Name used in SageMaker Studio Classic>" + + export DATASCIENTIST_POLICY_NAME="HyperPodUIAccessPolicy" + export EKS_CLUSTER_ARN=$(aws --region $REGION sagemaker describe-cluster --cluster-name $HYPERPOD_CLUSTER_NAME \ + --query 'Orchestrator.Eks.ClusterArn' --output text) + + export DATASCIENTIST_HYPERPOD_NAMESPACE="team-namespace" + + 2. Attach an Identity Policy enabling Model Deployment access. + + %%bash -x + + # Create access policy + cat << EOF > hyperpod-deployment-ui-access-policy.json + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "DescribeHyerpodClusterPermissions", + "Effect": "Allow", + "Action": [ + "sagemaker:DescribeCluster" + ], + "Resource": "$HYPERPOD_CLUSTER_ARN" + }, + { + "Sid": "UseEksClusterPermissions", + "Effect": "Allow", + "Action": [ + "eks:DescribeCluster", + "eks:AccessKubernetesApi", + "eks:MutateViaKubernetesApi", + "eks:DescribeAddon" + ], + "Resource": "$EKS_CLUSTER_ARN" + }, + { + "Sid": "ListPermission", + "Effect": "Allow", + "Action": [ + "sagemaker:ListClusters", + "sagemaker:ListEndpoints" + ], + "Resource": "*" + }, + { + "Sid": "SageMakerEndpointAccess", + "Effect": "Allow", + "Action": [ + "sagemaker:DescribeEndpoint", + "sagemaker:InvokeEndpoint" + ], + "Resource": "arn:aws:sagemaker:$REGION:$ACCOUNT_ID:endpoint/*" + } + ] + } + EOF + + aws iam put-role-policy --role-name DATASCIENTIST_ROLE_NAME --policy-name HyperPodDeploymentUIAccessInlinePolicy --policy-document file://hyperpod-deployment-ui-access-policy.json + + 3. Create an EKS Access Entry for the user mapping them to a kubernetes group. + + %%bash -x + + aws eks create-access-entry --cluster-name $EKS_CLUSTER_NAME \ + --principal-arn "arn:aws:iam::$ACCOUNT_ID:role/$DATASCIENTIST_ROLE_NAME" \ + --kubernetes-groups '["hyperpod-scientist-user-namespace-level","hyperpod-scientist-user-cluster-level"]' + + + 4. Create Kubernetes RBAC policies for the user. + + %%bash -x + + cat << EOF > cluster_level_config.yaml + kind: ClusterRole + apiVersion: rbac.authorization.k8s.io/v1 + metadata: + name: hyperpod-scientist-user-cluster-role + rules: + - apiGroups: [""] + resources: ["pods"] + verbs: ["list"] + - apiGroups: [""] + resources: ["nodes"] + verbs: ["list"] + - apiGroups: [""] + resources: ["namespaces"] + verbs: ["list"] + --- + apiVersion: rbac.authorization.k8s.io/v1 + kind: ClusterRoleBinding + metadata: + name: hyperpod-scientist-user-cluster-role-binding + subjects: + - kind: Group + name: hyperpod-scientist-user-cluster-level + apiGroup: rbac.authorization.k8s.io + roleRef: + kind: ClusterRole + name: hyperpod-scientist-user-cluster-role + apiGroup: rbac.authorization.k8s.io + EOF + + + kubectl apply -f cluster_level_config.yaml + + + cat << EOF > namespace_level_role.yaml + kind: Role + apiVersion: rbac.authorization.k8s.io/v1 + metadata: + namespace: $DATASCIENTIST_HYPERPOD_NAMESPACE + name: hyperpod-scientist-user-namespace-level-role + rules: + - apiGroups: [""] + resources: ["pods"] + verbs: ["create", "get"] + - apiGroups: [""] + resources: ["nodes"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["pods/log"] + verbs: ["get", "list"] + - apiGroups: [""] + resources: ["pods/exec"] + verbs: ["get", "create"] + - apiGroups: ["kubeflow.org"] + resources: ["pytorchjobs", "pytorchjobs/status"]