AWS sagemaker high security documentation change
Summary
Converted deployment instructions from Jupyter notebook-based workflow to terminal/CLI-focused approach. Removed Python SDK code in favor of AWS CLI and kubectl commands. Added explicit terminal prerequisites (kubectl, jq), restructured setup steps, and introduced YAML generation via heredoc. Removed TLS certificate configuration for model communication.
Security assessment
Security issue introduced by removal of TLS certificate configuration (tlsConfig section in YAML) without replacement. Original version explicitly configured TLS certificates via S3 bucket for secure model communication, while updated version omits this entirely. This weakens transport security for model deployments by default.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-js-kubectl.md b/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-js-kubectl.md index 90d2319b4..4fe59d275 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-js-kubectl.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-js-kubectl.md @@ -5 +5 @@ -PrerequisitesSetup and configuration +PrerequisitesSetup and configurationDeploy your modelManage your deployment @@ -11 +11 @@ The following steps show you how to deploy a JumpStart model to a HyperPod clust -The following instructions contain code cells and commands designed to run in a Jupyter notebook environment, such as Amazon SageMaker Studio or SageMaker Notebook Instances. Each code block represents a notebook cell that should be executed sequentially. The interactive elements, including model discovery tables and status monitoring commands, are optimized for the notebook interface and may not function properly in other environments. Ensure you have access to a notebook environment with the necessary AWS permissions before proceeding. +The following instructions contain code cells and commands designed to run in a terminal. Ensure you have configured your environment with AWS credentials before executing these commands. @@ -15 +15 @@ The following instructions contain code cells and commands designed to run in a -Ensure you've set up inference capabilities on your Amazon SageMaker HyperPod clusters. For more information, see [Setting up your HyperPod clusters for model deployment](./sagemaker-hyperpod-model-deployment-setup.html). +Before you begin, verify that you've: @@ -17,3 +17 @@ Ensure you've set up inference capabilities on your Amazon SageMaker HyperPod cl -## Setup and configuration - -### Select your Region + * Set up inference capabilities on your Amazon SageMaker HyperPod clusters. For more information, see [Setting up your HyperPod clusters for model deployment](./sagemaker-hyperpod-model-deployment-setup.html). @@ -21 +19 @@ Ensure you've set up inference capabilities on your Amazon SageMaker HyperPod cl -Choose the Region to deploy your HyperPod cluster to and where you want to run your inference workloads. You can also add other customizations to the `sagemaker_client`. + * Installed [kubectl](https://kubernetes.io/docs/reference/kubectl/) utility and configured [jq](https://jqlang.org/) in your terminal. @@ -24 +21,0 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - region_name = <REGION> @@ -26,9 +22,0 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - import boto3 - from botocore.config import Config - # Configure retry options - boto3_config = Config( - retries={ - 'max_attempts': 10, # Maximum number of retry attempts - 'mode': 'adaptive' # Use adaptive mode for exponential backoff - } - ) @@ -36 +24 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - sagemaker_client=boto3.client("sagemaker", region_name=region_name, config=boto3_config) +## Setup and configuration @@ -38 +26 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y -### Choose your model and cluster + 1. Select your Region. @@ -40 +28 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - 1. View all SageMaker public hub models and HyperPod clusters. + export REGION=<region> @@ -42 +30 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - interactive_view(get_all_public_hub_model_data(sagemaker_client)) + 2. View all SageMaker public hub models and HyperPod clusters. @@ -44 +32 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - interactive_view(get_all_cluster_data(sagemaker_client)) + 3. Select a `JumpstartModel` from JumpstartPublic Hub. JumpstartPublic hub has a large number of models available so you can use `NextToken` to iteratively list all available models in the public hub. @@ -46 +34 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y - 2. Configure the model ID and cluster name you've selected into the variables below. + aws sagemaker list-hub-contents --hub-name SageMakerPublicHub --hub-content-type Model --query '{Models: HubContentSummaries[].{ModelId:HubContentName,Version:HubContentVersion}, NextToken: NextToken}' --output json @@ -48 +36,2 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y -###### Note + export MODEL_ID="deepseek-llm-r1-distill-qwen-1-5b" + export MODEL_VERSION="2.0.4" @@ -50 +39 @@ Choose the Region to deploy your HyperPod cluster to and where you want to run y -Check with your cluster admin to ensure permissions are granted for this notebook execution role. You can run `!aws sts get-caller-identity --query "Arn"` to check which execution role you are using + 4. Configure the model ID and cluster name you’ve selected into the variables below. @@ -52,5 +41 @@ Check with your cluster admin to ensure permissions are granted for this noteboo - # Change the model_id based on your requirement. A list of model IDs is available in step 1 of this notebook. - # Proprietary models are not supported - model_id = "<insert model id here>" - - from sagemaker.hyperpod.inference.notebook_utils import validate_public_hub_model_is_not_proprietary +###### Note @@ -58 +43 @@ Check with your cluster admin to ensure permissions are granted for this noteboo - validate_public_hub_model_is_not_proprietary(sagemaker_client, model_id) +Check with your cluster admin to ensure permissions are granted for this role or user. You can run `!aws sts get-caller-identity --query "Arn"` to check which role or user you are using in your terminal. @@ -60,2 +45 @@ Check with your cluster admin to ensure permissions are granted for this noteboo - # Select the cluster name where you want to deploy the model. List of clusters is available in step 1 of this notebook. - cluster_name = "<insert cluster name here>" + aws sagemaker list-clusters --output table @@ -63,3 +47,2 @@ Check with your cluster admin to ensure permissions are granted for this noteboo - from sagemaker.hyperpod.inference.notebook_utils import validate_cluster_can_support_public_hub_modelfrom sagemaker.hyperpod.inference.notebook_utils import get_public_hub_model_compatible_instances - validate_cluster_can_support_public_hub_model(sagemaker_client, model_id, cluster_name) - interactive_view(get_public_hub_model_compatible_instances(sagemaker_client, model_id)) + # Select the cluster name where you want to deploy the model. + export HYPERPOD_CLUSTER_NAME="<insert cluster name here>" @@ -68,73 +51,2 @@ Check with your cluster admin to ensure permissions are granted for this noteboo - instance_type = "ml.g5.8xlarge" - - 3. Confirm with the cluster admin which namespace you are permitted to use. The admin should have created a hyperpod-inference service account in your namespace. - - cluster_namespace = "default" - - - - -### Configure the S3 bucket name - -Configure the S3 bucket names for certificates. This bucket needs to have a folder named "certificates" where certificates will be uploaded. The bucket also needs to be in the same Region defined above. - - - # Set the S3 bucket name where TLS certificates will be stored for secure model communication - certificate_bucket = "<insert bucket name here>" - - - import yaml - from datetime import datetime - - # Get current time in format suitable for endpoint name - current_time = datetime.now().strftime("%Y%m%d-%H%M%S") - sagemaker_endpoint_name=f"{model_id}-{current_time}" - - - def generate_jumpstart_model_yaml(model_id, model_version, namespace, instance_type, output_file_path, certificate_bucket): - """ - Generate a JumpStartModel YAML file with the provided parameters. - - Args: - model_id (str): The model ID - model_version (str): The model version - namespace (str): The namespace - instance_type (str): The instance type - output_file_path (str): Path where the YAML file will be saved - """ - - # Create the YAML structure - tlsCertificateOutputS3Uri = "s3://" + certificate_bucket + "/certificates/" - model_config = { - "apiVersion": "inference.sagemaker.aws.amazon.com/v1alpha1", - "kind": "JumpStartModel", - "metadata": { - "name": model_id, - "namespace": namespace - }, - "spec": { - "sageMakerEndpoint": { - "name": sagemaker_endpoint_name - }, - "model": { - "modelHubName": "SageMakerPublicHub", - "modelId": model_id, - # modelVersion is optional - "modelVersion": model_version - # acceptEula is optional, set value to True when using a gated model - }, - "server": { - "instanceType": instance_type - }, - "tlsConfig": { - "tlsCertificateOutputS3Uri": tlsCertificateOutputS3Uri - } - } - } - - # Write to YAML file - with open(output_file_path, 'w') as file: - yaml.dump(model_config, file, default_flow_style=False) - - print(f"YAML file created successfully at: {output_file_path}") - + # List availble instances in your HyperPod cluster + aws sagemaker describe-cluster --cluster-name=$HYPERPOD_CLUSTER_NAME --query "InstanceGroups[].{InstanceType:InstanceType,Count:CurrentCount}" --output table @@ -142,2 +54,2 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - # Import JumpStart utilities to retrieve model specifications and version information - from sagemaker.jumpstart import utilsfrom sagemaker.jumpstart.enums import JumpStartScriptScope + # List supported instance types for the selected model + aws sagemaker describe-hub-content --hub-name SageMakerPublicHub --hub-content-type Model --hub-content-name "$MODEL_ID" --output json | jq -r '.HubContentDocument | fromjson | {Default: .DefaultInferenceInstanceType, Supported: .SupportedInferenceInstanceTypes}' @@ -145,3 +56,0 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - model_specs = utils.verify_model_region_and_return_specs( - model_id, "*", JumpStartScriptScope.INFERENCE, region=region_name) - model_version = model_specs.version @@ -148,0 +58,3 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold + # Select and instance type from the cluster that is compatible with the model. + # Make sure that the selected instance is either default or supported instance type for the jumpstart model + export INSTANCE_TYPE="<Instance_type_In_cluster" @@ -150,10 +62 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - # Generate the output filename for the Kubernetes YAML configuration - output_file_path=f"jumpstart-model-{model_id}.yaml" - generate_jumpstart_model_yaml( - model_id=model_id, - model_version=model_version, - namespace=cluster_namespace, - instance_type=instance_type, - output_file_path=output_file_path, - certificate_bucket=certificate_bucket - ) + 5. Confirm with the cluster admin which namespace you are permitted to use. The admin should have created a `hyperpod-inference` service account in your namespace. @@ -161,3 +64 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - import os - os.environ["JUMPSTART_YAML_FILE_PATH"]=output_file_path - os.environ["MODEL_ID"]=model_id + export CLUSTER_NAMESPACE="default" @@ -165 +66 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold -### Deploy your model + 6. Set a name for endpoint and custom object to be create. @@ -167 +68 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold -#### Update your kubernetes configuration and deploy your model + export SAGEMAKER_ENDPOINT_NAME="deepsek-qwen-1-5b-test" @@ -169 +70 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - 1. Retrieve the EKS cluster name from HyperPod. + 7. The following is an example for a `deepseek-llm-r1-distill-qwen-1-5b` model deployment from Jumpstart. Create a similar deployment yaml file based on the model selected iin the above step. @@ -171 +72,39 @@ Configure the S3 bucket names for certificates. This bucket needs to have a fold - !aws sagemaker describe-cluster --cluster-name $cluster_name --query "Orchestrator.Eks.ClusterArn" + cat << EOF > jumpstart_model.yaml + --- + apiVersion: inference.sagemaker.aws.amazon.com/v1alpha1 + kind: JumpStartModel + metadata: