AWS sagemaker documentation change
Summary
Restructured deployment documentation from Jupyter notebook format to terminal commands, removed Python helper functions for YAML generation, added direct YAML examples, simplified setup steps, and updated cleanup/verification procedures
Security assessment
Changes focus on workflow simplification and environment switching (notebook to CLI). No security vulnerabilities or explicit security feature additions are mentioned. Removal of TLS certificate configuration in YAML could potentially reduce security controls, but this isn't explicitly addressed as a security fix
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md b/sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md index e1cf59f4d..8e92dc07c 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-model-deployment-deploy-ftm.md @@ -11 +11 @@ The following steps show you how to deploy models stored on Amazon S3 or Amazon -The following instructions contain code cells and commands designed to run in a Jupyter notebook environment, such as Amazon SageMaker Studio or SageMaker Notebook Instances. Each code block represents a notebook cell that should be executed sequentially. The interactive elements, including model discovery tables and status monitoring commands, are optimized for the notebook interface and may not function properly in other environments. Ensure you have access to a notebook environment with the necessary AWS permissions before proceeding. +The following instructions contain code cells and commands designed to run in a terminal. Ensure you have configured your environment with AWS credentials before executing these commands. @@ -15 +15 @@ The following instructions contain code cells and commands designed to run in a -Verify that you’ve set up inference capabilities on your Amazon SageMaker HyperPod clusters. For more information, see [Setting up your HyperPod clusters for model deployment](./sagemaker-hyperpod-model-deployment-setup.html). +Before you begin, verify that you've: @@ -17,8 +17 @@ Verify that you’ve set up inference capabilities on your Amazon SageMaker Hype -## Setup and configuration - -Replace all placeholder values with your actual resource identifiers. - - 1. Initialize your cluster name. This identifies the HyperPod cluster where your model will be deployed. - - # Specify your hyperpod cluster name here - hyperpod_cluster_name="<Hyperpod_cluster_name>" + * Set up inference capabilities on your Amazon SageMaker HyperPod clusters. For more information, see [Setting up your HyperPod clusters for model deployment](./sagemaker-hyperpod-model-deployment-setup.html). @@ -26,4 +19 @@ Replace all placeholder values with your actual resource identifiers. - # NOTE: For sample deployment, we use g5.8xlarge for deepseek-r1 1.5b model which has sufficient memory and GPU - instance_type="ml.g5.8xlarge" - - 2. Initialize your cluster namespace. Your cluster admin should've already created a hyperpod-inference service account in your namespace. + * Installed [kubectl](https://kubernetes.io/docs/reference/kubectl/) utility and configured [jq](https://jqlang.org/) in your terminal. @@ -31,114 +20,0 @@ Replace all placeholder values with your actual resource identifiers. - cluster_namespace="<namespace>" - - 3. Define the helper method to create YAML files for deployment - -The following helper function generates the Kubernetes YAML configuration files needed to deploy your model. This function creates different YAML structures depending on whether your model is stored on Amazon S3 or Amazon FSx, handling the storage-specific configurations automatically. You'll use this function in the next sections to generate the deployment files for your chosen storage backend. - - def generate_inferenceendpointconfig_yaml(deployment_name, model_id, namespace, instance_type, output_file_path, region, tls_certificate_s3_location, model_location, sagemaker_endpoint_name, fsxFileSystemId="", isFsx=False, s3_bucket=None): - """ - Generate a InferenceEndpointConfig YAML file for S3 storage with the provided parameters. - - Args: - deployment_name (str): The deployment name - model_id (str): The model ID - namespace (str): The namespace - instance_type (str): The instance type - output_file_path (str): Path where the YAML file will be saved - region (str): Region where bucket exists - tls_certificate_s3_location (str): S3 location for TLS certificate - model_location (str): Location of the model - sagemaker_endpoint_name (str): Name of the SageMaker endpoint - fsxFileSystemId (str): FSx filesystem ID (optional) - isFsx (bool): Whether to use FSx storage (optional) - s3_bucket (str): S3 bucket where model exists (optional, only needed when isFsx is False) - """ - - # Create the YAML structure - model_config = { - "apiVersion": "inference.sagemaker.aws.amazon.com/v1alpha1", - "kind": "InferenceEndpointConfig", - "metadata": { - "name": deployment_name, - "namespace": namespace - }, - "spec": { - "modelName": model_id, - "endpointName": sagemaker_endpoint_name, - "invocationEndpoint": "invocations", - "instanceType": instance_type, - "modelSourceConfig": {}, - "worker": { - "resources": { - "limits": { - "nvidia.com/gpu": 1, - }, - "requests": { - "nvidia.com/gpu": 1, - "cpu": "30000m", - "memory": "100Gi" - } - }, - "image": "763104351884.dkr.ecr.us-east-2.amazonaws.com/huggingface-pytorch-tgi-inference:2.4.0-tgi2.3.1-gpu-py311-cu124-ubuntu22.04-v2.0", - "modelInvocationPort": { - "containerPort": 8080, - "name": "http" - }, - "modelVolumeMount": { - "name": "model-weights", - "mountPath": "/opt/ml/model" - }, - "environmentVariables": [ - { - "name": "HF_MODEL_ID", - "value": "/opt/ml/model" - }, - { - "name": "SAGEMAKER_PROGRAM", - "value": "inference.py", - }, - { - "name": "SAGEMAKER_SUBMIT_DIRECTORY", - "value": "/opt/ml/model/code", - }, - { - "name": "MODEL_CACHE_ROOT", - "value": "/opt/ml/model" - }, - { - "name": "SAGEMAKER_ENV", - "value": "1", - } - ] - }, - "tlsConfig": { - "tlsCertificateOutputS3Uri": tls_certificate_s3_location, - } - }, - } - - if (not isFsx): - if s3_bucket is None: - raise ValueError("s3_bucket is required when isFsx is False") - model_config["spec"]["modelSourceConfig"] = { - "modelSourceType": "s3", - "s3Storage": { - "bucketName": s3_bucket, - "region": region, - }, - "modelLocation": model_location - } - else: - model_config["spec"]["modelSourceConfig"] = { - "modelSourceType": "fsx", - "fsxStorage": { - "fileSystemId": fsxFileSystemId, - }, - "modelLocation": model_location - } - - - # Write to YAML file - with open(output_file_path, 'w') as file: - yaml.dump(model_config, file, default_flow_style=False) - - print(f"YAML file created successfully at: {output_file_path}") @@ -147,0 +24 @@ The following helper function generates the Kubernetes YAML configuration files +## Setup and configuration @@ -149 +26 @@ The following helper function generates the Kubernetes YAML configuration files -## Deploy your model from Amazon S3 or Amazon FSx +Replace all placeholder values with your actual resource identifiers. @@ -151 +28 @@ The following helper function generates the Kubernetes YAML configuration files -Stage the model to Amazon S3 + 1. Select your Region in your environment. @@ -152,0 +30 @@ Stage the model to Amazon S3 + export REGION=<region> @@ -154,120 +32 @@ Stage the model to Amazon S3 - 1. Create the Amazon S3 bucket to store your model artifacts. The S3 bucket needs to be in the same Region as your HyperPod cluster. - - s3_client = boto3.client('s3', region_name=region_name, config=boto3_config) - base_name = "hyperpod-inference-s3-beta" - - def get_account_id(): - sts = boto3.client('sts') - return sts.get_caller_identity()["Account"] - - account_id = get_account_id() - s3_bucket = f"{base_name}-{account_id}" - - try: - s3_client.create_bucket( - Bucket=s3_bucket, - CreateBucketConfiguration={"LocationConstraint": region_name} - ) - print(f"Bucket '{s3_bucket}' is created successfully.") - except botocore.exceptions.ClientError as e: - error_code = e.response["Error"]["Code"] - if error_code in ("BucketAlreadyExists", "BucketAlreadyOwnedByYou"): - print(f"Bucket '{s3_bucket}' already exists. Skipping creation.") - else: - raise # Re-raise unexpected exceptions - - 2. Get Deployment YAML to deploy the model from the S3 bucket data. - - # Get current time in format suitable for endpoint name - current_time = datetime.now().strftime("%Y%m%d%H%M%S") - model_id = "deepseek15b" ## Can be a name of your choice - deployment_name = f"{model_id}-{current_time}" - model_location = "deepseek15b" ## This is the folder on your s3 file where the model is located - sagemaker_endpoint_name=f"{model_id}-{current_time}" - - output_file_path=f"inferenceendpointconfig-s3-model-{model_id}.yaml" - generate_inferenceendpointconfig_yaml( - deployment_name=deployment_name, - model_id=model_id, - model_location=model_location, - namespace=cluster_namespace, - instance_type=instance_type, - output_file_path=output_file_path, - sagemaker_endpoint_name=sagemaker_endpoint_name, - s3_bucket=s3_bucket, - region=region_name, - tls_certificate_s3_location=tls_certificate_s3_location - ) -