AWS sagemaker documentation change
Summary
Updated SageMaker pipeline documentation to include both Python SDK v3 and legacy v2 examples, restructured code blocks, simplified explanations, and removed redundant content.
Security assessment
Changes focus on code structure, SDK version examples, and documentation clarity without addressing security vulnerabilities, weaknesses, or incidents. No security-related terms, vulnerability fixes, or security feature additions are present.
Diff
diff --git a/sagemaker/latest/dg/define-pipeline.md b/sagemaker/latest/dg/define-pipeline.md index aef1c5faa..b7eba9ebf 100644 --- a//sagemaker/latest/dg/define-pipeline.md +++ b//sagemaker/latest/dg/define-pipeline.md @@ -166,0 +167,21 @@ Create a new SageMaker AI session using the following code block. This returns t +SageMaker Python SDK v3 + + + + import boto3 + import sagemaker + from sagemaker.core.workflow.pipeline_context import PipelineSession + from sagemaker.core.helper.session_helper import Session, get_execution_role + + region = boto3.Session().region_name + sagemaker_session = Session() + role = get_execution_role() + default_bucket = sagemaker_session.default_bucket() + + pipeline_session = PipelineSession() + + model_package_group_name = f"AbaloneModelPackageGroupName" + +SageMaker Python SDK v2 (Legacy) + + @@ -190,14 +211 @@ Custom IAM policies that allow Amazon SageMaker Studio or Amazon SageMaker Studi -Run the following steps from your SageMaker AI notebook instance to create a pipeline that includes steps for: - - * preprocessing - - * training - - * evaluation - - * conditional evaluation - - * model registration - - - +Run the following steps from your SageMaker AI notebook instance to create a pipeline that includes steps for preprocessing, training, evaluation, conditional evaluation, and model registration. @@ -207 +215 @@ Run the following steps from your SageMaker AI notebook instance to create a pip -You can use [ExecutionVariables](https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#execution-variables) and the [ Join](https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#execution-variables) function to specify your output location. `ExecutionVariables` is resolved at runtime. For instance, `ExecutionVariables.PIPELINE_EXECUTION_ID` is resolved to the ID of the current execution, which can be used as a unique identifier across different runs. +You can use [ExecutionVariables](https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#execution-variables) and the [ Join](https://sagemaker.readthedocs.io/en/stable/workflows/pipelines/sagemaker.workflow.pipelines.html#execution-variables) function to specify your output location. @@ -278 +286 @@ The number of rings in the abalone shell is a good approximation for its age usi -This code block defines the following parameters for your pipeline: +SageMaker Python SDK v3 @@ -280 +287,0 @@ This code block defines the following parameters for your pipeline: - * `processing_instance_count` – The instance count of the processing job. @@ -282 +288,0 @@ This code block defines the following parameters for your pipeline: - * `input_data` – The Amazon S3 location of the input data. @@ -284,3 +290,4 @@ This code block defines the following parameters for your pipeline: - * `batch_data` – The Amazon S3 location of the input data for batch transformation. - - * `model_approval_status` – The approval status to register the trained model with for CI/CD. For more information, see [MLOps Automation With SageMaker Projects](./sagemaker-projects.html). + from sagemaker.core.workflow.parameters import ( + ParameterInteger, + ParameterString, + ) @@ -287,0 +295,4 @@ This code block defines the following parameters for your pipeline: + processing_instance_count = ParameterInteger(name="ProcessingInstanceCount", default_value=1) + model_approval_status = ParameterString(name="ModelApprovalStatus", default_value="PendingManualApproval") + input_data = ParameterString(name="InputData", default_value=input_data_uri) + batch_data = ParameterString(name="BatchData", default_value=batch_data_uri) @@ -288,0 +300 @@ This code block defines the following parameters for your pipeline: +SageMaker Python SDK v2 (Legacy) @@ -297,16 +309,4 @@ This code block defines the following parameters for your pipeline: - processing_instance_count = ParameterInteger( - name="ProcessingInstanceCount", - default_value=1 - ) - model_approval_status = ParameterString( - name="ModelApprovalStatus", - default_value="PendingManualApproval" - ) - input_data = ParameterString( - name="InputData", - default_value=input_data_uri, - ) - batch_data = ParameterString( - name="BatchData", - default_value=batch_data_uri, - ) + processing_instance_count = ParameterInteger(name="ProcessingInstanceCount", default_value=1) + model_approval_status = ParameterString(name="ModelApprovalStatus", default_value="PendingManualApproval") + input_data = ParameterString(name="InputData", default_value=input_data_uri) + batch_data = ParameterString(name="BatchData", default_value=batch_data_uri) @@ -320,9 +320 @@ This section shows how to create a processing step to prepare the data from the - 1. Create a directory for the processing script. - - !mkdir -p abalone - - 2. Create a file in the `/abalone` directory named `preprocessing.py` with the following content. This preprocessing script is passed in to the processing step for running on the input data. The training step then uses the preprocessed training features and labels to train a model. The evaluation step uses the trained model and preprocessed test features and labels to evaluate the model. The script uses `scikit-learn` to do the following: - - * Fill in missing `sex` categorical data and encode it so it's suitable for training. - - * Scale and normalize all numerical fields except for `rings` and `sex`. + 1. Create an instance of a processor to pass in to the processing step. @@ -330 +322 @@ This section shows how to create a processing step to prepare the data from the - * Split the data into training, test, and validation datasets. +SageMaker Python SDK v3 @@ -332,7 +323,0 @@ This section shows how to create a processing step to prepare the data from the - %%writefile abalone/preprocessing.py - import argparse - import os - import requests - import tempfile - import numpy as np - import pandas as pd @@ -339,0 +325,2 @@ This section shows how to create a processing step to prepare the data from the + from sagemaker.core.processing import Processor + from sagemaker.core import image_uris @@ -341,55 +328 @@ This section shows how to create a processing step to prepare the data from the - from sklearn.compose import ColumnTransformer - from sklearn.impute import SimpleImputer - from sklearn.pipeline import Pipeline - from sklearn.preprocessing import StandardScaler, OneHotEncoder - - - # Because this is a headerless CSV file, specify the column names here. - feature_columns_names = [ - "sex", - "length", - "diameter", - "height", - "whole_weight", - "shucked_weight", - "viscera_weight", - "shell_weight", - ] - label_column = "rings" - - feature_columns_dtype = { - "sex": str, - "length": np.float64, - "diameter": np.float64, - "height": np.float64, - "whole_weight": np.float64, - "shucked_weight": np.float64, - "viscera_weight": np.float64, - "shell_weight": np.float64 - } - label_column_dtype = {"rings": np.float64} - - - def merge_two_dicts(x, y): - z = x.copy() - z.update(y) - return z - - - if __name__ == "__main__": - base_dir = "/opt/ml/processing" - - df = pd.read_csv( - f"{base_dir}/input/abalone-dataset.csv", - header=None, - names=feature_columns_names + [label_column], - dtype=merge_two_dicts(feature_columns_dtype, label_column_dtype) - ) - numeric_features = list(feature_columns_names) - numeric_features.remove("sex") - numeric_transformer = Pipeline( - steps=[ - ("imputer", SimpleImputer(strategy="median")), - ("scaler", StandardScaler()) - ] - ) + framework_version = "0.23-1" @@ -397,6 +330,5 @@ This section shows how to create a processing step to prepare the data from the - categorical_features = ["sex"] - categorical_transformer = Pipeline( - steps=[ - ("imputer", SimpleImputer(strategy="constant", fill_value="missing")), - ("onehot", OneHotEncoder(handle_unknown="ignore")) - ] + sklearn_image_uri = image_uris.retrieve( + framework="sklearn", + region=region, + version=framework_version, + instance_type="ml.m5.xlarge" @@ -405,5 +337,7 @@ This section shows how to create a processing step to prepare the data from the - preprocess = ColumnTransformer( - transformers=[ - ("num", numeric_transformer, numeric_features), - ("cat", categorical_transformer, categorical_features) - ] + sklearn_processor = Processor( + image_uri=sklearn_image_uri, + instance_type="ml.m5.xlarge", + instance_count=processing_instance_count, + base_job_name="sklearn-abalone-process", + sagemaker_session=pipeline_session, + role=role, @@ -412,5 +346 @@ This section shows how to create a processing step to prepare the data from the - y = df.pop("rings") - X_pre = preprocess.fit_transform(df) - y_pre = y.to_numpy().reshape(len(y), 1) -