AWS sagemaker documentation change
Summary
Updated model training documentation with SageMaker Python SDK v3 examples alongside existing v2 examples, restructured content flow, added version-specific installation notes, and introduced new ModelTrainer class for v3.
Security assessment
Changes focus on SDK version updates, training job configuration improvements, and code restructuring. No security vulnerabilities, patches, or security-specific features are mentioned or addressed in the modifications.
Diff
diff --git a/sagemaker/latest/dg/ex1-train-model.md b/sagemaker/latest/dg/ex1-train-model.md index fba282d55..05256bd8f 100644 --- a//sagemaker/latest/dg/ex1-train-model.md +++ b//sagemaker/latest/dg/ex1-train-model.md @@ -11 +11 @@ Choose the Training AlgorithmCreate and Run a Training Job -In this step, you choose a training algorithm and run a training job for the model. The [Amazon SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable) provides framework estimators and generic estimators to train your model while orchestrating the machine learning (ML) lifecycle accessing the SageMaker AI features for training and the AWS infrastructures, such as Amazon Elastic Container Registry (Amazon ECR), Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3). For more information about SageMaker AI built-in framework estimators, see [Frameworks](https://sagemaker.readthedocs.io/en/stable/frameworks/index.html)in the [Amazon SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable) documentation. For more information about built-in algorithms, see [Built-in algorithms and pretrained models in Amazon SageMaker](./algos.html). +In this step, you choose a training algorithm and run a training job for the model. The [Amazon SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable) provides classes to train your model while orchestrating the machine learning (ML) lifecycle accessing the SageMaker AI features for training and the AWS infrastructures, such as Amazon Elastic Container Registry (Amazon ECR), Amazon Elastic Compute Cloud (Amazon EC2), Amazon Simple Storage Service (Amazon S3). For more information about built-in algorithms, see [Built-in algorithms and pretrained models in Amazon SageMaker](./algos.html). @@ -32 +32 @@ If you want SageMaker AI to find an appropriate model for your tabular dataset, -After you figured out which model to use, start constructing a SageMaker AI estimator for training. This tutorial uses the XGBoost built-in algorithm for the SageMaker AI generic estimator. +After you figured out which model to use, start constructing a training job. This tutorial uses the XGBoost built-in algorithm. @@ -37,0 +38,23 @@ After you figured out which model to use, start constructing a SageMaker AI esti +SageMaker Python SDK v3 + + + from sagemaker.core.helper.session_helper import Session, get_execution_role + + sagemaker_session = Session() + region = sagemaker_session.boto_region_name + print(f"AWS Region: {region}") + + role = get_execution_role() + print(f"RoleArn: {role}") + +###### Note + +Check the SageMaker Python SDK version by running `sagemaker.__version__`. This tutorial is based on `sagemaker>=3.0`. If the SDK is outdated, install the latest version by running the following command: + + ! pip install -qU sagemaker + +If you run this installation in your exiting SageMaker Studio or notebook instances, you need to manually refresh the kernel to finish applying the version update. + +SageMaker Python SDK v2 (Legacy) + + @@ -45,0 +69,8 @@ After you figured out which model to use, start constructing a SageMaker AI esti +###### Note + +Check the SageMaker Python SDK version by running `sagemaker.__version__`. This tutorial is based on `sagemaker>=2.20`. If the SDK is outdated, install the latest version by running the following command: + + ! pip install -qU sagemaker + +If you run this installation in your exiting SageMaker Studio or notebook instances, you need to manually refresh the kernel to finish applying the version update. + @@ -52 +83 @@ This returns the following information: -###### Note + 2. Create a training configuration and set hyperparameters for the XGBoost algorithm. @@ -54 +85 @@ This returns the following information: -Check the SageMaker Python SDK version by running `sagemaker.__version__`. This tutorial is based on `sagemaker>=2.20`. If the SDK is outdated, install the latest version by running the following command: +SageMaker Python SDK v3 @@ -56 +86,0 @@ Check the SageMaker Python SDK version by running `sagemaker.__version__`. This - ! pip install -qU sagemaker @@ -58 +88,44 @@ Check the SageMaker Python SDK version by running `sagemaker.__version__`. This -If you run this installation in your exiting SageMaker Studio or notebook instances, you need to manually refresh the kernel to finish applying the version update. +Create a `ModelTrainer` using the `sagemaker.train.ModelTrainer` class with hyperparameters passed directly in the constructor. In the following example code, the ModelTrainer is named `xgb_model_trainer`. + + from sagemaker.train import ModelTrainer + from sagemaker.train.configs import Compute, OutputDataConfig + from sagemaker.core import image_uris + + s3_output_location='s3://{}/{}/{}'.format(bucket, prefix, 'xgboost_model') + + container = image_uris.retrieve("xgboost", region, "1.2-1") + print(container) + + compute = Compute( + instance_type='ml.m4.xlarge', + instance_count=1, + volume_size_in_gb=5 + ) + + xgb_model_trainer = ModelTrainer( + training_image=container, + role=role, + compute=compute, + output_data_config=OutputDataConfig(s3_output_path=s3_output_location), + hyperparameters={ + "max_depth": "5", + "eta": "0.2", + "gamma": "4", + "min_child_weight": "6", + "subsample": "0.7", + "objective": "binary:logistic", + "num_round": "1000" + } + ) + +To construct the SageMaker AI `ModelTrainer`, specify the following parameters: + + * `training_image` – Specify the training container image URI. In this example, the SageMaker AI XGBoost training container URI is specified using `image_uris.retrieve`. + + * `role` – The AWS Identity and Access Management (IAM) role that SageMaker AI uses to perform tasks on your behalf (for example, reading training results, call model artifacts from Amazon S3, and writing training results to Amazon S3). + + * `compute` – A `Compute` configuration object that specifies the type and number of Amazon EC2 ML compute instances to use for model training. For this training exercise, you use a single `ml.m4.xlarge` instance, which has 4 CPUs, 16 GB of memory, an Amazon Elastic Block Store (Amazon EBS) storage, and a high network performance. For more information about EC2 compute instance types, see [Amazon EC2 Instance Types](https://aws.amazon.com/ec2/instance-types/). For more information about billing, see [Amazon SageMaker pricing](https://aws.amazon.com/sagemaker/pricing/). + + * `hyperparameters` – A dictionary of hyperparameters for the training algorithm. All values must be strings. + +SageMaker Python SDK v2 (Legacy) @@ -60 +133,2 @@ If you run this installation in your exiting SageMaker Studio or notebook instan - 2. Create an XGBoost estimator using the `sagemaker.estimator.Estimator` class. In the following example code, the XGBoost estimator is named `xgb_model`. + +Create an XGBoost estimator using the `sagemaker.estimator.Estimator` class. In the following example code, the XGBoost estimator is named `xgb_model`. @@ -100,5 +174 @@ To construct the SageMaker AI estimator, specify the following parameters: -###### Tip - -If you want to run distributed training of large sized deep learning models, such as convolutional neural networks (CNN) and natural language processing (NLP) models, use SageMaker AI Distributed for data parallelism or model parallelism. For more information, see [Distributed training in Amazon SageMaker AI](./distributed-training.html). - - 3. Set the hyperparameters for the XGBoost algorithm by calling the `set_hyperparameters` method of the estimator. For a complete list of XGBoost hyperparameters, see [XGBoost hyperparameters](./xgboost_hyperparameters.html). +Set the hyperparameters for the XGBoost algorithm by calling the `set_hyperparameters` method of the estimator. For a complete list of XGBoost hyperparameters, see [XGBoost hyperparameters](./xgboost_hyperparameters.html). @@ -117,0 +188,4 @@ If you want to run distributed training of large sized deep learning models, suc +If you want to run distributed training of large sized deep learning models, such as convolutional neural networks (CNN) and natural language processing (NLP) models, use SageMaker AI Distributed for data parallelism or model parallelism. For more information, see [Distributed training in Amazon SageMaker AI](./distributed-training.html). + +###### Tip + @@ -120 +194,22 @@ You can also tune the hyperparameters using the SageMaker AI hyperparameter opti - 4. Use the `TrainingInput` class to configure a data input flow for training. The following example code shows how to configure `TrainingInput` objects to use the training and validation datasets you uploaded to Amazon S3 in the [Split the Dataset into Train, Validation, and Test Datasets](./ex1-preprocess-data.html#ex1-preprocess-data-transform) section. + 3. Configure the data input for training. + +SageMaker Python SDK v3 + + +Use the `InputData` class to configure a data input flow for training. The following example code shows how to configure `InputData` objects to use the training and validation datasets you uploaded to Amazon S3 in the [Split the Dataset into Train, Validation, and Test Datasets](./ex1-preprocess-data.html#ex1-preprocess-data-transform) section. + + from sagemaker.train.configs import InputData + + train_input = InputData( + channel_name="train", + data_source="s3://{}/{}/{}".format(bucket, prefix, "data/train.csv") + ) + validation_input = InputData( + channel_name="validation", + data_source="s3://{}/{}/{}".format(bucket, prefix, "data/validation.csv") + ) + +SageMaker Python SDK v2 (Legacy) + + +Use the `TrainingInput` class to configure a data input flow for training. The following example code shows how to configure `TrainingInput` objects to use the training and validation datasets you uploaded to Amazon S3 in the [Split the Dataset into Train, Validation, and Test Datasets](./ex1-preprocess-data.html#ex1-preprocess-data-transform) section. @@ -131 +226,13 @@ You can also tune the hyperparameters using the SageMaker AI hyperparameter opti - 5. To start model training, call the estimator's `fit` method with the training and validation datasets. By setting `wait=True`, the `fit` method displays progress logs and waits until training is complete. + 4. Start model training. + +SageMaker Python SDK v3 + + +To start model training, call the trainer's `train` method with the training and validation datasets. By default, the `train` method displays progress logs and waits until training is complete. + + xgb_model_trainer.train(input_data_config=[train_input, validation_input]) + +SageMaker Python SDK v2 (Legacy) + + +To start model training, call the estimator's `fit` method with the training and validation datasets. By setting `wait=True`, the `fit` method displays progress logs and waits until training is complete. @@ -142,0 +250,10 @@ Run the following code to specify the S3 bucket URI where the Debugger training +SageMaker Python SDK v3 + + + training_job = xgb_model_trainer._latest_training_job + rule_output_path = training_job.output_data_config.s3_output_path + "/" + training_job.training_job_name + "/rule-output" + ! aws s3 ls {rule_output_path} --recursive + +SageMaker Python SDK v2 (Legacy) + + @@ -156,0 +274,16 @@ The following IPython script returns the file link of the Debugger profiling rep +SageMaker Python SDK v3 + + + # Note: In V3, debugger rule outputs can be accessed via the SageMaker console + # or the boto3 DescribeTrainingJob API (DebugRuleEvaluationStatuses field). + # Example using boto3: + # import boto3 + # sm = boto3.client("sagemaker") + # resp = sm.describe_training_job(TrainingJobName=training_job.training_job_name) + # rule_statuses = resp["DebugRuleEvaluationStatuses"] + profiler_report_name = "ProfilerReport-1234567890" + display("Click link below to view the profiler report", FileLink(profiler_report_name+"/profiler-output/profiler-report.html")) + +SageMaker Python SDK v2 (Legacy) + + @@ -172 +305,10 @@ To identify training issues, such as overfitting, vanishing gradients, and other -You now have a trained XGBoost model. SageMaker AI stores the model artifact in your S3 bucket. To find the location of the model artifact, run the following code to print the model_data attribute of the `xgb_model` estimator: +You now have a trained XGBoost model. SageMaker AI stores the model artifact in your S3 bucket. To find the location of the model artifact, run the following code to print the `model_data` attribute: + +SageMaker Python SDK v3 + + + + xgb_model_trainer._latest_training_job.model_artifacts.s3_model_artifacts + +SageMaker Python SDK v2 (Legacy) +