AWS sagemaker documentation change
Summary
Updated documentation to replace 'SageMaker PyTorch estimator' with 'SageMaker PyTorch ModelTrainer' throughout the file. Added new code examples for SageMaker Python SDK v3 configuration while retaining v2 examples. Enhanced sharded data parallelism implementation guidance.
Security assessment
The changes are documentation updates reflecting SDK changes (estimator to ModelTrainer) and adding new code samples. No security vulnerabilities, exploits, or security features are mentioned. Changes focus on parallelism configuration and performance tuning without security implications.
Diff
diff --git a/sagemaker/latest/dg/model-parallel-extended-features-pytorch-sharded-data-parallelism.md b/sagemaker/latest/dg/model-parallel-extended-features-pytorch-sharded-data-parallelism.md index 854d26295..d8d1448ca 100644 --- a//sagemaker/latest/dg/model-parallel-extended-features-pytorch-sharded-data-parallelism.md +++ b//sagemaker/latest/dg/model-parallel-extended-features-pytorch-sharded-data-parallelism.md @@ -54 +54 @@ After setting up sharded data parallelism, make sure you find the most optimal t -To get started with sharded data parallelism, apply required modifications to your training script, and set up the SageMaker PyTorch estimator with the sharded-data-parallelism-specific parameters. Also consider to take reference values and example notebooks as a starting point. +To get started with sharded data parallelism, apply required modifications to your training script, and set up the SageMaker PyTorch ModelTrainer with the sharded-data-parallelism-specific parameters. Also consider to take reference values and example notebooks as a starting point. @@ -78 +78 @@ If your model is built with `torch.nn.Module` and uses parameters that is not de -### Set up the SageMaker PyTorch estimator +### Set up the SageMaker PyTorch ModelTrainer @@ -80 +80 @@ If your model is built with `torch.nn.Module` and uses parameters that is not de -When configuring a SageMaker PyTorch estimator in [Step 2: Launch a Training Job Using the SageMaker Python SDK](./model-parallel-sm-sdk.html), add the parameters for sharded data parallelism. +When configuring a SageMaker PyTorch ModelTrainer in [Step 2: Launch a Training Job Using the SageMaker Python SDK](./model-parallel-sm-sdk.html), add the parameters for sharded data parallelism. @@ -82 +82 @@ When configuring a SageMaker PyTorch estimator in [Step 2: Launch a Training Job -To turn on sharded data parallelism, add the `sharded_data_parallel_degree` parameter to the SageMaker PyTorch Estimator. This parameter specifies the number of GPUs over which the training state is sharded. The value for `sharded_data_parallel_degree` must be an integer between one and the data parallelism degree and must evenly divide the data parallelism degree. Note that the library automatically detects the number of GPUs so thus the data parallel degree. The following additional parameters are available for configuring sharded data parallelism. +To turn on sharded data parallelism, add the `sharded_data_parallel_degree` parameter to the SageMaker PyTorch ModelTrainer. This parameter specifies the number of GPUs over which the training state is sharded. The value for `sharded_data_parallel_degree` must be an integer between one and the data parallelism degree and must evenly divide the data parallelism degree. Note that the library automatically detects the number of GPUs so thus the data parallel degree. The following additional parameters are available for configuring sharded data parallelism. @@ -98,0 +99,63 @@ The following code shows an example of how to configure sharded data parallelism +SageMaker Python SDK v3 + + + + import sagemaker + from sagemaker.train import ModelTrainer + from sagemaker.train.configs import SourceCode, Compute, InputData + from sagemaker.core import image_uris + from sagemaker.core.helper.session_helper import get_execution_role + + smp_options = { + "enabled": True, + "parameters": { + # "pipeline_parallel_degree": 1, # Optional, default is 1 + # "tensor_parallel_degree": 1, # Optional, default is 1 + "ddp": True, + # parameters for sharded data parallelism + "sharded_data_parallel_degree": 2, # Add this to activate sharded data parallelism + "sdp_reduce_bucket_size": int(5e8), # Optional + "sdp_param_persistence_threshold": int(1e6), # Optional + "sdp_max_live_parameters": int(1e9), # Optional + "sdp_hierarchical_allgather": True, # Optional + "sdp_gradient_clipping": 1.0 # Optional + } + } + + mpi_options = { + "enabled" : True, # Required + "processes_per_host" : 8 # Required + } + + # Retrieve the training image for the desired PyTorch version + training_image = image_uris.retrieve( + framework="pytorch", + region="us-west-2", + version='1.13.1', + py_version='py3', + instance_type='ml.p3.16xlarge', + image_scope="training" + ) + + smp_model_trainer = ModelTrainer( + training_image=training_image, + source_code=SourceCode(entry_script="your_training_script.py"), + role=get_execution_role(), + compute=Compute( + instance_type='ml.p3.16xlarge', + instance_count=1 + ), + distribution={ + "smdistributed": {"modelparallel": smp_options}, + "mpi": mpi_options + }, + base_job_name="sharded-data-parallel-job" + ) + + smp_model_trainer.train(input_data_config=[ + InputData(channel_name="training", data_source='s3://my_bucket/my_training_data/') + ]) + +SageMaker Python SDK v2 (Legacy) + + @@ -180 +243,62 @@ Since the SageMaker model parallelism library version 1.13.0, the `"ddp_dist_bac -The following code example shows how to set up a PyTorch estimator using the sharded data parallelism with the `"ddp_dist_backend"` parameter, which is set to `"auto"` by default and, therefore, optional to add. +The following code example shows how to set up a PyTorch ModelTrainer using the sharded data parallelism with the `"ddp_dist_backend"` parameter, which is set to `"auto"` by default and, therefore, optional to add. + +SageMaker Python SDK v3 + + + + import sagemaker + from sagemaker.train import ModelTrainer + from sagemaker.train.configs import SourceCode, Compute, InputData + from sagemaker.core import image_uris + from sagemaker.core.helper.session_helper import get_execution_role + + smp_options = { + "enabled":True, + "parameters": { + "partitions": 1, + "ddp": True, + "sharded_data_parallel_degree": 64 + "bf16": True, + **"ddp_dist_backend": "auto" # Specify "nccl" to force to use NCCL.** + } + } + + mpi_options = { + "enabled" : True, # Required + "processes_per_host" : 8 # Required + } + + # Retrieve the training image for the desired PyTorch version + training_image = image_uris.retrieve( + framework="pytorch", + region="us-west-2", + version='1.13.1', + py_version='py3', + instance_type='ml.p4d.24xlarge', + image_scope="training" + ) + + smd_mp_model_trainer = ModelTrainer( + training_image=training_image, + source_code=SourceCode( + source_dir="location_to_your_script", + entry_script="your_training_script.py" + ), + role=get_execution_role(), + compute=Compute( + instance_type='ml.p4d.24xlarge', + instance_count=8 + ), + distribution={ + "smdistributed": {"modelparallel": smp_options}, + "mpi": mpi_options + }, + base_job_name="sharded-data-parallel-demo", + ) + + smd_mp_model_trainer.train(input_data_config=[ + InputData(channel_name="training", data_source='s3://my_bucket/my_training_data/') + ]) + +SageMaker Python SDK v2 (Legacy) + @@ -281 +405,57 @@ Some collectives might fall back to use NCCL; hence, you might not get the perfo -The following code shows how you can configure the environment variables by appending them to `mpi_options` in the distribution parameter for the PyTorch estimator. +The following code shows how you can configure the environment variables by appending them to `mpi_options` in the distribution parameter for the PyTorch ModelTrainer. + +SageMaker Python SDK v3 + + + + import sagemaker + from sagemaker.train import ModelTrainer + from sagemaker.core.helper.session_helper import get_execution_role + + smp_options = { + .... # All modelparallel configuration options go here + } + + mpi_options = { + "enabled" : True, # Required + "processes_per_host" : 8 # Required + } + + **# Use the following two lines to tune values of the environment variables for buffer + mpioptions += " -x SMDDP_AG_SCRATCH_BUFFER_SIZE_BYTES=8192" + mpioptions += " -x SMDDP_AG_SORT_BUFFER_SIZE_BYTES=8192"** + + # Retrieve the training image for the desired PyTorch version + training_image = image_uris.retrieve( + framework="pytorch", + region="us-west-2", + version='1.13.1', + py_version='py3', + instance_type='ml.p4d.24xlarge', + image_scope="training" + ) + + smd_mp_model_trainer = ModelTrainer( + training_image=training_image, + source_code=SourceCode( + source_dir="location_to_your_script", + entry_script="your_training_script.py" + ), + role=get_execution_role(), + compute=Compute( + instance_type='ml.p4d.24xlarge', + instance_count=8 + ), + distribution={ + "smdistributed": {"modelparallel": smp_options}, + "mpi": mpi_options + }, + base_job_name="sharded-data-parallel-demo-with-tuning", + ) + + smd_mp_model_trainer.train(input_data_config=[ + InputData(channel_name="training", data_source='s3://my_bucket/my_training_data/') + ])