AWS sagemaker documentation change
Summary
Removed TensorFlow support documentation and legacy SDK examples, clarified PyTorch-only support for SMDDP
Security assessment
Change removes discontinued TensorFlow support documentation and cleans up outdated SDK references. No security vulnerabilities or security features are mentioned in the changes.
Diff
diff --git a/sagemaker/latest/dg/data-parallel-framework-estimator.md b/sagemaker/latest/dg/data-parallel-framework-estimator.md index 7dd85f7fa..5fcaf5c06 100644 --- a//sagemaker/latest/dg/data-parallel-framework-estimator.md +++ b//sagemaker/latest/dg/data-parallel-framework-estimator.md @@ -9 +9 @@ -You can launch distributed training by adding the `distribution` argument to the SageMaker AI framework estimators, [`PyTorch`](https://sagemaker.readthedocs.io/en/stable/api/sagemaker_train.html) or [`TensorFlow`](https://sagemaker.readthedocs.io/en/stable/api/sagemaker_train.html). For more details, choose one of the frameworks supported by the SageMaker AI distributed data parallelism (SMDDP) library from the following selections. +You can launch distributed training by adding the `distribution` argument to the SageMaker AI [`PyTorch`](https://sagemaker.readthedocs.io/en/stable/api/sagemaker_train.html) framework estimator. The SageMaker AI distributed data parallelism (SMDDP) library supports PyTorch distributed training. @@ -11 +11 @@ You can launch distributed training by adding the `distribution` argument to the -SageMaker Python SDK v3 +###### Note @@ -12,0 +13 @@ SageMaker Python SDK v3 +SMDDP discontinued TensorFlow support after v2.11.0. For distributed training with TensorFlow, use alternative distribution strategies. @@ -136,81 +136,0 @@ For more information about specifying the source directory to place the `require -SageMaker Python SDK v2 (Legacy) - - -**PyTorch** - -The following launcher options are available for launching PyTorch distributed training. - - * `pytorchddp` – This option runs `mpirun` and sets up environment variables needed for running PyTorch distributed training on SageMaker AI. To use this option, pass the following dictionary to the `distribution` parameter. - - { "pytorchddp": { "enabled": True } } - - * `torch_distributed` – This option runs `torchrun` and sets up environment variables needed for running PyTorch distributed training on SageMaker AI. To use this option, pass the following dictionary to the `distribution` parameter. - - { "torch_distributed": { "enabled": True } } - - * `smdistributed` – This option also runs `mpirun` but with `smddprun` that sets up environment variables needed for running PyTorch distributed training on SageMaker AI. - - { "smdistributed": { "dataparallel": { "enabled": True } } } - - - - -The following code sample shows the basic structure of a PyTorch estimator with distributed training options. - - - from sagemaker.pytorch import PyTorch - - pt_estimator = PyTorch( - base_job_name="training_job_name_prefix", - source_dir="subdirectory-to-your-code", - entry_point="adapted-training-script.py", - role="SageMakerRole", - py_version="py310", - framework_version="2.0.1", - - # For running a multi-node distributed training job, specify a value greater than 1 - # Example: 2,3,4,..8 - instance_count=2, - - # Instance types supported by the SageMaker AI data parallel library: - # ml.p4d.24xlarge, ml.p4de.24xlarge - instance_type="ml.p4d.24xlarge", - - # Activate distributed training with SMDDP - distribution={ "pytorchddp": { "enabled": True } } # mpirun, activates SMDDP AllReduce OR AllGather - # distribution={ "torch_distributed": { "enabled": True } } # torchrun, activates SMDDP AllGather - # distribution={ "smdistributed": { "dataparallel": { "enabled": True } } } # mpirun, activates SMDDP AllReduce OR AllGather - ) - - pt_estimator.fit("s3://bucket/path/to/training/data") - -**TensorFlow** - -###### Important - -The SMDDP library discontinued support for TensorFlow and is no longer available in DLCs for TensorFlow later than v2.11.0. To find previous TensorFlow DLCs with the SMDDP library installed, see [TensorFlow (deprecated)](./distributed-data-parallel-support.html#distributed-data-parallel-supported-frameworks-tensorflow). - - - from sagemaker.tensorflow import TensorFlow - - tf_estimator = TensorFlow( - base_job_name = "training_job_name_prefix", - entry_point="adapted-training-script.py", - role="SageMakerRole", - framework_version="2.11.0", - py_version="py38", - - # For running a multi-node distributed training job, specify a value greater than 1 - # Example: 2,3,4,..8 - instance_count=2, - - # Instance types supported by the SageMaker AI data parallel library: - # ml.p4d.24xlarge, ml.p3dn.24xlarge, and ml.p3.16xlarge - instance_type="ml.p3.16xlarge", - - # Training using the SageMaker AI data parallel distributed training strategy - distribution={ "smdistributed": { "dataparallel": { "enabled": True } } } - ) - - tf_estimator.fit("s3://bucket/path/to/training/data") -