AWS sagemaker documentation change
Summary
Updated documentation to replace 'estimator' with 'ModelTrainer' terminology throughout the file. Added new code examples for both SageMaker Python SDK v3 and legacy v2 implementations. Included detailed profiler output URI retrieval method for v3 SDK.
Security assessment
Changes involve terminology updates (estimator → ModelTrainer) and SDK version-specific implementations. No security vulnerabilities, patches, or security-related configurations are mentioned. Changes focus on profiler usage patterns and SDK migration without security implications.
Diff
diff --git a/sagemaker/latest/dg/profiler-prepare.md b/sagemaker/latest/dg/profiler-prepare.md index 962bb4d57..92f44094c 100644 --- a//sagemaker/latest/dg/profiler-prepare.md +++ b//sagemaker/latest/dg/profiler-prepare.md @@ -7 +7 @@ -Step 1: Adapt your training script using the SageMaker Profiler Python modulesStep 2: Create a SageMaker AI framework estimator and activate SageMaker Profiler(Optional) Install the SageMaker Profiler Python package +Step 1: Adapt your training script using the SageMaker Profiler Python modulesStep 2: Create a SageMaker AI ModelTrainer and activate SageMaker Profiler(Optional) Install the SageMaker Profiler Python package @@ -17 +17 @@ Setting up to running a training job with the SageMaker Profiler consists of two - * Step 2: Create a SageMaker AI framework estimator and activate SageMaker Profiler + * Step 2: Create a SageMaker AI ModelTrainer and activate SageMaker Profiler @@ -28 +28 @@ To start capturing kernel runs on GPUs while the training job is running, modify -Note that the annotators extract operations from GPUs. For profiling operations in CPUs, you don’t need to add any additional annotations. CPU profiling is also activated when you specify the profiling configuration, which you’ll practice in Step 2: Create a SageMaker AI framework estimator and activate SageMaker Profiler. +Note that the annotators extract operations from GPUs. For profiling operations in CPUs, you don’t need to add any additional annotations. CPU profiling is also activated when you specify the profiling configuration, which you’ll practice in Step 2: Create a SageMaker AI ModelTrainer and activate SageMaker Profiler. @@ -50,0 +51,3 @@ You can wrap full functions with the `smprof.annotate()` context manager. This w +SageMaker Python SDK v3 + + @@ -84,0 +88,46 @@ You can wrap full functions with the `smprof.annotate()` context manager. This w +SageMaker Python SDK v2 (Legacy) + + + + import smprof + + SMProf = smprof.SMProfiler.instance() + config = smprof.Config() + config.profiler = { + "EnableCuda": "1", + } + SMProf.configure(config) + SMProf.start_profiling() + + for epoch in range(args.epochs): + if world_size > 1: + sampler.set_epoch(epoch) + tstart = time.perf_counter() + for i, data in enumerate(trainloader, 0): + step_annotator = smprof.annotation_begin("step_" + str(i)) + + inputs, labels = data + inputs = inputs.to("cuda", non_blocking=True) + labels = labels.to("cuda", non_blocking=True) + optimizer.zero_grad() + + forward_annotator = smprof.annotation_begin("Forward") + outputs = net(inputs) + smprof.annotation_end(forward_annotator) + + loss_annotator = smprof.annotation_begin("Loss") + loss = criterion(outputs, labels) + smprof.annotation_end(loss_annotator) + + backward_annotator = smprof.annotation_begin("Backward") + loss.backward() + smprof.annotation_end(backward_annotator) + + optimizer_annotator = smprof.annotation_begin("Optimizer") + optimizer.step() + smprof.annotation_end(optimizer_annotator) + + smprof.annotation_end(step_annotator) + + SMProf.stop_profiling() + @@ -88,0 +138,3 @@ You can also define annotations to profile specific code lines. You can set the +SageMaker Python SDK v3 + + @@ -131,0 +184,37 @@ You can also define annotations to profile specific code lines. You can set the +SageMaker Python SDK v2 (Legacy) + + + + import smprof + + SMProf = smprof.SMProfiler.instance() + config = smprof.Config() + config.profiler = { + "EnableCuda": "1", + } + SMProf.configure(config) + SMProf.start_profiling() + + for epoch in range(args.epochs): + if world_size > 1: + sampler.set_epoch(epoch) + tstart = time.perf_counter() + for i, data in enumerate(trainloader, 0): + with smprof.annotate("step_"+str(i)): + inputs, labels = data + inputs = inputs.to("cuda", non_blocking=True) + labels = labels.to("cuda", non_blocking=True) + + optimizer.zero_grad() + + with smprof.annotate("Forward"): + outputs = net(inputs) + with smprof.annotate("Loss"): + loss = criterion(outputs, labels) + with smprof.annotate("Backward"): + loss.backward() + with smprof.annotate("Optimizer"): + optimizer.step() + + SMProf.stop_profiling() + @@ -134 +223 @@ After annotating and setting up the profiler initiation modules, save the script -## Step 2: Create a SageMaker AI framework estimator and activate SageMaker Profiler +## Step 2: Create a SageMaker AI ModelTrainer and activate SageMaker Profiler @@ -136 +225 @@ After annotating and setting up the profiler initiation modules, save the script -The following procedure shows how to prepare a SageMaker AI framework estimator for training using the SageMaker Python SDK. +The following procedure shows how to prepare a SageMaker AI ModelTrainer for training using the SageMaker Python SDK. @@ -151 +240,4 @@ The following is the description of the `Profiler` module and its argument. - 2. Create a SageMaker AI framework estimator with the `profiler_config` object created in the previous step. The following code shows an example of creating a PyTorch estimator. If you want to create a TensorFlow estimator, import `sagemaker.tensorflow.TensorFlow` instead, and specify one of the [TensorFlow versions](./profiler-support.html#profiler-support-frameworks-tensorflow) supported by SageMaker Profiler. For more information about supported frameworks and instance types, see [SageMaker AI framework images pre-installed with SageMaker Profiler](./profiler-support.html#profiler-support-frameworks). + 2. Create a SageMaker AI ModelTrainer with the `profiler_config` object created in the previous step. The following code shows an example of creating a PyTorch ModelTrainer. If you want to create a TensorFlow ModelTrainer, import `sagemaker.tensorflow.TensorFlow` instead, and specify one of the [TensorFlow versions](./profiler-support.html#profiler-support-frameworks-tensorflow) supported by SageMaker Profiler. For more information about supported frameworks and instance types, see [SageMaker AI framework images pre-installed with SageMaker Profiler](./profiler-support.html#profiler-support-frameworks). + +SageMaker Python SDK v3 + @@ -153,2 +245,9 @@ The following is the description of the `Profiler` module and its argument. - import sagemaker - from sagemaker.pytorch import PyTorch + from sagemaker.train import ModelTrainer + from sagemaker.train.configs import SourceCode + from sagemaker.core.helper.session_helper import get_execution_role + from sagemaker.core import image_uris + + training_image = image_uris.retrieve( + framework="pytorch", region=region, version="2.0.0", + image_scope="training", instance_type=ml.p4d.24xlarge + ) @@ -156,5 +255,4 @@ The following is the description of the `Profiler` module and its argument. - estimator = PyTorch( - framework_version="2.0.0", - role=sagemaker.get_execution_role(), - entry_point="train_with_profiler_demo.py", # your training job entry point - source_dir=source_dir, # source directory for your training script + model_trainer = ModelTrainer( + training_image=training_image, + role=get_execution_role(), + source_code=SourceCode(entry_script="train_with_profiler_demo.py", source_dir=source_dir), # your training job entry point @@ -169 +267 @@ The following is the description of the `Profiler` module and its argument. - 3. Start the training job by running the `fit` method. With `wait=False`, you can silence the training job logs and let it run in the background. +SageMaker Python SDK v2 (Legacy) @@ -171 +269,37 @@ The following is the description of the `Profiler` module and its argument. - estimator.fit(wait=False) + + import smprof + + SMProf = smprof.SMProfiler.instance() + config = smprof.Config() + config.profiler = { + "EnableCuda": "1", + } + SMProf.configure(config) + SMProf.start_profiling() + + for epoch in range(args.epochs): + if world_size > 1: + sampler.set_epoch(epoch) + tstart = time.perf_counter() + for i, data in enumerate(trainloader, 0): + with smprof.annotate("step_"+str(i)): + inputs, labels = data + inputs = inputs.to("cuda", non_blocking=True) + labels = labels.to("cuda", non_blocking=True) + + optimizer.zero_grad() + + with smprof.annotate("Forward"): + outputs = net(inputs) + with smprof.annotate("Loss"): + loss = criterion(outputs, labels) + with smprof.annotate("Backward"): + loss.backward() + with smprof.annotate("Optimizer"): + optimizer.step() + + SMProf.stop_profiling() + + 3. Start the training job by running the `train` method. With `wait=False`, you can silence the training job logs and let it run in the background. + + model_trainer.train(wait=False) @@ -179,0 +314,30 @@ If you want to directly access the profile data saved in the Amazon S3 bucket, u +SageMaker Python SDK v3 + + + + import os + # This is an ad-hoc function to get the S3 URI + # to where the profile output data is saved + def get_detailed_profiler_output_uri(model_trainer): + config_name = None + for processing in model_trainer.profiler_rule_configs: + params = processing.get("RuleParameters", dict()) + rule = config_name = params.get("rule_to_invoke", "") + if rule == "DetailedProfilerProcessing": + config_name = processing.get("RuleConfigurationName") + break + return os.path.join( + model_trainer.output_path,