AWS bedrock documentation change
Summary
Updated documentation for creating reinforcement fine-tuning jobs for Amazon Nova models, including restructured prerequisites, simplified console steps, expanded monitoring metrics, and updated references.
Security assessment
The changes primarily restructure documentation and update procedures without addressing specific vulnerabilities. The IAM role reference update points to security documentation but doesn't introduce new security content. No evidence of patching vulnerabilities or addressing security incidents is present.
Diff
diff --git a/bedrock/latest/userguide/rft-submit-job.md b/bedrock/latest/userguide/rft-submit-job.md index fb2cdd4b0..7602a01db 100644 --- a//bedrock/latest/userguide/rft-submit-job.md +++ b//bedrock/latest/userguide/rft-submit-job.md @@ -5,20 +5 @@ -PrerequisitesCreate your RFT jobRFT job workflowSet up inference - -# Create a reinforcement fine-tuning job - -You can create a reinforcement fine-tuning job using the Amazon Bedrock console or API. The RFT job can take several hours depending on the size of your training data, number of epochs, and complexity of your reward functions. - -###### Topics - - * Prerequisites - - * Create your RFT job - - * RFT job workflow - - * Set up inference - - * [Monitor your RFT training job](./rft-monitor-job.html) - - * [Evaluate your RFT model](./rft-evaluate-model.html) - +PrerequisitesCreate your RFT jobMonitor your RFT training jobSet up inference @@ -25,0 +7 @@ You can create a reinforcement fine-tuning job using the Amazon Bedrock console +# Create and manage fine-tuning jobs for Amazon Nova models @@ -26,0 +9 @@ You can create a reinforcement fine-tuning job using the Amazon Bedrock console +You can create a reinforcement fine-tuning (RFT) job using the Amazon Bedrock console or API. The RFT job can take a few hours depending on the size of your training data, number of epochs, and complexity of your reward functions. @@ -30 +13 @@ You can create a reinforcement fine-tuning job using the Amazon Bedrock console - * Create an IAM service role to access the Amazon S3 bucket where you want to store your RFT training data and output artifacts. You can create this role automatically using the AWS Management Console or manually. For RFT-specific permissions, see [Reinforcement fine-tuning access and security](./rft-access-security.html). + * Create an IAM service role with the required permissions. For comprehensive security and permissions information including RFT-specific permissions, see [Access and security for Amazon Nova models](./rft-access-security.html). @@ -46,5 +29 @@ To submit an RFT job in the console, carry out the following steps: - 1. Sign in to the AWS Management Console and open the Amazon Bedrock console at [ https://console.aws.amazon.com/bedrock](https://console.aws.amazon.com/bedrock). - - 2. From the left navigation pane, choose **Custom models** under **Tune**. - - 3. In the Models table, choose **Create**. Then, choose **Create reinforcement fine-tuning job**. + 1. Open the Amazon Bedrock console and navigate to **Custom models** under **Tune**. @@ -52 +31 @@ To submit an RFT job in the console, carry out the following steps: - 4. In the **Model details** section, choose **Amazon Nova 2 Lite** as your base model. + 2. Choose **Create** , then **Create reinforcement fine-tuning job**. @@ -54 +33 @@ To submit an RFT job in the console, carry out the following steps: - 5. In the **Customization details** section, enter the customization name. + 3. In the **Model details** section, choose **Amazon Nova 2 Lite** as your base model. @@ -56 +35 @@ To submit an RFT job in the console, carry out the following steps: - 6. In the **Training data** section, choose your data source: + 4. In the **Customization details** section, enter the customization name. @@ -58,3 +37 @@ To submit an RFT job in the console, carry out the following steps: - * **Use stored invocation logs** \- Select from your available invocation logs stored in Amazon S3 - - * **Upload new dataset** \- Select the Amazon S3 location of your training dataset file or upload a file directly from your device + 5. In the **Training data** section, choose your data source. Either select from your available invocation logs stored in Amazon S3, or select the Amazon S3 location of your training dataset file, or upload a file directly from your device. @@ -66 +43 @@ Your training dataset should be in the OpenAI Chat Completions data format. If y - 7. In the **Reward function** section, set up your reward mechanism: + 6. In the **Reward function** section, set up your reward mechanism: @@ -76,7 +53 @@ The console's **Model as judge** option automatically converts your configuratio -For more information, see [Setting up reward functions](./reward-functions.html). - - 8. (Optional) In the **Hyperparameters** section, adjust training parameters or use default values. - - 9. In the **Output data** section, enter the Amazon S3 location where Bedrock should save job outputs. - - 10. In the **Role configuration** section, select: +For more information, see [Setting up reward functions for Amazon Nova models](./reward-functions.html). @@ -84 +55 @@ For more information, see [Setting up reward functions](./reward-functions.html) - * **Choose an existing role** \- Select from dropdown list + 7. (Optional) In the **Hyperparameters** section, adjust training parameters or use default values. @@ -86 +57 @@ For more information, see [Setting up reward functions](./reward-functions.html) - * **Create a role** \- Enter a name for the service role + 8. In the **Output data** section, enter the Amazon S3 location where Amazon Bedrock should save job outputs. @@ -88 +59 @@ For more information, see [Setting up reward functions](./reward-functions.html) - 11. (Optional) In the **Additional configuration** section, configure: + 9. In the **Role configuration** section, either choose an existing role from the dropdown list or enter a name for the service role to create. @@ -90 +61 @@ For more information, see [Setting up reward functions](./reward-functions.html) - * Validation data by pointing to an Amazon S3 bucket + 10. (Optional) In the **Additional configuration** section, configure the validation data by pointing to an Amazon S3 bucket, KMS encryption settings, and job and model tags. @@ -92,5 +63 @@ For more information, see [Setting up reward functions](./reward-functions.html) - * KMS encryption settings - - * Job and model tags - - 12. Choose **Create reinforcement fine-tuning job** to begin the job. + 11. Choose **Create reinforcement fine-tuning job** to begin the job. @@ -104,21 +71 @@ API -Send a CreateModelCustomizationJob request with `customizationType` set to `REINFORCEMENT_FINE_TUNING`. You must provide the following fields: - -**Required fields:** - - * `roleArn` \- ARN of the service role with RFT permissions - - * `baseModelIdentifier` \- Model ID or ARN of the foundation model to customize - - * `customModelName` \- Name for the newly customized model - - * `jobName` \- Name for the training job - - * `customizationType` \- Set to `REINFORCEMENT_FINE_TUNING` - - * `trainingDataConfig` \- Amazon S3 URI of training dataset or invocation log configuration - - * `outputDataConfig` \- Amazon S3 URI to write output data - - * `rftConfig` \- Reward function configuration (RLVR or RLAIF) and hyper paramerters configuration - - +Send a CreateModelCustomizationJob request with `customizationType` set to `REINFORCEMENT_FINE_TUNING`. @@ -125,0 +73 @@ Send a CreateModelCustomizationJob request with `customizationType` set to `REIN +**Required fields:** `roleArn`, `baseModelIdentifier`, `customModelName`, `jobName`, `trainingDataConfig`, `outputDataConfig`, `rftConfig` @@ -214 +162 @@ Send a CreateModelCustomizationJob request with `customizationType` set to `REIN -## RFT job workflow +## Monitor your RFT training job @@ -216 +164 @@ Send a CreateModelCustomizationJob request with `customizationType` set to `REIN -The RFT job follows this automated workflow: +Amazon Bedrock provides real-time monitoring with visual graphs and metrics during RFT training. These metrics help you understand whether the model converges properly and if the reward function effectively guides the learning process. @@ -218 +166 @@ The RFT job follows this automated workflow: - 1. **Response Generation** \- The actor model generates responses from training prompts +### Job status tracking @@ -220 +168 @@ The RFT job follows this automated workflow: - 2. **Reward Computation** \- Reward functions evaluate prompt-response pairs +You can monitor your RFT job status through the validation and training phases in the Amazon Bedrock console. @@ -222 +170 @@ The RFT job follows this automated workflow: - 3. **Actor Model Training** \- Model learns from scored pairs using GRPO +**Completion indicators:** @@ -223,0 +172 @@ The RFT job follows this automated workflow: + * Job status changes to **Completed** when training completes successfully @@ -224,0 +174,3 @@ The RFT job follows this automated workflow: + * Custom model ARN becomes available for deployment + + * Training metrics reach convergence thresholds @@ -227 +178,0 @@ The RFT job follows this automated workflow: -During training, you can monitor progress using real-time graphs with training and validation metrics such as loss, rewards, reward margin, and accuracy. Once successful, an RFT model is created with a custom model ARN. @@ -229 +179,0 @@ During training, you can monitor progress using real-time graphs with training a -## Set up inference @@ -231 +181,50 @@ During training, you can monitor progress using real-time graphs with training a -After job completion, you can deploy the resulting RFT model with one click for on-demand inference. You can also use Provisioned Throughput for mission-critical workloads that require consistent performance. Once inference is set up, use **Test in Playground** to interactively evaluate and compare responses side-by-side with the base model. +### Real-time training metrics + +Amazon Bedrock provides real-time monitoring during RFT training with visual graphs displaying training and validation metrics. + +#### Core training metrics + + * **Training loss** \- Measures how well the model is learning from the training data + + * **Training reward statistics** \- Shows reward scores assigned by your reward functions + + * **Reward margin** \- Measures the difference between good and bad response rewards + + * **Accuracy on training and validation sets** \- Shows model performance on both the training and held-out data + + + + +**Detailed metric categories** + + * **Reward metrics** – `critic/rewards/mean`, `critic/rewards/max`, `critic/rewards/min` (reward distribution), and `val-score/rewards/mean@1` (validation rewards) + + * **Model behavior** – `actor/entropy` (policy variation; higher equals more exploratory) + + * **Training health** – `actor/pg_loss` (policy gradient loss), `actor/pg_clipfrac` (frequency of clipped updates), and `actor/grad_norm` (gradient magnitude) + + * **Response characteristics** – `prompt_length/mean`, `prompt_length/max`, `prompt_length/min` (input token statistics), `response_length/mean`, `response_length/max`, `response_length/min` (output token statistics), and `response/aborted_ratio` (incomplete generation rate; 0 equals all completed) + + * **Performance** – `perf/throughput` (training throughput), `perf/time_per_step` (time per training step), and `timing_per_token_ms/*` (per-token processing times) + + * **Resource usage** – `perf/max_memory_allocated_gb`, `perf/max_memory_reserved_gb` (GPU memory), and `perf/cpu_memory_used_gb` (CPU memory) + + + + +#### Training progress visualization + +The console displays interactive graphs that update in real-time as your RFT job progresses. These visualizations can help you: + + * Track convergence toward optimal performance + + * Identify potential training issues early + + * Determine optimal stopping points + + * Compare performance across different epochs + + + + +## Set up inference @@ -233 +232 @@ After job completion, you can deploy the resulting RFT model with one click for -For monitoring your RFT job progress, see [Monitor your RFT training job](./rft-monitor-job.html). +After job completion, deploy the RFT model for on-demand inference or use Provisioned Throughput for consistent performance. For setting up inference, see [Set up inference for a custom model](./model-customization-use.html). @@ -235 +234 @@ For monitoring your RFT job progress, see [Monitor your RFT training job](./rft- -For evaluating your completed RFT model, see [Evaluate your RFT model](./rft-evaluate-model.html). +Use **Test in Playground** to evaluate and compare responses with the base model. For evaluating your completed RFT model, see [Evaluate your RFT model](./rft-evaluate-model.html). @@ -245 +244 @@ Setting up reward functions -Monitor your RFT training job +Fine-tune open-weight models using OpenAI APIs