AWS nova documentation change
Summary
Updated evaluation workflow documentation: changed title to 'Evaluate with Inspect AI SDK', removed standalone provider installation step, renumbered subsequent steps, updated quickstart reference to sample notebooks, and modified related resources section.
Security assessment
The changes are structural/instructional improvements to the evaluation workflow documentation. No security vulnerabilities, configurations, or features are mentioned. The step removal and renumbering suggest process optimization without security implications.
Diff
diff --git a/nova/latest/nova2-userguide/nova-eval-on-sagemaker-inference.md b/nova/latest/nova2-userguide/nova-eval-on-sagemaker-inference.md index b4cbbfee6..de9187fab 100644 --- a//nova/latest/nova2-userguide/nova-eval-on-sagemaker-inference.md +++ b//nova/latest/nova2-userguide/nova-eval-on-sagemaker-inference.md @@ -7 +7 @@ -OverviewSupported modelsPrerequisitesStep 1: Deploy a SageMaker endpointStep 2: Install evaluation dependenciesStep 3: Configure AWS credentialsStep 4: Install the SageMaker providerStep 5: Download evaluation benchmarksStep 6: Run evaluationsStep 7: View resultsManaging endpointsOnboarding custom benchmarksTroubleshootingRelated resources +OverviewSupported modelsPrerequisitesStep 1: Deploy a SageMaker endpointStep 2: Install evaluation dependenciesStep 3: Configure AWS credentialsStep 4: Download evaluation benchmarksStep 5: Run evaluationsStep 6: View resultsManaging endpointsOnboarding custom benchmarksTroubleshootingRelated resources @@ -9 +9 @@ OverviewSupported modelsPrerequisitesStep 1: Deploy a SageMaker endpointStep 2: -# Evaluate Models Hosted on SageMaker Inference +# Evaluate with Inspect AI SDK @@ -15 +15 @@ This guide explains how to evaluate your customized Amazon Nova models deployed -For a hands-on walkthrough, see the [SageMaker Inspect AI quickstart notebook](https://github.com/aws-samples/amazon-nova-samples/tree/main/customization/sagemaker-inference/sagemaker_inspect_quickstart.ipynb). +For hands-on walkthroughs, see the [SageMaker Inspect AI sample notebooks](https://github.com/aws-samples/amazon-nova-samples/tree/main/customization/sagemaker-inspect-ai). @@ -135,5 +135 @@ If running on Amazon EC2 or SageMaker notebooks, the instance's IAM role is used -## Step 4: Install the SageMaker provider - -The SageMaker provider enables Inspect AI to communicate with your SageMaker endpoints. The provider installation process is streamlined in the [quickstart notebook](https://github.com/aws-samples/amazon-nova-samples/tree/main/customization/sagemaker-inference/sagemaker_inspect_quickstart.ipynb). - -## Step 5: Download evaluation benchmarks +## Step 4: Download evaluation benchmarks @@ -159 +155 @@ This repository includes benchmarks such as: -## Step 6: Run evaluations +## Step 5: Run evaluations @@ -194 +190 @@ Setting `--max-connections` too high may overwhelm the endpoint and cause thrott -## Step 7: View results +## Step 6: View results @@ -322 +318 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -API reference +Evaluation with Inspect AI @@ -324 +320 @@ API reference -Abuse detection for Amazon Nova Forge +Inspect AI Container