AWS bedrock documentation change
Summary
Updated documentation headers and terminology for clarity, changed 'Perform' to 'Prepare' in section titles, fixed grammatical errors, updated link references, and improved consistency in dataset preparation instructions
Security assessment
Changes focus on terminology consistency and documentation clarity. The CORS configuration mention refers to existing security requirements but doesn't introduce new security content. No evidence of addressing vulnerabilities or security incidents.
Diff
diff --git a/bedrock/latest/userguide/model-evaluation-prompt-datasets-judge.md b/bedrock/latest/userguide/model-evaluation-prompt-datasets-judge.md index 3888a6b84..8dd5b570b 100644 --- a//bedrock/latest/userguide/model-evaluation-prompt-datasets-judge.md +++ b//bedrock/latest/userguide/model-evaluation-prompt-datasets-judge.md @@ -5 +5 @@ -Perform an evaluation job where Amazon Bedrock invokes models for youPerform an evaluation job using your own inference response data +Prepare a dataset for an evaluation job where Amazon Bedrock invokes models for youPrepare a dataset for an evaluation job using your own inference response data @@ -7 +7 @@ Perform an evaluation job where Amazon Bedrock invokes models for youPerform an -# Create a custom prompt dataset for a model evaluation job that uses a model as judge +# Create a prompt dataset for a model evaluation job that uses a model as judge @@ -11 +11 @@ To create a model evaluation job that uses a model as judge you must specify a p -If you want to evaluate non-Amazon Bedrock models using responses that you've already generated, include them in the prompt dataset as described in Perform an evaluation job using your own inference response data. When you provide your own inference response data, Amazon Bedrock skips the model-invoke step and performs the evaluation job with the data you provide. +If you want to evaluate non-Amazon Bedrock models using responses that you've already generated, include them in the prompt dataset as described in Prepare a dataset for an evaluation job using your own inference response data. When you provide your own inference response data, Amazon Bedrock skips the model-invoke step and performs the evaluation job with the data you provide. @@ -15 +15 @@ Custom prompt datasets must be stored in Amazon S3, and use the JSON line format -For job created using the console you must update the Cross Origin Resource Sharing (CORS) configuration on the S3 bucket. To learn more about the required CORS permissions, see [Required Cross Origin Resource Sharing (CORS) permissions on S3 buckets](./model-evaluation-security-cors.html). +For jobs created using the console you must update the Cross Origin Resource Sharing (CORS) configuration on the S3 bucket. To learn more about the required CORS permissions, see [Required Cross Origin Resource Sharing (CORS) permissions on S3 buckets](./model-evaluation-security-cors.html). @@ -17 +17 @@ For job created using the console you must update the Cross Origin Resource Shar -## Perform an evaluation job where Amazon Bedrock invokes models for you +## Prepare a dataset for an evaluation job where Amazon Bedrock invokes models for you @@ -19 +19 @@ For job created using the console you must update the Cross Origin Resource Shar -To run an evaluation job where Amazon Bedrock invokes the models for you, provide a prompt dataset containing the following key-value pairs: +To run an evaluation job where Amazon Bedrock invokes the models for you, create a prompt dataset containing the following key-value pairs: @@ -32 +32 @@ To run an evaluation job where Amazon Bedrock invokes the models for you, provid -If you choose to supply a ground truth response (`referenceResponse)`, Amazon Bedrock will use this parameter when calculating the **Completeness** (`Builtin.Completeness`) and **Correctness** (`Builtin.Correctness`) metrics. You can also use these metrics without supplying a ground truth response. To see the judge prompts for both of these scenarios, refer to the section for your chosen judge model in [Evaluator prompts based used in judge-based model evaluation job](./model-evaluation-type-judge-prompt.html). +If you choose to supply a ground truth response (`referenceResponse)`, Amazon Bedrock will use this parameter when calculating the **Completeness** (`Builtin.Completeness`) and **Correctness** (`Builtin.Correctness`) metrics. You can also use these metrics without supplying a ground truth response. To see the judge prompts for both of these scenarios, refer to the section for your chosen judge model in [Built-in metric evaluator prompts for model-as-a-judge evaluation jobs](./model-evaluation-type-judge-prompt.html). @@ -53 +53 @@ The following example is a single entry expanded for clarity. In your actual pro -## Perform an evaluation job using your own inference response data +## Prepare a dataset for an evaluation job using your own inference response data @@ -55 +55 @@ The following example is a single entry expanded for clarity. In your actual pro -To run an evaluation job using responses you've already generated, you provide a prompt dataset containing the following key-value pairs: +To run an evaluation job using responses you've already generated, create a prompt dataset containing the following key-value pairs: @@ -74 +74 @@ To run an evaluation job using responses you've already generated, you provide a -If you choose to supply a ground truth response (`referenceResponse)`, Amazon Bedrock will use this parameter when calculating the **Completeness** (`Builtin.Completeness`) and **Correctness** (`Builtin.Correctness`) metrics. You can also use these metrics without supplying a ground truth response. To see the judge prompts for both of these scenarios, refer to the section for your chosen judge model in [Evaluator prompts based used in judge-based model evaluation job](./model-evaluation-type-judge-prompt.html). +If you choose to supply a ground truth response (`referenceResponse)`, Amazon Bedrock will use this parameter when calculating the **Completeness** (`Builtin.Completeness`) and **Correctness** (`Builtin.Correctness`) metrics. You can also use these metrics without supplying a ground truth response. To see the judge prompts for both of these scenarios, refer to the section for your chosen judge model in [Built-in metric evaluator prompts for model-as-a-judge evaluation jobs](./model-evaluation-type-judge-prompt.html). @@ -107 +107 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Creating job +LLM as a judge model evaluation jobs @@ -109 +109 @@ Creating job -Evaluator prompts +Evaluation metrics