AWS bedrock documentation change
Summary
Updated prompt dataset requirements for RAG evaluations, added documentation split for retrieve-only vs retrieve-and-generate jobs, and clarified evaluation workflow differences
Security assessment
Changes expand evaluation job types and data requirements without addressing security vulnerabilities or introducing security-specific features. CORS mention is a pre-existing requirement.
Diff
diff --git a/bedrock/latest/userguide/knowledge-base-evaluation-prompt.md b/bedrock/latest/userguide/knowledge-base-evaluation-prompt.md index ba41a5065..8566377ba 100644 --- a/bedrock/latest/userguide/knowledge-base-evaluation-prompt.md +++ b/bedrock/latest/userguide/knowledge-base-evaluation-prompt.md @@ -5 +5 @@ -# Use a prompt dataset for a knowledge base evaluation in Amazon Bedrock +# Create a prompt dataset for a RAG evaluation in Amazon Bedrock @@ -7 +7 @@ -You must provide a prompt dataset for a knowledge base evaluation job. The custom prompt datasets must be stored in Amazon S3, and use the JSON line format and use the `.jsonl` file extension. Each line must be a valid JSON object. There can be up to 1000 prompts in your dataset per evaluation job. The maximum number of turns per conversation is 5. +To evaluate retrieval and generation for an Amazon Bedrock Knowledge Base or for your own Retrieval Augmented Generation (RAG) system, you provide a prompt dataset. When you provide response data from your own RAG system, Amazon Bedrock skips the Knowledge Base invoke step and performs the evaluation job directly on your data. @@ -9 +9 @@ You must provide a prompt dataset for a knowledge base evaluation job. The custo -Use the sections below to learn more about key value pairs that are required based on the type of evaluation job you select. +Prompt datasets must be stored in Amazon S3 and use the JSON line format and `.jsonl` file extension. Each line must be a valid JSON object. There can be up to 1000 prompts in your dataset per evaluation job. For reteieve-and-generate evaluation jobs, the maximum number of turns for each conversation is 5. For retrieve-only evaluations, you can specify only a single turn. @@ -12,0 +13,11 @@ For jobs created using the console you must update the Cross Origin Resource Sha +See the following topics to learn more about key value pairs that are required based on the type of evaluation job you select. + +###### Topics + + * [Create a prompt dataset for retrieve-only RAG evaluation jobs](./knowledge-base-evaluation-prompt-retrieve.html) + + * [Creating a prompt dataset for retrieve-and-generate RAG evaluation jobs](./knowledge-base-evaluation-prompt-retrieve-generate.html) + + + + @@ -21 +32 @@ Prerequisites -Creating a prompt dataset for Retrieve only evaluation jobs +Prompt datasets for retrieve-only evaluation jobs