AWS bedrock documentation change
Summary
Added documentation for a new 'Fidelity report' feature that measures policy accuracy against source documents, including coverage/accuracy scores and grounding details.
Security assessment
The change introduces documentation about policy validation accuracy reporting but contains no evidence of addressing security vulnerabilities, weaknesses, or incidents. It focuses on general policy accuracy measurement without security-specific context.
Diff
diff --git a/bedrock/latest/userguide/automated-reasoning-checks-concepts.md b/bedrock/latest/userguide/automated-reasoning-checks-concepts.md index dcc6b50ea..9c4166480 100644 --- a//bedrock/latest/userguide/automated-reasoning-checks-concepts.md +++ b//bedrock/latest/userguide/automated-reasoning-checks-concepts.md @@ -5 +5 @@ -PoliciesRulesVariablesCustom types (enums)Translation: from natural language to formal logicFindings and validation resultsConfidence thresholds +PoliciesFidelity reportRulesVariablesCustom types (enums)Translation: from natural language to formal logicFindings and validation resultsConfidence thresholds @@ -40,0 +41,26 @@ For step-by-step instructions on creating a policy, see [Create your Automated R +## Fidelity report + +A _fidelity report_ measures how accurately an extracted policy represents the source documents it was generated from. The report is automatically generated when you create a policy from a source document, and provides two key scores along with detailed grounding information that links every rule and variable back to specific statements in your source content. + +The fidelity report is designed to help non-technical subject matter experts explore and validate a policy without needing to understand formal logic. In the console, the **Source Document** tab displays the fidelity report as a table of numbered atomic statements extracted from your document, showing which rules and variables each statement grounds. You can filter by specific rules or variables and search for content within the statements. + +The fidelity report includes two scores, each ranging from 0.0 to 1.0: + + * **Coverage score** — Indicates how well the policy covers the statements in the source documents. A higher score means more of the source content is represented in the policy. + + * **Accuracy score** — Indicates how faithfully the policy rules represent the source material. A higher score means the extracted rules more closely match the intent of the original document. + + + + +Beyond the aggregate scores, the fidelity report provides detailed grounding for each rule and variable in the policy: + + * **Rule reports** — For each rule, the report identifies the specific statements from the source documents that support it (grounding statements), explains how those statements justify the rule (grounding justifications), and provides an individual accuracy score with a justification. + + * **Variable reports** — For each variable, the report identifies the source statements that support the variable definition, explains the justification, and provides an individual accuracy score. + + * **Document sources** — The source documents are broken down into atomic statements — individual, indivisible facts extracted from the text. The document content is annotated with line numbers so you can trace each rule and variable back to the exact location in the original document. + + + +