AWS bedrock-agentcore documentation change
Summary
Updated documentation formatting and grammar: replaced straight apostrophes with curly apostrophes, fixed markdown table formatting, adjusted code examples for better readability, and added example headers.
Security assessment
The changes are purely cosmetic and editorial, focusing on improving the readability and formatting of the documentation. There is no mention of security vulnerabilities, patches, or new security features.
Diff
diff --git a/bedrock-agentcore/latest/devguide/ground-truth-evaluations.md b/bedrock-agentcore/latest/devguide/ground-truth-evaluations.md index fb42c1a15..d805b3db8 100644 --- a//bedrock-agentcore/latest/devguide/ground-truth-evaluations.md +++ b//bedrock-agentcore/latest/devguide/ground-truth-evaluations.md @@ -11 +11 @@ Ground truth is the known correct answer or expected behavior for a given input -With ground truth evaluations, you provide reference inputs alongside your session spans when calling the Evaluate API. The service uses these reference inputs to score your agent's actual behavior against the expected behavior. Evaluators that don't use a particular ground truth field ignore it and report which fields were not used in the response. +With ground truth evaluations, you provide reference inputs alongside your session spans when calling the Evaluate API. The service uses these reference inputs to score your agent’s actual behavior against the expected behavior. Evaluators that don’t use a particular ground truth field ignore it and report which fields were not used in the response. @@ -38 +38 @@ The following table shows which built-in evaluators support ground truth and whi -Evaluators that support ground truth Evaluator | Level | Ground truth field | Description +Evaluator | Level | Ground truth field | Description @@ -40,2 +40,2 @@ Evaluators that support ground truth Evaluator | Level | Ground truth field | De -`Builtin.Correctness` | Trace | `expectedResponse` | Measures how accurately the agent's response matches the expected answer. Uses LLM-as-a-Judge scoring. -`Builtin.GoalSuccessRate` | Session | `assertions` | Validates whether the agent's behavior satisfies natural language assertions across the entire session. Uses LLM-as-a-Judge scoring. +`Builtin.Correctness` | Trace | `expectedResponse` | Measures how accurately the agent’s response matches the expected answer. Uses LLM-as-a-Judge scoring. +`Builtin.GoalSuccessRate` | Session | `assertions` | Validates whether the agent’s behavior satisfies natural language assertions across the entire session. Uses LLM-as-a-Judge scoring. @@ -52 +52 @@ The following table describes the ground truth fields. -Ground truth fields Field | Type | Scope | Description +Field | Type | Scope | Description @@ -55 +55 @@ Ground truth fields Field | Type | Scope | Description -`assertions` | List of strings | Session | Natural language statements that should be true about the agent's behavior across the session. +`assertions` | List of strings | Session | Natural language statements that should be true about the agent’s behavior across the session. @@ -62 +62 @@ Ground truth fields Field | Type | Scope | Description - * You don't need to provide `expectedResponse` for every trace. Traces without ground truth are evaluated using the ground truth-free variant of the evaluator. + * You don’t need to provide `expectedResponse` for every trace. Traces without ground truth are evaluated using the ground truth-free variant of the evaluator. @@ -94 +94 @@ The examples assume a two-turn session: - 2. **Turn 2:** "What's the weather?" — agent uses the `weather` tool and responds with the current weather. + 2. **Turn 2:** "What’s the weather?" — agent uses the `weather` tool and responds with the current weather. @@ -112 +112 @@ The following constants are used throughout the examples on this page. Replace t -`Builtin.Correctness` is a trace-level evaluator that measures how accurately the agent's response matches an expected answer. When you provide `expectedResponse`, the evaluator compares the agent's actual response against your ground truth using LLM-as-a-Judge scoring. +`Builtin.Correctness` is a trace-level evaluator that measures how accurately the agent’s response matches an expected answer. When you provide `expectedResponse` , the evaluator compares the agent’s actual response against your ground truth using LLM-as-a-Judge scoring. @@ -114 +114 @@ The following constants are used throughout the examples on this page. Replace t -AgentCore SDK +###### Example @@ -115,0 +116 @@ AgentCore SDK +AgentCore SDK @@ -118 +119 @@ AgentCore SDK - from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs + 1. from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs @@ -151 +150,0 @@ To target a specific trace, pass `expected_response` as a dict mapping trace IDs -AgentCore CLI @@ -155 +154,4 @@ AgentCore CLI - # Expected response matched against the last trace +AgentCore CLI + + + 1. # Expected response matched against the last trace @@ -177 +178,0 @@ AgentCore CLI -Starter Toolkit SDK @@ -181 +182,4 @@ Starter Toolkit SDK - from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs +Starter Toolkit SDK + + + 1. from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs @@ -210 +212,0 @@ To target a specific trace, pass a tuple of `(trace_id, expected_response)`: -Starter Toolkit CLI @@ -214 +216,4 @@ Starter Toolkit CLI - # Expected response matched against the last trace +Starter Toolkit CLI + + + 1. # Expected response matched against the last trace @@ -237 +241,0 @@ Starter Toolkit CLI -AWS SDK (boto3) @@ -241 +245,4 @@ AWS SDK (boto3) - import boto3 +AWS SDK (boto3) + + + 1. import boto3 @@ -275 +285 @@ AWS SDK (boto3) -`Builtin.GoalSuccessRate` is a session-level evaluator that validates whether the agent's behavior satisfies a set of natural language assertions. Assertions can check tool usage, response content, ordering of actions, or any other observable behavior across the entire conversation. +`Builtin.GoalSuccessRate` is a session-level evaluator that validates whether the agent’s behavior satisfies a set of natural language assertions. Assertions can check tool usage, response content, ordering of actions, or any other observable behavior across the entire conversation. @@ -281 +291 @@ The examples below use assertions that validate tool usage, but assertions are f -AgentCore SDK +###### Example @@ -282,0 +293 @@ AgentCore SDK +AgentCore SDK @@ -285 +296 @@ AgentCore SDK - from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs + 1. from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs @@ -306 +316,0 @@ AgentCore SDK -AgentCore CLI @@ -310 +320,4 @@ AgentCore CLI - agentcore run eval \ +AgentCore CLI + + + 1. agentcore run eval \ @@ -326 +338,0 @@ AgentCore CLI -Starter Toolkit SDK @@ -330 +342,4 @@ Starter Toolkit SDK - from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs +Starter Toolkit SDK + + + 1. from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs @@ -350 +364,0 @@ Starter Toolkit SDK -Starter Toolkit CLI @@ -354 +368,4 @@ Starter Toolkit CLI - agentcore eval run \ +Starter Toolkit CLI + + + 1. agentcore eval run \ @@ -362 +378,0 @@ Starter Toolkit CLI -AWS SDK (boto3) @@ -366 +382,4 @@ AWS SDK (boto3) - import boto3 +AWS SDK (boto3) + + + 1. import boto3 @@ -394 +416 @@ AWS SDK (boto3) -The trajectory evaluators compare the agent's actual tool call sequence against an expected sequence of tool names. Three variants are available, each with different matching strictness. All three are session-level evaluators and use programmatic scoring (no LLM calls, so token usage is zero). +The trajectory evaluators compare the agent’s actual tool call sequence against an expected sequence of tool names. Three variants are available, each with different matching strictness. All three are session-level evaluators and use programmatic scoring (no LLM calls, so token usage is zero). @@ -396 +418 @@ The trajectory evaluators compare the agent's actual tool call sequence against -Trajectory matching variants Evaluator | Matching rule | Example +Evaluator | Matching rule | Example @@ -400 +422 @@ Trajectory matching variants Evaluator | Matching rule | Example -`Builtin.TrajectoryAnyOrderMatch` | All expected tools must be present, order doesn't matter, extras allowed | Expected: `[calculator, weather]`, Actual: `[weather, calculator]` → Pass. +`Builtin.TrajectoryAnyOrderMatch` | All expected tools must be present, order doesn’t matter, extras allowed | Expected: `[calculator, weather]` , Actual: `[weather, calculator]` → Pass. @@ -402 +424 @@ Trajectory matching variants Evaluator | Matching rule | Example -AgentCore SDK +###### Example @@ -403,0 +426 @@ AgentCore SDK +AgentCore SDK @@ -406 +429 @@ AgentCore SDK - from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs + 1. from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs @@ -427 +449,0 @@ AgentCore SDK -AgentCore CLI @@ -430 +451,0 @@ AgentCore CLI -Tool names are passed as a comma-separated list: @@ -431,0 +453,4 @@ Tool names are passed as a comma-separated list: +AgentCore CLI + + + 1. Tool names are passed as a comma-separated list: @@ -448 +472,0 @@ Tool names are passed as a comma-separated list: -Starter Toolkit SDK @@ -452 +476,4 @@ Starter Toolkit SDK - from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs +Starter Toolkit SDK + + + 1. from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs @@ -472 +498,0 @@ Starter Toolkit SDK -Starter Toolkit CLI @@ -475 +500,0 @@ Starter Toolkit CLI -Tool names are passed as a comma-separated list: @@ -476,0 +502,4 @@ Tool names are passed as a comma-separated list: +Starter Toolkit CLI + + + 1. Tool names are passed as a comma-separated list: @@ -486 +514,0 @@ Tool names are passed as a comma-separated list: -AWS SDK (boto3) @@ -490 +518,4 @@ AWS SDK (boto3) - import boto3 +AWS SDK (boto3) + + + 1. import boto3 @@ -521 +555 @@ AWS SDK (boto3) -You can pass all ground truth fields together in a single evaluation call. The service routes each field to the appropriate evaluator and ignores fields that a given evaluator doesn't use. This means you can construct your reference inputs once and reuse them across different evaluators without modifying the payload. +You can pass all ground truth fields together in a single evaluation call. The service routes each field to the appropriate evaluator and ignores fields that a given evaluator doesn’t use. This means you can construct your reference inputs once and reuse them across different evaluators without modifying the payload. @@ -523 +557 @@ You can pass all ground truth fields together in a single evaluation call. The s -AgentCore SDK +###### Example @@ -524,0 +559 @@ AgentCore SDK +AgentCore SDK @@ -527 +562 @@ AgentCore SDK - from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs + 1. from bedrock_agentcore.evaluation import EvaluationClient, ReferenceInputs @@ -557 +591,0 @@ AgentCore SDK -AgentCore CLI @@ -561 +595,4 @@ AgentCore CLI - agentcore run eval \ +AgentCore CLI + + + 1. agentcore run eval \ @@ -573 +609,0 @@ AgentCore CLI -Starter Toolkit SDK @@ -577 +613,4 @@ Starter Toolkit SDK - from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs +Starter Toolkit SDK + + + 1. from bedrock_agentcore_starter_toolkit import Evaluation, ReferenceInputs @@ -604 +642,0 @@ Starter Toolkit SDK -AWS SDK (boto3) @@ -608 +646,4 @@ AWS SDK (boto3) - import boto3 +AWS SDK (boto3) + + + 1. import boto3 @@ -651 +695 @@ AWS SDK (boto3)