AWS bedrock-agentcore documentation change
Summary
Updated documentation for the CreateEvaluator API with minor typographical fixes (apostrophe formatting), restructured example sections with numbered steps, added a note about running CLI commands from an AgentCore project directory, and adjusted heading formatting.
Security assessment
The changes are purely documentation formatting, structural improvements, and clarification of usage steps. There is no mention of security vulnerabilities, patches, or new security features. The note about running commands from the correct directory is a usability tip, not a security advisory.
Diff
diff --git a/bedrock-agentcore/latest/devguide/create-evaluator.md b/bedrock-agentcore/latest/devguide/create-evaluator.md index 83861f3cf..392a57cdf 100644 --- a//bedrock-agentcore/latest/devguide/create-evaluator.md +++ b//bedrock-agentcore/latest/devguide/create-evaluator.md @@ -9 +9 @@ Code samples for AgentCore CLI, AgentCore SDK, and AWS SDKCustom evaluator confi -The `CreateEvaluator` API creates a new custom evaluator that defines how to assess specific aspects of your agent's behavior. This asynchronous operation returns immediately while the evaluator is being provisioned. The API returns the evaluator ARN, ID, creation timestamp, and initial status. Once created, the evaluator can be referenced in online evaluation configurations. +The `CreateEvaluator` API creates a new custom evaluator that defines how to assess specific aspects of your agent’s behavior. This asynchronous operation returns immediately while the evaluator is being provisioned. The API returns the evaluator ARN, ID, creation timestamp, and initial status. Once created, the evaluator can be referenced in online evaluation configurations. @@ -32 +32 @@ The `CreateEvaluator` API creates a new custom evaluator that defines how to ass - * `context` – All information from previous turns, including user prompts, tool calls, and assistant responses, plus the current turn's user prompt and tool call. + * `context` – All information from previous turns, including user prompts, tool calls, and assistant responses, plus the current turn’s user prompt and tool call. @@ -40 +40 @@ The `CreateEvaluator` API creates a new custom evaluator that defines how to ass - * `context` – All information from previous turns (user prompts, tool call details, assistant responses) plus the current turn's user prompt and any tool calls made before the tool call being evaluated. + * `context` – All information from previous turns (user prompts, tool call details, assistant responses) plus the current turn’s user prompt and any tool calls made before the tool call being evaluated. @@ -138 +138 @@ Using the above JSON, you can create the custom evaluator through the API client -AgentCore CLI +###### Example @@ -139,0 +140 @@ AgentCore CLI +AgentCore CLI @@ -142 +143 @@ AgentCore CLI - agentcore add evaluator \ + 1. agentcore add evaluator \ @@ -146,0 +148,9 @@ AgentCore CLI +This command adds the evaluator to your local `agentcore.json` configuration. Run `agentcore deploy` to create it in your AWS account. + +###### Note + +Run this from inside an AgentCore project directory (created with `agentcore create` ). + + + + @@ -180,2 +190 @@ AgentCore SDK - - import json + 1. import json @@ -198 +206,0 @@ AgentCore SDK -AWS SDK @@ -202 +210,4 @@ AWS SDK - import boto3 +AWS SDK + + + 1. import boto3 @@ -218 +228,0 @@ AWS SDK -AWS CLI @@ -222 +232,4 @@ AWS CLI - aws bedrock-agentcore-control create-evaluator \ +AWS CLI + + + 1. aws bedrock-agentcore-control create-evaluator \ @@ -230,0 +247,2 @@ The following examples show how to create custom evaluators that use ground trut +###### Example + @@ -234 +252 @@ Trajectory compliance evaluator (session-level) -This evaluator uses an LLM to compare the expected and actual tool trajectories, allowing for nuanced judgment — for example, tolerating minor deviations like extra helper tool calls. It uses the `expected_tool_trajectory` and `actual_tool_trajectory` placeholders. + 1. This evaluator uses an LLM to compare the expected and actual tool trajectories, allowing for nuanced judgment — for example, tolerating minor deviations like extra helper tool calls. It uses the `expected_tool_trajectory` and `actual_tool_trajectory` placeholders. @@ -270 +289 @@ Assertion checker evaluator (session-level) -This evaluator checks whether the agent's behavior satisfies a set of assertions, returning a categorical PASS/FAIL/INCONCLUSIVE verdict. It uses the `assertions` placeholder along with `context` and `available_tools`. + 1. This evaluator checks whether the agent’s behavior satisfies a set of assertions, returning a categorical PASS/FAIL/INCONCLUSIVE verdict. It uses the `assertions` placeholder along with `context` and `available_tools`. @@ -306 +326 @@ Response similarity evaluator (trace-level) -This evaluator compares the agent's actual response against an expected response, scoring semantic similarity. It uses the `expected_response` placeholder to receive the ground truth at evaluation time. + 1. This evaluator compares the agent’s actual response against an expected response, scoring semantic similarity. It uses the `expected_response` placeholder to receive the ground truth at evaluation time. @@ -342 +363 @@ Create the evaluator: -You can create custom evaluators using the Amazon Bedrock AgentCore console's visual interface. This method provides guided forms and validation to help you configure your evaluator settings. +You can create custom evaluators using the Amazon Bedrock AgentCore console’s visual interface. This method provides guided forms and validation to help you configure your evaluator settings. @@ -344 +365 @@ You can create custom evaluators using the Amazon Bedrock AgentCore console's vi -###### To create an AgentCore custom evaluator +**To create an AgentCore custom evaluator** @@ -407 +428 @@ Writing well-structured evaluator instructions is critical for accurate assessme - * Context Management: In your instruction, choose context placeholders strategically based on your specific requirements. Find the right balance between providing sufficient information and avoiding evaluator confusion. Adjust context depth according to your judge model's capabilities and limitations. + * Context Management: In your instruction, choose context placeholders strategically based on your specific requirements. Find the right balance between providing sufficient information and avoiding evaluator confusion. Adjust context depth according to your judge model’s capabilities and limitations.