AWS bedrock-agentcore documentation change
Summary
Updated documentation to replace references from 'AgentCore starter toolkit' to 'AgentCore CLI', updated command syntax from 'agentcore eval run' to 'agentcore run eval', changed parameters from agent-id to runtime, added interactive TUI instructions for on-demand evaluation, and updated code examples with proper string formatting.
Security assessment
These changes are primarily tooling updates, command syntax changes, and user interface improvements. There is no evidence of addressing security vulnerabilities or weaknesses. The changes focus on usability, terminology updates, and providing clearer instructions for running evaluations.
Diff
diff --git a/bedrock-agentcore/latest/devguide/getting-started-on-demand.md b/bedrock-agentcore/latest/devguide/getting-started-on-demand.md index 0b84d8cdc..fded2a36f 100644 --- a//bedrock-agentcore/latest/devguide/getting-started-on-demand.md +++ b//bedrock-agentcore/latest/devguide/getting-started-on-demand.md @@ -122 +122 @@ Once you have made a few invocations to your agent, you are ready to evaluate it - * **AgentCore starter toolkit** does this for you automatically and is the easiest to get started with. + * **AgentCore CLI** does this for you automatically and is the easiest to get started with. @@ -124 +124 @@ Once you have made a few invocations to your agent, you are ready to evaluate it - * If you are not using starter toolkit, we will show how to download logs using session-id and use them for evaluation using AWS SDK. + * If you are not using the AgentCore CLI, we will show how to download logs using session-id and use them for evaluation using the AWS SDK. @@ -131 +131 @@ Once you have made a few invocations to your agent, you are ready to evaluate it - * Code samples for Starter Toolkit, AgentCore SDK, and AWS SDK + * Code samples for AgentCore CLI and AgentCore SDK @@ -138 +138 @@ Once you have made a few invocations to your agent, you are ready to evaluate it -### Code samples for Starter Toolkit, AgentCore SDK, and AWS SDK +### Code samples for AgentCore CLI and AgentCore SDK @@ -142 +142 @@ The following code samples demonstrate how to run on-demand evaluations using di -AgentCore starter toolkit CLI +AgentCore CLI @@ -146 +146 @@ AgentCore starter toolkit CLI - # Runs evaluation for the specified agent and session. + # Runs evaluation for the specified runtime and session. @@ -149 +149 @@ AgentCore starter toolkit CLI - AGENT_ID="YOUR_AGENT_ID" + RUNTIME_NAME="your_runtime_name" @@ -151,2 +151,2 @@ AgentCore starter toolkit CLI - agentcore eval run \ - --agent-id $AGENT_ID \ + agentcore run eval \ + --runtime $RUNTIME_NAME \ @@ -157,3 +157,3 @@ AgentCore starter toolkit CLI - # Auto reads default agentId and sessionId from current agent config (.bedrock_agentcore.yaml) if available - # Verify using ```agentcore status``` or look for ".bedrock_agentcore.yaml" - agentcore eval run \ + # Auto reads default runtime from current project config if available + # Verify using ```agentcore status``` + agentcore run eval \ @@ -163 +163,17 @@ AgentCore starter toolkit CLI -AgentCore starter toolkit SDK +Interactive + + +Run `agentcore` to open the TUI, then select **run** and choose **On-demand Evaluation** : + + 1. Select evaluators to run against agent traces: + + + + 2. Review the configuration and press Enter to confirm: + + + + + + +AgentCore SDK @@ -222,3 +238,3 @@ It takes a couple of minutes for logs to get populated in CloudWatch, so its pos - region = region-code - agent_id = add the agent-id from step-2 here - session_id = use the session-id from step-3 here + region = "region-code" + agent_id = "agent-id-from-step-2" + session_id = "session-id-from-step-3" @@ -468 +484 @@ IAM permissions for on-demand evaluation -Understanding input spans +Ground truth evaluations