AWS bedrock-agentcore documentation change

Service: bedrock-agentcore · 2026-04-01 · Documentation low

File: bedrock-agentcore/latest/devguide/create-online-evaluations.md

Summary

Updated documentation to replace references from 'Starter Toolkit' to 'AgentCore CLI' and revised the CLI command syntax and parameters for creating online evaluations, adding interactive console instructions with screenshots.

Security assessment

The changes are focused on updating tooling references and command syntax, along with adding user interface guidance. There is no mention of security vulnerabilities, patches, or security features. The removal of IAM role creation notes and agent ID parameters suggests a simplification of the CLI interface, but does not indicate a security fix.

Diff

diff --git a/bedrock-agentcore/latest/devguide/create-online-evaluations.md b/bedrock-agentcore/latest/devguide/create-online-evaluations.md
index cf6d2dce1..17dcfbd58 100644
--- a//bedrock-agentcore/latest/devguide/create-online-evaluations.md
+++ b//bedrock-agentcore/latest/devguide/create-online-evaluations.md
@@ -5 +5 @@
-Execution status controlEvaluator protectionCode samples for Starter Toolkit, AgentCore SDK, and AWS SDKConsole
+Execution status controlEvaluator protectionCode samples for AgentCore CLI, AgentCore SDK, and AWS SDKConsole
@@ -19 +19 @@ When you create an online evaluation, you specify a unique configuration name, t
-  * Code samples for Starter Toolkit, AgentCore SDK, and AWS SDK
+  * Code samples for AgentCore CLI, AgentCore SDK, and AWS SDK
@@ -48 +48 @@ When you create an evaluation configuration with `executionStatus` set to `ENABL
-## Code samples for Starter Toolkit, AgentCore SDK, and AWS SDK
+## Code samples for AgentCore CLI, AgentCore SDK, and AWS SDK
@@ -57,7 +57,4 @@ AgentCore CLI
-    # - Config names must use underscores, not hyphens
-    # - By default, auto-creates IAM execution role and enables the config
-    AGENT_ID="YOUR_AGENT_ID"
-    CONFIG_NAME="YOUR_CONFIG_NAME"
-    agentcore eval online create \
-      --name $CONFIG_NAME \
-      --agent-id $AGENT_ID \
+    agentcore add online-eval \
+      --name "your_config_name" \
+      --runtime "your_runtime_name" \
+      --evaluator "Builtin.GoalSuccessRate" "Builtin.Helpfulness" \
@@ -65,3 +62,27 @@ AgentCore CLI
-      --evaluator "Builtin.GoalSuccessRate" \
-      --evaluator "Builtin.Helpfulness" \
-      --description "Online Evaluation Config"
+      --enable-on-create
+
+Interactive
+    
+
+  1. Enter a name for your online evaluation configuration.
+
+![Online eval config name input](/images/bedrock-agentcore/latest/devguide/images/tui/online-eval-add-name.png)
+
+  2. Select the evaluators to include. You can choose from built-in evaluators and any custom evaluators you have created.
+
+![Evaluator multi-select list](/images/bedrock-agentcore/latest/devguide/images/tui/online-eval-add-evaluators.png)
+
+  3. Set the sampling rate — the percentage of agent requests that will be evaluated.
+
+![Sampling rate input](/images/bedrock-agentcore/latest/devguide/images/tui/online-eval-add-sampling-rate.png)
+
+  4. Choose whether to enable evaluation automatically after deployment.
+
+![Enable on deploy selection](/images/bedrock-agentcore/latest/devguide/images/tui/online-eval-add-enable.png)
+
+  5. Review the configuration and press Enter to confirm.
+
+![Review online eval configuration](/images/bedrock-agentcore/latest/devguide/images/tui/online-eval-add-confirm.png)
+
+
+