AWS bedrock-agentcore documentation change
Summary
Added CLI commands for pausing/resuming online evaluations, added deployment note, reformatted code examples with step numbers, and updated apostrophe style.
Security assessment
Changes are documentation enhancements and feature clarifications. The addition of pause/resume CLI commands provides operational control but does not address a security vulnerability or document a security feature. The note about evaluator locking upon enabling an evaluation configuration is a pre-existing feature description, not a new security addition.
Diff
diff --git a/bedrock-agentcore/latest/devguide/create-online-evaluations.md b/bedrock-agentcore/latest/devguide/create-online-evaluations.md index 17dcfbd58..25f837bca 100644 --- a//bedrock-agentcore/latest/devguide/create-online-evaluations.md +++ b//bedrock-agentcore/latest/devguide/create-online-evaluations.md @@ -9 +9 @@ Execution status controlEvaluator protectionCode samples for AgentCore CLI, Agen -The `CreateOnlineEvaluationConfig` API creates a new online evaluation configuration that continuously monitors your agent's performance using live traffic. This asynchronous operation sets up the service to evaluate agent traces as they are generated during normal operation. +The `CreateOnlineEvaluationConfig` API creates a new online evaluation configuration that continuously monitors your agent’s performance using live traffic. This asynchronous operation sets up the service to evaluate agent traces as they are generated during normal operation. @@ -36,0 +37,9 @@ The `executionStatus` parameter determines whether the evaluation job actively p +You can control execution status using the CLI: + + + # Pause a running online evaluation + agentcore pause online-eval "your_config_name" + + # Resume a paused online evaluation + agentcore resume online-eval "your_config_name" + @@ -39 +48 @@ The `executionStatus` parameter determines whether the evaluation job actively p -When you create an evaluation configuration with `executionStatus` set to `ENABLED`, the system automatically locks any custom evaluators you've selected. Once locked: +When you create an evaluation configuration with `executionStatus` set to `ENABLED` , the system automatically locks any custom evaluators you’ve selected. Once locked: @@ -41 +50 @@ When you create an evaluation configuration with `executionStatus` set to `ENABL - * **No modifications allowed** – You cannot update the evaluator's configuration, prompts, or settings. Clone a new evaluator if you need to make changes. + * **No modifications allowed** – You cannot update the evaluator’s configuration, prompts, or settings. Clone a new evaluator if you need to make changes. @@ -52 +61 @@ The following code samples demonstrate how to create online evaluation configura -AgentCore CLI +###### Example @@ -53,0 +63 @@ AgentCore CLI +AgentCore CLI @@ -56 +66 @@ AgentCore CLI - # Create online evaluation configuration + 1. # Create online evaluation configuration @@ -63,0 +74,9 @@ AgentCore CLI +This command adds the online evaluation configuration to your local `agentcore.json` . Run `agentcore deploy` to create it in your AWS account. + +###### Note + +Run this from inside an AgentCore project directory (created with `agentcore create` ). + + + + @@ -93,2 +112 @@ AgentCore SDK - - from bedrock_agentcore_starter_toolkit import Evaluation + 1. from bedrock_agentcore_starter_toolkit import Evaluation @@ -122 +139,0 @@ AgentCore SDK -AWS SDK @@ -126 +143,4 @@ AWS SDK - import boto3 +AWS SDK + + + 1. import boto3 @@ -156 +175,0 @@ AWS SDK -AWS CLI @@ -160 +179,4 @@ AWS CLI - aws bedrock-agentcore-control create-online-evaluation-config \ +AWS CLI + + + 1. aws bedrock-agentcore-control create-online-evaluation-config \ @@ -171 +196 @@ AWS CLI -You can create online evaluation configurations using the Amazon Bedrock AgentCore console's visual interface. This method provides guided forms and validation to help you configure your evaluation settings. +You can create online evaluation configurations using the Amazon Bedrock AgentCore console’s visual interface. This method provides guided forms and validation to help you configure your evaluation settings. @@ -173 +198 @@ You can create online evaluation configurations using the Amazon Bedrock AgentCo -###### To create an AgentCore online evaluation +**To create an AgentCore online evaluation** @@ -183 +208 @@ You can create online evaluation configurations using the Amazon Bedrock AgentCo - 2. (Optional) To enable the evaluation configuration after it's created, select the checkbox under the evaluation name. + 2. (Optional) To enable the evaluation configuration after it’s created, select the checkbox under the evaluation name.