AWS cli documentation change
Summary
Updated AWS CLI version reference from 2.34.16 to 2.34.19 and expanded the create-evaluator command documentation to include code-based evaluators using customer-managed Lambda functions alongside existing LLM-as-a-Judge configurations.
Security assessment
The change introduces a new configuration option (code-based evaluators with Lambda functions) but does not address any specific security vulnerability, weakness, or incident. It documents a new feature that allows customers to use their own Lambda functions for evaluation, which could have security implications (like proper IAM roles and function security), but the change itself is a feature addition, not a security fix.
Diff
diff --git a/cli/latest/reference/bedrock-agentcore-control/create-evaluator.md b/cli/latest/reference/bedrock-agentcore-control/create-evaluator.md index 0dce30ade..7d4266a23 100644 --- a//cli/latest/reference/bedrock-agentcore-control/create-evaluator.md +++ b//cli/latest/reference/bedrock-agentcore-control/create-evaluator.md @@ -15 +15 @@ - * [AWS CLI 2.34.16 Command Reference](../../index.html) » + * [AWS CLI 2.34.19 Command Reference](../../index.html) » @@ -59 +59 @@ First time using the AWS CLI? See the [User Guide](https://docs.aws.amazon.com/c -Creates a custom evaluator for agent quality assessment. Custom evaluators use LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings to evaluate agent performance at tool call, trace, or session levels. +Creates a custom evaluator for agent quality assessment. Custom evaluators can use either LLM-as-a-Judge configurations with user-defined prompts, rating scales, and model settings, or code-based configurations with customer-managed Lambda functions to evaluate agent performance at tool call, trace, or session levels. @@ -135 +135 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc -> The configuration for the evaluator, including LLM-as-a-Judge settings with instructions, rating scale, and model configuration. +> The configuration for the evaluator. Specify either LLM-as-a-Judge settings with instructions, rating scale, and model configuration, or code-based settings with a customer-managed Lambda function. @@ -139 +139 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc -> This is a Tagged Union structure. Only one of the following top level keys can be set: `llmAsAJudge`. +> This is a Tagged Union structure. Only one of the following top level keys can be set: `llmAsAJudge`, `codeBased`. @@ -286,0 +287,33 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc +> +> codeBased -> (tagged union structure) +> +>> Configuration for a code-based evaluator that uses a customer-managed Lambda function to programmatically assess agent performance. +>> +>> ### Note +>> +>> This is a Tagged Union structure. Only one of the following top level keys can be set: `lambdaConfig`. +>> +>> lambdaConfig -> (structure) +>> +>>> The Lambda function configuration for code-based evaluation. +>>> +>>> lambdaArn -> (string) [required] +>>> +>>>> The Amazon Resource Name (ARN) of the Lambda function that implements the evaluation logic. +>>>> +>>>> Constraints: +>>>> +>>>> * pattern: `arn:(aws[a-zA-Z-]*)?:lambda:([a-z]{2}(-gov)?-[a-z]+-\d{1}):(\d{12}):function:([a-zA-Z0-9-_.]+)(:(\$LATEST|[a-zA-Z0-9-_]+))?` +>>>> + +>>> +>>> lambdaTimeoutInSeconds -> (integer) +>>> +>>>> The timeout in seconds for the Lambda function invocation. Defaults to 60. Must be between 1 and 300. +>>>> +>>>> Constraints: +>>>> +>>>> * min: `1` +>>>> * max: `300` +>>>> + @@ -322,0 +356,6 @@ JSON Syntax: + }, + "codeBased": { + "lambdaConfig": { + "lambdaArn": "string", + "lambdaTimeoutInSeconds": integer + } @@ -545 +584 @@ status -> (string) - * [AWS CLI 2.34.16 Command Reference](../../index.html) » + * [AWS CLI 2.34.19 Command Reference](../../index.html) »