AWS AmazonCloudWatch documentation change
Summary
Added detailed documentation about Application Signals' support for LLM models, fine-grained metrics, and OpenTelemetry GenAI attributes for Amazon Bedrock API calls
Security assessment
The changes focus on observability and monitoring features for generative AI applications, with no mention of security vulnerabilities or security-specific features
Diff
diff --git a/AmazonCloudWatch/latest/monitoring/Services-example-scenario-GenerativeAI.md index 7d5738f8c..8eb988517 100644 --- a/AmazonCloudWatch/latest/monitoring/Services-example-scenario-GenerativeAI.md +++ b/AmazonCloudWatch/latest/monitoring/Services-example-scenario-GenerativeAI.md @@ -4,0 +5,2 @@ +Fine-grained metrics and tracesOpenTelemetry GenAI attributes support + @@ -7 +9,83 @@ -You can use Application Signals to troubleshoot your generative AI applications that interact with Amazon Bedrock models. Application Signals allows you to drill down into the metrics, trace and logs of your application and its dependencies such as Amazon Bedrock models to gain a deeper understanding of how failures like model validation exceptions, model configuration, and latency in different models impact the end-user experience. For more information, see the AWS blog post, [Improve Amazon Bedrock Observability with Amazon CloudWatch Application Signals.](https://aws.amazon.com/blogs/mt/improve-amazon-bedrock-observability-with-amazon-cloudwatch-appsignals/) +You can use Application Signals to troubleshoot your generative AI applications that interact with Amazon Bedrock models. Application Signals streamlines this process by providing out-of-the-box telemetry data, offering deeper insights into your application's interactions with LLM models. It helps address key use cases such as: + + * Model configuration issues + + * Model usage costs + + * Model latency + + * Model response generation stopped reasons + + + + +[Enabling Application Signals](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Signals-Enable.html) with LLM/GenAI Observability provides real-time visibility into your application's interactions with Amazon Bedrock services. Application Signals automatically generates and correlates performance metrics and traces for Amazon Bedrock API calls. + +Application Signals currently support the following LLM Models from Amazon Bedrock. + + * AI21 Jamba + + * Amazon Titan + + * Anthropic Claude + + * Cohere Command + + * Meta Llama + + * Mistral AI + + * Nova + + + + +## Fine-grained metrics and traces + +For each Amazon Bedrock API call, Application Signals generates detailed performance metrics at the resource level, including: + + * Model ID + + * Guardrails ID + + * Knowledge Base ID + + * Bedrock Agent ID + + + + +Additionally, correlated trace spans at the same level help provide a comprehensive view of request execution and dependencies. + + + +## OpenTelemetry GenAI attributes support + +Application Signals generates the following GenAI attributes for Amazon Bedrock API calls with OpenTelemetry semantic convention. These attributes help analyze model usage, cost, and response quality, and can be leveraged through [Transaction Search](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Transaction-Search.html) for deeper insights. + + * gen_ai.system + + * gen_ai.request.model + + * gen_ai.request.max_tokens + + * gen_ai.request.temperature + + * gen_ai.request.top_p + + * gen_ai.usage.input_tokens + + * gen_ai.usage.output_tokens + + * gen_ai.response.finish_reasons + + + + + + +For example, your can leverage the analytic capability from Transaction Search to compare the token usage and cost across different LLM models for the same prompt, enabling cost-efficient model selection. + + + +For more information, see [Improve Amazon Bedrock Observability with CloudWatch Application Signals](https://aws.amazon.com/blogs/mt/improve-amazon-bedrock-observability-with-amazon-cloudwatch-appsignals/).