AWS wellarchitected high security documentation change
Summary
Added FSIOPS05-BP03 on implementing generative AI observability including security monitoring.
Security assessment
Adds security-focused monitoring for generative AI including data leakage detection, unauthorized access attempts, and hallucination monitoring. Specific evidence includes implementation for data leakage detection and unauthorized access monitoring.
Diff
diff --git a/wellarchitected/latest/financial-services-industry-lens/fsiops5.md b/wellarchitected/latest/financial-services-industry-lens/fsiops5.md index ff628afc1..82238b7ec 100644 --- a//wellarchitected/latest/financial-services-industry-lens/fsiops5.md +++ b//wellarchitected/latest/financial-services-industry-lens/fsiops5.md @@ -5 +5 @@ -FSIOPS05-BP01 Use enhanced monitoring in the cloudFSIOPS05-BP02 Monitor cloud provider events +FSIOPS05-BP01 Use enhanced monitoring in the cloudFSIOPS05-BP02 Monitor cloud provider eventsFSIOPS05-BP03 Implement comprehensive generative AI observability @@ -52,0 +53,20 @@ You can use organizational view and delegated administrator access to manage AWS +## FSIOPS05-BP03 Implement comprehensive generative AI observability + +Deploy [specialized monitoring](https://aws.amazon.com/cloudwatch/features/generative-ai-observability/) for generative AI workloads that tracks model performance, output quality, token usage, latency, and cost metrics. Monitor for hallucinations, bias, and drift in model outputs. Implement automated alerting for anomalous model behavior and establish prompt performance monitoring to track effectiveness and model response quality over time with automated alerting. + +Define escalation procedures for prompt-related security incidents and integrate with existing incident response frameworks. + +### Prescriptive guidance + +Use Amazon CloudWatch custom metrics to track generative AI-specific KPIs (like tokens per second, prompt success rates, and output validation scores) and customer satisfaction metrics for generative AI-powered interactions. Implement Amazon Bedrock's built-in logging capabilities to capture all model interactions. + +Deploy automated quality checks using AWS Lambda to validate model outputs against expected patterns. Use Amazon SageMaker AI Model Monitor for continuous model performance tracking. + +Set up cost alerting for token usage to prevent unexpected expenses and optimize resource utilization. + +Use Amazon Bedrock's model evaluation capabilities for automated quality assessments and performance benchmarking. + +Establish baseline performance metrics and use Amazon CloudWatch anomaly detection for automated drift identification and alerting. + +Implement data leakage detection in model outputs and monitor unauthorized access attempts to generative AI endpoints. +