AWS wellarchitected documentation change
Summary
Added new best practice FSIREL08-BP05 for monitoring AI model performance/drift with KPI tracking, alerting, and evaluation procedures.
Security assessment
New content addresses operational monitoring of AI model performance and data drift, not security vulnerabilities. Focuses on performance metrics and prediction accuracy without mentioning security controls, threats, or vulnerabilities.
Diff
diff --git a/wellarchitected/latest/financial-services-industry-lens/fsirel08.md b/wellarchitected/latest/financial-services-industry-lens/fsirel08.md index 7285e94c2..a93bdcc92 100644 --- a//wellarchitected/latest/financial-services-industry-lens/fsirel08.md +++ b//wellarchitected/latest/financial-services-industry-lens/fsirel08.md @@ -5 +5 @@ -FSIREL08-BP01 Use a single pane of glass for monitoringFSIREL08-BP02 Alert on the absence of an eventFSIREL08-BP03 Identify metrics and validate alerts through load testingFSIREL08-BP04 Use distributed tracing tools for service-oriented architectures +FSIREL08-BP01 Use a single pane of glass for monitoringFSIREL08-BP02 Alert on the absence of an eventFSIREL08-BP03 Identify metrics and validate alerts through load testingFSIREL08-BP04 Use distributed tracing tools for service-oriented architectures FSIREL08-BP05 Monitor AI model performance and drift @@ -28,0 +29,4 @@ As systems become more distributed with the implementation of microservices arch +## FSIREL08-BP05 Monitor AI model performance and drift + +Continuous monitoring should track key performance indicators against established baselines, with automated alerts for significant deviations and configurable thresholds with escalation procedures. Establish regular cadences for model evaluation using production data, comparing predictions against actual outcomes. Implement comprehensive logging systems that capture input data characteristics, prediction outputs, and environmental factors to facilitate root cause analysis when performance issues arise. For regulated applications, consider deploying parallel inference systems where both current and candidate models run simultaneously to compare outputs before deployment. +