AWS wellarchitected documentation change
Summary
Expanded explanation about challenges with foundation model randomness and added guidance for maintaining performance thresholds/remediation strategies
Security assessment
The changes focus on performance management challenges with non-deterministic AI models and operational strategies for maintaining performance. There is no mention of security vulnerabilities, access controls, data protection, or other security-related concepts. The additions relate to technical performance monitoring rather than security controls.
Diff
diff --git a/wellarchitected/latest/generative-ai-lens/genperf02.md b/wellarchitected/latest/generative-ai-lens/genperf02.md index bc2894dd3..e9f61e71f 100644 --- a//wellarchitected/latest/generative-ai-lens/genperf02.md +++ b//wellarchitected/latest/generative-ai-lens/genperf02.md @@ -10 +10,3 @@ GENPERF02: How do you verify your generative AI workload maintains acceptable pe -Foundation models are inherently non-deterministic. They introduce an element of randomness into systems which must be accounted for. Furthermore, while they are flexible and multi-purposed, foundation models are compute-intensive resources that may require tuning and customization to meet your organization requirements. +Foundation models are inherently non-deterministic. They introduce an element of randomness into systems. This randomness can be difficult to account for, especially when traditional performance evaluation techniques rely on a determinism. Furthermore, while they are flexible, broadly applicable, and capable performing multiple tasks, foundation models are compute-intensive resources that may require tuning and customization to meet your organization AI requirements. + +Developing a methodology for maintaining consistent model performance in a rapidly evolving environment of available models requires well-understood minimum performance thresholds, clear requirements for each model task, and a suite of remediation actions in the case of performance degradation or new model availability.