AWS wellarchitected documentation change
Summary
Added comprehensive monitoring guidance for SageMaker AI HyperPod with EKS/Slurm, including observability capabilities, metric collection, alerting, and incident response. Updated section headings and related content references.
Security assessment
The changes focus on operational monitoring and observability improvements for performance metrics (latency, requests, error rates) and system health. While monitoring can support security, there's no evidence of addressing specific vulnerabilities, threats, or security controls. Changes describe operational best practices rather than security features.
Diff
diff --git a/wellarchitected/latest/generative-ai-lens/genops02-bp02.md b/wellarchitected/latest/generative-ai-lens/genops02-bp02.md index 21bd11d08..7daeab21e 100644 --- a//wellarchitected/latest/generative-ai-lens/genops02-bp02.md +++ b//wellarchitected/latest/generative-ai-lens/genops02-bp02.md @@ -31,0 +32,12 @@ If you're using Amazon SageMaker AI for hosting models, use the invocation and r +For SageMaker AI HyperPod with both Amazon EKS and Slurm orchestration, use the system's comprehensive one-click observability capabilities that automatically collect and visualize key metrics across operational layers. + +For EKS-based HyperPod, use the integrated Amazon EKS add-on for SageMaker AI HyperPod observability that consolidates health and performance data from NVIDIA DCGM, Kubernetes node exporters, Elastic Fabric Adapter (EFA), and file systems into unified Amazon Managed Grafana dashboards with metrics automatically published to Amazon Managed Service for Prometheus. + +Configure CloudWatch Container Insights for enhanced monitoring of CPU, GPU, Trainium, EFA, and file system metrics up to the container level, while implementing automated alerting for model invocation latency, concurrent requests, error rates, and token-level metrics. + +For Slurm-based HyperPod, implement comprehensive monitoring through node exporters for system metrics, NVIDIA DCGM for GPU health monitoring, and EFA metrics for network performance tracking, all integrated with the unified observability solution. + +Both systems benefit from SageMaker AI HyperPod's real-time task performance metric tracking with automated alerting capabilities, automatic root cause remediation with customer-defined policies, and inference observability that captures essential model performance data including invocation latency, concurrent requests, error rates, and token-level metrics through standardized Prometheus endpoints. + +Additionally, establish incident response playbooks for when alerts trigger, configure custom thresholds based on workload-specific patterns, and use a unified dashboard that reduces troubleshooting time from days to minutes through pre-built, actionable insights. + @@ -99 +111 @@ To enable automated responses to specific events, consider implementing Amazon E -**Related practices:** +**Related best practices:** @@ -112 +124 @@ To enable automated responses to specific events, consider implementing Amazon E -**Related guides, videos, and documentation:** +**Related documents:** @@ -123,0 +136,4 @@ To enable automated responses to specific events, consider implementing Amazon E + * [Accelerate Foundation Model Development with One-Click Observability in Amazon SageMaker AI HyperPod](https://aws.amazon.com/blogs/machine-learning/accelerate-foundation-model-development-with-one-click-observability-in-amazon-sagemaker-hyperpod/) + + * [Amazon SageMaker AI HyperPod launches model deployments to accelerate the generative AI model development lifecycle](https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-hyperpod-launches-model-deployments-to-accelerate-the-generative-ai-model-development-lifecycle/) + @@ -159 +175 @@ GENOPS02-BP01 Monitor all application layers -GENOPS02-BP03 Implement rate limiting and throttling to mitigate the risk of system overload +GENOPS02-BP03 Implement solutions to mitigate the risk of system overload