AWS sagemaker documentation change
Summary
Restructured observability documentation to emphasize SageMaker HyperPod's native observability add-on and CloudWatch integration, removed detailed CloudWatch setup instructions, added links to new observability implementation guides
Security assessment
The changes focus on improving observability documentation and monitoring capabilities rather than addressing specific security vulnerabilities. While enhanced observability can help detect operational issues, there is no direct evidence of security vulnerability mitigation or new security features being documented.
Diff
diff --git a/sagemaker/latest/dg/sagemaker-hyperpod-eks-cluster-observability-cluster.md b/sagemaker/latest/dg/sagemaker-hyperpod-eks-cluster-observability-cluster.md index 9724079a0..d9e0cd525 100644 --- a//sagemaker/latest/dg/sagemaker-hyperpod-eks-cluster-observability-cluster.md +++ b//sagemaker/latest/dg/sagemaker-hyperpod-eks-cluster-observability-cluster.md @@ -5 +5 @@ -Amazon CloudWatch Container InsightsSet up an Amazon Managed Grafana workspace +# Cluster and task observability @@ -7 +7 @@ Amazon CloudWatch Container InsightsSet up an Amazon Managed Grafana workspace -# Cluster observability +There are two options for monitoring SageMaker HyperPod clusters: @@ -9 +9 @@ Amazon CloudWatch Container InsightsSet up an Amazon Managed Grafana workspace -To gain visibility into the cluster resource utilization, set up Amazon CloudWatch Container Insights and Amazon Managed Grafana to extract metrics and visualize them on various dashboards. +**The SageMaker HyperPod observability add-on** —SageMaker HyperPod provides a comprehensive, out-of-the-box dashboard that gives you insights into foundation model (FM) development tasks and cluster resources. This unified observability solution automatically publishes key metrics to Amazon Managed Service for Prometheus and displays them in Amazon Managed Grafana dashboards. The dashboards are optimized specifically for FM development with deep coverage of hardware health, resource utilization, and task-level performance. With this add-on, you can consolidate health and performance data from NVIDIA DCGM, instance-level Kubernetes node exporters, Elastic Fabric Adapter, integrated file systems, Kubernetes APIs, Kueue, and SageMaker HyperPod task operators. @@ -11,49 +11 @@ To gain visibility into the cluster resource utilization, set up Amazon CloudWat -###### Topics - - * Amazon CloudWatch Container Insights - - * Set up an Amazon Managed Grafana workspace - - - - -## Amazon CloudWatch Container Insights - -Use [Amazon CloudWatch Container Insights](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights.html) to collect, aggregate, and summarize metrics and logs from the containerized applications and micro-services on the EKS cluster associated with a HyperPod cluster. - -Amazon CloudWatch Insights collects metrics for compute resources, such as CPU, memory, disk, and network. Container Insights also provides diagnostic information, such as container restart failures, to help you isolate issues and resolve them quickly. You can also set CloudWatch alarms on metrics that Container Insights collects. - -To find a complete list of metrics, see [Amazon EKS and Kubernetes Container Insights metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-metrics-EKS.html) in the _Amazon EKS User Guide_. - -### Install CloudWatch Container Insights - -Cluster admin users should set up CloudWatch Container Insights following the instructions at [Install the CloudWatch agent by using the Amazon CloudWatch Observability EKS add-on or the Helm chart](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Observability-EKS-addon.html) in the _CloudWatch User Guide_. For more information about Amazon EKS add-on, see also [Install the Amazon CloudWatch Observability EKS add-on](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-EKS-addon.html) in the _Amazon EKS User Guide_. - -After the installation has completed, verify that the CloudWatch Observability add-on is visible in the EKS cluster add-on tab. It might take about a couple of minutes until the dashboard loads. - -###### Note - -SageMaker HyperPod requires the CloudWatch Insight v2.0.1-eksbuild.1 or later. - - - -### Access CloudWatch container insights dashboard - - 1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). - - 2. Choose **Insights** , and then choose **Container Insights**. - - 3. Select the EKS cluster set up with the HyperPod cluster you're using. - - 4. View the Pod/Cluster level metrics. - - - - - - -### Access CloudWatch container insights logs - - 1. Open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/). - - 2. Choose **Logs** , and then choose **Log groups**. +**Amazon CloudWatch Insights** —Amazon CloudWatch Insights collects metrics for compute resources, such as CPU, memory, disk, and network. Container Insights also provides diagnostic information, such as container restart failures, to help you isolate issues and resolve them quickly. You can also set CloudWatch alarms on metrics that Container Insights collects. @@ -60,0 +13 @@ SageMaker HyperPod requires the CloudWatch Insight v2.0.1-eksbuild.1 or later. +###### Topics @@ -61,0 +15 @@ SageMaker HyperPod requires the CloudWatch Insight v2.0.1-eksbuild.1 or later. + * [Observability with Amazon Managed Grafana and Amazon Managed Service for Prometheus](./sagemaker-hyperpod-observability-addon.html) @@ -62,0 +17 @@ SageMaker HyperPod requires the CloudWatch Insight v2.0.1-eksbuild.1 or later. + * [Observability with Amazon CloudWatch](./sagemaker-hyperpod-eks-cluster-observability-cluster-cloudwatch-ci.html) @@ -64 +18,0 @@ SageMaker HyperPod requires the CloudWatch Insight v2.0.1-eksbuild.1 or later. -When you have the HyperPod clusters integrated with Amazon CloudWatch Container Insights, you can access the relevant log groups in the following format: `/aws/containerinsights /<eks-cluster-name>/*`. Within this log group, you can find and explore various types of logs such as Performance logs, Host logs, Application logs, and Data plane logs. @@ -66 +19,0 @@ When you have the HyperPod clusters integrated with Amazon CloudWatch Container -## Set up an Amazon Managed Grafana workspace @@ -68 +20,0 @@ When you have the HyperPod clusters integrated with Amazon CloudWatch Container -You can integrate SageMaker HyperPod with Amazon Managed Grafana and Amazon Managed Service for Prometheus to gain comprehensive cluster observability and visualize in various Grafana dashboards: the Kubernetes cluster monitoring dashboard, the NVIDIA DCGM exporter dashboard, and the FSx for Lustre metrics dashboard, and the EFA metrics dashboard. @@ -78 +30 @@ Model observability -HyperPod in Studio +Observability with Amazon Managed Grafana and Amazon Managed Service for Prometheus