AWS prometheus documentation change
Summary
Reorganized warning sections, added new quota-related warnings (Active alerts, Alert aggregation group size, Alerts size), updated remediation steps for quota limits
Security assessment
Changes focus on operational troubleshooting for quota limits and alert formatting. While quota exhaustion could impact availability, there's no evidence of addressing vulnerabilities or exploits. Security documentation additions are about system limits rather than security features.
Diff
diff --git a/prometheus/latest/userguide/Troubleshooting-alerting.md b/prometheus/latest/userguide/Troubleshooting-alerting.md index 1e4e3fe28..8891f49cc 100644 --- a//prometheus/latest/userguide/Troubleshooting-alerting.md +++ b//prometheus/latest/userguide/Troubleshooting-alerting.md @@ -5 +5 @@ -Empty content warningNon ASCII warningInvalid key/value warningMessage limit warningNo resource based policy errorNot authorized to call KMS +Active alerts warningAlert aggregation group size warningAlerts size too big warningEmpty content warningInvalid key/value warningMessage limit warningNo resource based policy errorNon ASCII warningNot authorized to call KMS @@ -13 +13 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs - * Empty content warning + * Active alerts warning @@ -15 +15,5 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs - * Non ASCII warning + * Alert aggregation group size warning + + * Alerts size too big warning + + * Empty content warning @@ -22,0 +27,2 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs + * Non ASCII warning + @@ -28 +34 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs -## Empty content warning +## Active alerts warning @@ -34 +40 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs - "workspaceId": "ws-abcd1234-ef56-78ab-cd90-1234abcd0000", + "workspaceId": "ws-efdc5b42-b051-11ec-b123-4567ac120002", @@ -36 +42 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs - "log": "Message has been modified because the content was empty." + "log": "too many alerts, limit: 1000", @@ -42 +48 @@ Using [Monitor Amazon Managed Service for Prometheus events with CloudWatch Logs -This means that the Alert manager template resolved the outbound alert to an empty message. +This means that the Alert manager **Active alerts** quota is exceeded. @@ -46 +52 @@ This means that the Alert manager template resolved the outbound alert to an emp -Validate your Alert manager template and ensure that you have a valid template for all receiver pathways. +Request a quota increase. Sign in to the AWS Management Console and open the Service Quotas console at [https://console.aws.amazon.com/servicequotas/](https://console.aws.amazon.com/servicequotas/). @@ -48 +54,43 @@ Validate your Alert manager template and ensure that you have a valid template f -## Non ASCII warning +## Alert aggregation group size warning + +**When the log contains the following warning** + + + { + "workspaceId": "ws-efdc5b42-b051-11ec-b123-4567ac120002", + "message": { + "log": "Too many aggregation groups, cannot create new group for alert, groups=1000, limit=1000, alert=sample-alert", + "level": "WARN" + }, + "component": "alertmanager" + } + +This means that the Alert manager Alert aggregation group size quota has been exceeded. + +**Action to take** + +Reduce the Alert aggregation group size by using the `group_by` parameter. For more information, see [Route-related settings in the](https://prometheus.io/docs/alerting/latest/configuration/) _Prometheus documentation_. + +You can also request a quota increase. Sign in to the AWS Management Console and open the Service Quotas console at [https://console.aws.amazon.com/servicequotas/](https://console.aws.amazon.com/servicequotas/). + +## Alerts size too big warning + +**When the log contains the following warning** + + + { + "workspaceId": "ws-efdc5b42-b051-11ec-b123-4567ac120002", + "message": { + "log": "alerts too big, total size limit: 20000000 bytes", + "level": "WARN" + }, + "component": "alertmanager" + } + +This means that Alert manager Alerts per workspace, in size quota has been exceeded. + +**Action to take** + +Remove unnecessary annotations and labels to reduce alert size. + +## Empty content warning @@ -56 +104 @@ Validate your Alert manager template and ensure that you have a valid template f - "log": "Subject has been modified because it contains control or non-ASCII characters." + "log": "Message has been modified because the content was empty." @@ -62 +110 @@ Validate your Alert manager template and ensure that you have a valid template f -This means that the subject has non-ASCII characters. +This means that the Alert manager template resolved the outbound alert to an empty message. @@ -66 +114 @@ This means that the subject has non-ASCII characters. -Remove references in subject field of your template to the labels that might contain non-ASCII characters. +Validate your Alert manager template and ensure that you have a valid template for all receiver pathways. @@ -127,0 +176,20 @@ Validate that the access policy on your Amazon SNS topic grants Amazon Managed S +## Non ASCII warning + +**When the log contains the following warning** + + + { + "workspaceId": "ws-abcd1234-ef56-78ab-cd90-1234abcd0000", + "message": { + "log": "Subject has been modified because it contains control or non-ASCII characters." + "level": "WARN" + }, + "component": "alertmanager" + } + +This means that the subject has non-ASCII characters. + +**Action to take** + +Remove references in subject field of your template to the labels that might contain non-ASCII characters. +