AWS managedservices high security documentation change
Summary
Updated monitoring metrics documentation with added security context for EC2 instances (mentioning malicious attacks as potential cause for high CPU) and OpenSearch domain (linking KMS key inaccessibility to encryption documentation). Restructured FSx sections and added new volume capacity metric.
Security assessment
EC2 metrics now explicitly mention 'malicious attacks' as a potential cause for high CPU utilization, indicating security monitoring. OpenSearch KMSKeyInaccessible alarm links to encryption at rest documentation, a security feature.
Diff
diff --git a/managedservices/latest/accelerate-guide/monitoring-default-metrics.md b/managedservices/latest/accelerate-guide/monitoring-default-metrics.md index 8941058f9..9fbe4aead 100644 --- a/managedservices/latest/accelerate-guide/monitoring-default-metrics.md +++ b/managedservices/latest/accelerate-guide/monitoring-default-metrics.md @@ -21 +21 @@ ALB instance | ApplicationLoadBalancerErrorCount (HTTPCode_ELB_5XX_Count/Request -RejectedConnectionCount sum > 0% for 1 min, 5 consecutive times. | CloudWatch alarm if the number of connections that were rejected because the load balancer reached its maximum +ALB instance | RejectedConnectionCount sum > 0% for 1 min, 5 consecutive times. | CloudWatch alarm if the number of connections that were rejected because the load balancer reached its maximum @@ -23,10 +23,8 @@ ALB target | TargetConnectionErrorCount (HTTPCode_Target_5XX_Count/RequestCount) -ApplicationLoadBalancerTargetGroupErrorCount sum > 0% for 1 min, 5 consecutive times. | CloudWatch alarm if number of connections were unsuccessfully established between the load balancer and the registered instances. -Aurora | Average CPU utilization > 90% for 20 mins, 5 consecutive times. | CloudWatch Alarm. -AWS Backup | DeleteRecoveryPoint An unexpected IAM role principal or IAM user principal has deleted an AWS Backup recovery point. | CloudWatch event. Emitted when a backup recovery point is deleted. -EC2 instance - all OSs | CPUUtilization***** > 95% for 5 mins, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. This is a Direct-Customer-Alerts alarm. -StatusCheckFailed > 0% for 5 minute , 3 consecutive times. -EC2 instance - Linux | Minimum mem_used_percent >= 95% for 5 minutes, 6 consecutive times. -Average swap_used_percent >= 95% for 5 minutes, 6 consecutive times. -Maximum disk_used_percent >= 95% for 5 minutes, 6 consecutive times. -EC2 instance - Windows | Minimum Memory % Committed Bytes in Use >= 95% for 5 minutes, 6 consecutive times. -Maximum LogicalDisk % Free Space <= 5% for 5 minutes, 6 consecutive times. +ALB target | ApplicationLoadBalancerTargetGroupErrorCount sum > 0% for 1 min, 5 consecutive times. | CloudWatch alarm if number of connections were unsuccessfully established between the load balancer and the registered instances. +EC2 instance - all OSs | CPUUtilization***** > 95% for 5 mins, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - all OSs | StatusCheckFailed > 0% for 5 minute , 3 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - Linux | Minimum mem_used_percent >= 95% for 5 minutes, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - Linux | Average swap_used_percent >= 95% for 5 minutes, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - Linux | Maximum disk_used_percent >= 95% for 5 minutes, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - Windows | Minimum Memory % Committed Bytes in Use >= 95% for 5 minutes, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. +EC2 instance - Windows | Maximum LogicalDisk % Free Space <= 5% for 5 minutes, 6 consecutive times. | CloudWatch alarm. High CPU utilization is an indicator of a change in application state such as deadlocks, infinite loops, malicious attacks, and other anomalies. These are Direct-Customer-Alerts alarms. @@ -34,3 +32,3 @@ EFS | AMSEFSBurstCreditBalanceExhausted. BurstCreditBalance less than 1000 for f -AMSEFSClientConnectionsLimit. ClientConnections > 24,000 for fifteen minutes. | CloudWatch alarm on the ClientConnections of the EFS file system. -AMSEFSThroughputUtilizationLimit. EFS Throughput Utilization > 80% for one hour. | CloudWatch alarm on the Throughput Utilization of the EFS file system. -AMSEFSPercentIOLimit. PercentIOLimit > 95 for seventy five minutes. | CloudWatch alarm on the PercentIOLimit of the EFS file system. +EFS | AMSEFSClientConnectionsLimit. ClientConnections > 24,000 for fifteen minutes. | CloudWatch alarm on the ClientConnections of the EFS file system. +EFS | AMSEFSThroughputUtilizationLimit. EFS Throughput Utilization > 80% for one hour. | CloudWatch alarm on the Throughput Utilization of the EFS file system. +EFS | AMSEFSPercentIOLimit. PercentIOLimit > 95 for seventy five minutes. | CloudWatch alarm on the PercentIOLimit of the EFS file system. @@ -39,7 +37,8 @@ ELB instance | SpilloverCountBackendConnectionErrors > 1 for 1 minute , 15 conse -HTTPCode_ELB_5XX_Count sum > 0 for 5 min, 3 consecutive times. | CloudWatch alarm on excess number of HTTP 5XX response codes that originate from the load balancer. -SurgeQueueLength > 100 for 1 minute, 15 consecutive times. | CloudWatch alarm if an excess number of requests are pending routing. -Amazon FSx ONTAP | AMSFSXONTAPThroughputUtilization. FSX:ONTAP IOPS Utilization > 80% for two hours. | CloudWatch alarm on the IOPS utilization limit of the FSX:ONTAP instance. -AMSFSXONTAPIOPSUtilization. FSX:ONTAP Throughput Utilization > 80% for two hours. | CloudWatch alarm on the volume capacity utilization limit of the FSX:ONTAP volume. -AMSFSXONTAPVolumeInodeUtilization. FSX:ONTAP Inode Utilization > 80% for two hours. | CloudWatch alarm on the file capacity utilization limit of the FSX:ONTAP volume. -Amazon FSx Windows | AMSFSXWindowsThroughputUtilization. FSX:Windows Throughput Utilization > 80% for two hours. | CloudWatch alarm on the IOPS utilization limit of the FSX:Windows instance. -AMSFSXWindowsIOPSUtilization. FSX:Windows IOPS Utilization > 80% for two hours. | CloudWatch alarm on the IOPS utilization limit of the FSX:Windows instance. +ELB instance | HTTPCode_ELB_5XX_Count sum > 0 for 5 min, 3 consecutive times. | CloudWatch alarm on excess number of HTTP 5XX response codes that originate from the load balancer. +ELB instance | SurgeQueueLength > 100 for 1 minute, 15 consecutive times. | CloudWatch alarm if an excess number of requests are pending routing. +FSx for ONTAP | AMSFSXONTAPIOPSUtilization. FSX:ONTAP IOPS Utilization > 80% for two hours. | CloudWatch alarm on the IOPS utilization limit of the FSX:ONTAP instance. +FSx for ONTAP | AMSFSXONTAPThroughputUtilization. FSX:ONTAP Throughput Utilization > 80% for two hours. | CloudWatch alarm on the throughput limit of the FSX:ONTAP volume. +FSx for ONTAP | AMSFSXONTAPVolumeInodeUtilization. FSX:ONTAP Inode Utilization > 80% for two hours. | CloudWatch alarm on the file capacity utilization limit of the FSX:ONTAP volume. +FSx for ONTAP | AMSFSXONTAPVolumeCapacityUtilization. FSX:ONTAP Volume Capacity Utilization > 80% for two hours. | CloudWatch alarm on the volume capacity utilization limit of the FSX:ONTAP volume. +Amazon FSx Windows | AMSFSXWindowsThroughputUtilization. FSX:Windows Throughput Utilization > 80% for two hours. | CloudWatch alarm on the throughput limit of the FSX:Windows instance. +Amazon FSx Windows | AMSFSXWindowsIOPSUtilization. FSX:Windows IOPS Utilization > 80% for two hours. | CloudWatch alarm on the IOPS utilization limit of the FSX:Windows instance. @@ -49 +48 @@ IAM | EC2 IAM Instance Profile does not exist. The instance profile is missing. -EC2 IAM Instance Profile has too many policies. The IAM instance profile has 10 policies and additional policies cannot be added. | +IAM | EC2 IAM Instance Profile has too many policies. The IAM instance profile has 10 policies and additional policies cannot be added. | @@ -58 +57 @@ NATGateways | PacketsDropCount : Alarm if packetsdropcount is > 0 over 15 minute -ErrorPortAllocation : Alarm if NAT Gateways could not allocate port for over 15 minutes evaluation period | The number of times the NAT gateway could not allocate a source port. A value greater than Zero indicates that too many concurrent connecations are open.. +NATGateways | ErrorPortAllocation : Alarm if NAT Gateways could not allocate port for over 15 minutes evaluation period | The number of times the NAT gateway could not allocate a source port. A value greater than Zero indicates that too many concurrent connecations are open.. @@ -61,9 +60,9 @@ OpenSearch domain | KMSKeyError >= 1 for 1 minute, 1 consecutive time. | CloudW -KMSKeyInaccessible >= 1 for 1 minute, 1 consecutive time. -ClusterStatus yellow maximum is >= 1 for 1 minute, 1 consecutive time. | At least one replica shard is not allocated to a node. To learn more, see [ Yellow Cluster Status](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status). -FreeStorageSpace minimum is <= 20480 for 1 minute, 1 consecutive time. | A node in your cluster is down to 20 GiB of free storage space. To learn more, see [ Lack of Available Storage Space](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-watermark). -ClusterIndexWritesBlocked >= 1 for 5 minutes, 1 consecutive time. | The cluster is blocking write requests. To learn more, see [ ClusterBlockException](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#troubleshooting-cluster-block). -Nodes minimum < x for 1 day, 1 consecutive time. | x is the number of nodes in your cluster. This alarm indicates that at least one node in your cluster has been unreachable for one day. To learn more, see [ Failed Cluster Nodes](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-failed-cluster-nodes). -CPUUtilization average >= 80% for 15 minutes, 3 consecutive times. | 100% CPU utilization isn't uncommon, but sustained high averages are problematic. Consider right-sizing an existing instance types or adding instances. -JVMMemoryPressure maximum >= 80% for 5 minutes, 3 consecutive times. | The cluster could encounter out of memory errors if usage increases. Consider scaling vertically. Amazon ES uses half of an instance's RAM for the Java heap, up to a heap size of 32 GiB. You can scale instances vertically up to 64 GiB of RAM, at which point you can scale horizontally by adding instances. -MasterCPUUtilization average >= 50% for 15 minutes, 3 consecutive times. | Consider using larger instance types for your [ dedicated master nodes](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-managedomains-dedicatedmasternodes.html). Because of their role in cluster stability and [ blue/green deployments](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-createupdatedomains.html#es-managedomains-configuration-changes), dedicated master nodes should have lower average CPU usage than data nodes. -MasterJVMMemoryPressure maximum >= 80% for 15 minutes, 1 consecutive time. +OpenSearch domain | KMSKeyInaccessible >= 1 for 1 minute, 1 consecutive time. | CloudWatch alarm. At least one primary shard and its replicas are not allocated to a node. To learn more, see [Encryption of Data at Rest for Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/encryption-at-rest.html). +OpenSearch domain | ClusterStatus yellow maximum is >= 1 for 1 minute, 1 consecutive time. | At least one replica shard is not allocated to a node. To learn more, see [Yellow Cluster Status](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status). +OpenSearch domain | FreeStorageSpace minimum is <= 20480 for 1 minute, 1 consecutive time. | A node in your cluster is down to 20 GiB of free storage space. To learn more, see [ Lack of Available Storage Space](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-watermark). +OpenSearch domain | ClusterIndexWritesBlocked >= 1 for 5 minutes, 1 consecutive time. | The cluster is blocking write requests. To learn more, see [ClusterBlockException](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#troubleshooting-cluster-block). +OpenSearch domain | Nodes minimum < x for 1 day, 1 consecutive time. | x is the number of nodes in your cluster. This alarm indicates that at least one node in your cluster has been unreachable for one day. To learn more, see [Failed Cluster Nodes](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-failed-cluster-nodes). +OpenSearch domain | CPUUtilization average >= 80% for 15 minutes, 3 consecutive times. | 100% CPU utilization isn't uncommon, but sustained high averages are problematic. Consider right-sizing an existing instance types or adding instances. +OpenSearch domain | JVMMemoryPressure maximum >= 80% for 5 minutes, 3 consecutive times. | The cluster could encounter out of memory errors if usage increases. Consider scaling vertically. Amazon ES uses half of an instance's RAM for the Java heap, up to a heap size of 32 GiB. You can scale instances vertically up to 64 GiB of RAM, at which point you can scale horizontally by adding instances. +OpenSearch domain | MasterCPUUtilization average >= 50% for 15 minutes, 3 consecutive times. | Consider using larger instance types for your [ dedicated master nodes](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-managedomains-dedicatedmasternodes.html). Because of their role in cluster stability and [blue/green deployments](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-createupdatedomains.html#es-managedomains-configuration-changes), dedicated master nodes should have lower average CPU usage than data nodes. +OpenSearch domain | MasterJVMMemoryPressure maximum >= 80% for 15 minutes, 1 consecutive time. | Consider using larger instance types for your [ dedicated master nodes](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-managedomains-dedicatedmasternodes.html). Because of their role in cluster stability and [blue/green deployments](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/es-createupdatedomains.html#es-managedomains-configuration-changes), dedicated master nodes should have lower average CPU usage than data nodes. @@ -72,2 +71,2 @@ RDS | Average CPU utilization > 90% for 15 mins, 2 consecutive times. | CloudWa -Sum of DiskQueueDepth > 75% for 1 mins, 15 consecutive times. -Average FreeStorageSpace < 1,073,741,824 bytes for 5 mins, 2 consecutive times. +Sum of DiskQueueDepth > 75% for 1 mins, 15 consecutive times. | CloudWatch alarms. +Average FreeStorageSpace < 1,073,741,824 bytes for 5 mins, 2 consecutive times. | CloudWatch alarms.