AWS msk: Add CloudWatch metric guidance for MSK IAM connection quotas
Summary
Expands the MSK broker quota table entries for maximum IAM TCP connections and IAM connection creation rate with instructions to monitor them using the ClientConnectionCount and ConnectionCreationRate CloudWatch metrics (Sum statistic, one-minute period, broker-level aggregation).
Security assessment
The change is purely operational observability guidance: it tells customers how to watch IAM listener connection counts/creation rate so they can react before hitting limits that cause unavailability. It documents no security control, no vulnerability fix, and no credential/authorization behavior change, so it is only tangentially security-adjacent (availability monitoring).
Evidence
+Maximum TCP connections per broker ([IAM Access control](./iam-access-control.html)) | 3000 | To increase this limit, you can adjust the `listener.name.client_iam.max.connections` or the `listener.name.client_iam_public.max.connections` configuration property using the Kafka `AlterConfig` API or the `kafka-configs.sh` tool. It's important to note that increasing either property to a high value can result in unavailability. To monitor this quota, use the `ClientConnectionCount` CloudWatch metric with dimensions `Cluster Name`, `Broker ID`, and `Client Authentication` (set to `IAM`). Use the **Sum** statistic with a one-minute period. A single broker reports `ClientConnectionCount` as multiple data points across its network processors. Summing at the broker level aggregates across all network threads for the IAM listener to produce the total IAM connection count for that broker.
Diff
diff --git a/msk/latest/developerguide/limits.md b/msk/latest/developerguide/limits.md index 3ba3383e0..c932bec4a 100644 --- a//msk/latest/developerguide/limits.md +++ b//msk/latest/developerguide/limits.md @@ -73,2 +73,2 @@ Maximum storage per broker | 16384 GiB | -Maximum TCP connections per broker ([IAM Access control](./iam-access-control.html)) | 3000 | To increase this limit, you can adjust the `listener.name.client_iam.max.connections` or the `listener.name.client_iam_public.max.connections` configuration property using the Kafka `AlterConfig` API or the `kafka-configs.sh` tool. It's important to note that increasing either property to a high value can result in unavailability. -Maximum TCP connections rate per broker (IAM) | 100 per second (M5 and M7g instance sizes) 4 per second (t3 instance size) | To handle retries on failed connections, you can set the `reconnect.backoff.ms` configuration parameter on the client side. For example, if you want a client to retry connections after 1 second, set `reconnect.backoff.ms` to `1000`. For more information, see [reconnect.backoff.ms](https://kafka.apache.org/documentation/#producerconfigs_reconnect.backoff.ms) in the Apache Kafka documentation. +Maximum TCP connections per broker ([IAM Access control](./iam-access-control.html)) | 3000 | To increase this limit, you can adjust the `listener.name.client_iam.max.connections` or the `listener.name.client_iam_public.max.connections` configuration property using the Kafka `AlterConfig` API or the `kafka-configs.sh` tool. It's important to note that increasing either property to a high value can result in unavailability. To monitor this quota, use the `ClientConnectionCount` CloudWatch metric with dimensions `Cluster Name`, `Broker ID`, and `Client Authentication` (set to `IAM`). Use the **Sum** statistic with a one-minute period. A single broker reports `ClientConnectionCount` as multiple data points across its network processors. Summing at the broker level aggregates across all network threads for the IAM listener to produce the total IAM connection count for that broker. +Maximum TCP connections rate per broker (IAM) | 100 per second (M5 and M7g instance sizes) 4 per second (t3 instance size) | To handle retries on failed connections, you can set the `reconnect.backoff.ms` configuration parameter on the client side. For example, if you want a client to retry connections after 1 second, set `reconnect.backoff.ms` to `1000`. For more information, see [reconnect.backoff.ms](https://kafka.apache.org/documentation/#producerconfigs_reconnect.backoff.ms) in the Apache Kafka documentation. To monitor this quota, use the `ConnectionCreationRate` CloudWatch metric with dimensions `Cluster Name` and `Broker ID`. Use the **Sum** statistic with a one-minute period. A single broker reports `ConnectionCreationRate` as multiple data points across its network processors. Summing at the broker level aggregates across all network threads and client listeners to produce the total connection creation rate for that broker. @@ -93,2 +93,2 @@ Maximum TCP connections per broker (IAM Access control) | 3000 | To increase th -Setting these properties to a high value might result in cluster unavailability. -Maximum TCP connections rate per broker (IAM) | 100 per second | To handle retries on failed connections, you can set the `reconnect.backoff.ms` configuration parameter on the client side. For example, if you want a client to retry connections after 1 second, set `reconnect.backoff.ms` to `1000`. For more information, see [reconnect.backoff.ms](https://kafka.apache.org/documentation/#producerconfigs_reconnect.backoff.ms) in the Apache Kafka documentation. +Setting these properties to a high value might result in cluster unavailability. To monitor this quota, use the `ClientConnectionCount` CloudWatch metric with dimensions `Cluster Name`, `Broker ID`, and `Client Authentication` (set to `IAM`). Use the **Sum** statistic with a one-minute period. A single broker reports `ClientConnectionCount` as multiple data points across its network processors. Summing at the broker level aggregates across all network threads for the IAM listener to produce the total IAM connection count for that broker. +Maximum TCP connections rate per broker (IAM) | 100 per second | To handle retries on failed connections, you can set the `reconnect.backoff.ms` configuration parameter on the client side. For example, if you want a client to retry connections after 1 second, set `reconnect.backoff.ms` to `1000`. For more information, see [reconnect.backoff.ms](https://kafka.apache.org/documentation/#producerconfigs_reconnect.backoff.ms) in the Apache Kafka documentation. To monitor this quota, use the `ConnectionCreationRate` CloudWatch metric with dimensions `Cluster Name` and `Broker ID`. Use the **Sum** statistic with a one-minute period. A single broker reports `ConnectionCreationRate` as multiple data points across its network processors. Summing at the broker level aggregates across all network threads and client listeners to produce the total connection creation rate for that broker.