AWS emr documentation change
Summary
Added known issue about Spark UI port conflict with in-transit encryption and provided workaround configuration.
Security assessment
The change documents a functional bug (port conflict impacting monitoring) rather than a security vulnerability. While encryption context is security-related, the issue itself is operational.
Diff
diff --git a/emr/latest/ManagementGuide/emr-encryption-support-matrix.md b/emr/latest/ManagementGuide/emr-encryption-support-matrix.md index 0cdfc70d9..2065ce6fd 100644 --- a//emr/latest/ManagementGuide/emr-encryption-support-matrix.md +++ b//emr/latest/ManagementGuide/emr-encryption-support-matrix.md @@ -34,0 +35,26 @@ YARN NodeManager | spark.shuffle.service.port1 | 7337 | Spark AES-based encry +**Known issue** + +On intransit enabled clusters `spark.yarn.historyServer.address` configuration is currently using port `18080`, which prevents access to spark application UI using YARN tracking URL. **Affects Version:** EMR - 7.3.0 to EMR - 7.9.0. + +Use the following workaround: + + 1. Modify the `spark.yarn.historyServer.address` configuration in `/etc/spark/conf/spark-defaults.conf` to use the `HTTPS` port number `18480` on a running cluster. + + 2. This can also be provided in configuration overrides while launching the cluster. + + + + +Example configuration: + + + [ + { + "Classification": "spark-defaults", + "Properties": { + "spark.yarn.historyServer.address": "${hadoopconf-yarn.resourcemanager.hostname}:18480" + } + } + + ] +