AWS emr medium security documentation change
Summary
Updated release notes from EMR 7.12.0 to 7.13.0, including new features (Python 3.11 default, Iceberg configuration property), application upgrades, and removal of known issues about missing logs after 3 hours.
Security assessment
The change documents the resolution of a bug where step/container logs were not uploaded to S3 for jobs running longer than 3 hours (from EMR 7.9 to 7.12). This is a security-related issue because missing logs can impact security auditing, incident investigation, and compliance monitoring. The fix in EMR 7.13.0 ensures proper log collection, which is critical for security visibility.
Diff
diff --git a/emr/latest/ReleaseGuide/emr-whatsnew.md b/emr/latest/ReleaseGuide/emr-whatsnew.md index 04312d1f3..8fe5d3a60 100644 --- a//emr/latest/ReleaseGuide/emr-whatsnew.md +++ b//emr/latest/ReleaseGuide/emr-whatsnew.md @@ -7 +7 @@ -Spark AgentsAmazon EMR 7.12.0Amazon EMR 6.15.0Amazon EMR 5.36.2SigV4 compatibility +Spark AgentsAmazon EMR 7.13.0Amazon EMR 6.15.0Amazon EMR 5.36.2SigV4 compatibility @@ -13 +13 @@ This page describes the changes and functionality available in the latest releas -These release notes are also available on the [Amazon EMR 7.12.0](./emr-7120-release.html), [Amazon EMR 6.15.0](./emr-6150-release.html), and [Amazon EMR 5.36.2](./emr-5362-release.html) pages, along with the application versions, component versions, and available configuration classifications for each release. +These release notes are also available on the [Amazon EMR 7.13.0](./emr-7130-release.html), [Amazon EMR 6.15.0](./emr-6150-release.html), and [Amazon EMR 5.36.2](./emr-5362-release.html) pages, along with the application versions, component versions, and available configuration classifications for each release. @@ -40 +40 @@ You can use the agent to troubleshoot PySpark and Scala applications failures. T -## Amazon EMR 7.12.0 (latest release of 7.x series) +## Amazon EMR 7.13.0 (latest release of 7.x series) @@ -44 +44 @@ New Amazon EMR releases are made available in different Regions over a period of -The following release notes include information for Amazon EMR release 7.11.0. +The following release notes include information for Amazon EMR release 7.13.0. @@ -46 +46 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **New features** +### What's new @@ -48 +48 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Application upgrades** – Amazon EMR 7.11.0 application upgrades include Delta 3.3.2-amzn-0, Flink 1.20.0-amzn-5, HBase 2.6.2-amzn-2, HCatalog 3.1.3-amzn-20, Hadoop 3.4.1-amzn-3, Hive 3.1.3-amzn-20, Hudi 1.0.2-amzn-0, Iceberg 1.9.1-amzn-0, Presto 0.287-amzn-5, Spark 3.5.6-amzn-0, TensorFlow 2.19.0, Tez 0.10.2-amzn-18, Trino 475-amzn-0, and ZooKeeper 3.9.3-amzn-3. + * **Python 3.11 default for PySpark and Spark workloads** — Python 3.11 is now the default Python version for PySpark and Spark workloads. Python 3.9 remains the default for all other applications. Both Python 3.9 and 3.11 are included in the release. @@ -50 +49,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * Amazon EMR on EC2 now supports IAM Identity Center User Background Sessions @@ -52 +50,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **User Background Sessions** : Enables long-running Spark workloads to continue running even after users log off from SageMaker Unified Studio, supporting sessions up to 90 days @@ -54 +51,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Flexible Background Session Configuration** : Two-level configuration (IAM Identity Center instance and Amazon EMR-EC2 cluster) with customizable background session duration from 15 minutes to 90 days (default: 7 days) @@ -56 +53 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Trusted Identity Propagation** : Maintains secure identity context throughout the background session lifecycle using Amazon EMR's trusted identity propagation feature +### Changes, enhancements, and resolved issues @@ -58 +55 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **SageMaker Unified Studio Integration** : Background sessions initiated through Livy interactive sessions in SageMaker Unified Studio + * **Iceberg configuration property** — Amazon EMR 7.13 adds a new Iceberg configuration property, `spark.sql.catalog.spark_catalog.route-non-iceberg-drop-to-session-catalog`. When set to `true`, `DROP TABLE` on non-Iceberg managed tables in `SparkSessionCatalog` deletes both the table metadata and the underlying Amazon S3 data. The default value is `false`. @@ -60 +56,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Long-running sessions with corporate identities** \- Amazon SageMaker Unified Studio now supports long-running sessions with corporate identities through IAM Identity Center's Trusted Identity Propagation (TIP). Users can launch interactive notebooks and data processing sessions on Amazon EMR and AWS Glue that persist using corporate credentials, even when logged off or sessions expire. Sessions run for up to 90 days (default 7 days) while maintaining identity permissions and consistent security controls. @@ -62 +57,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Known issues and limitations** @@ -64 +58,0 @@ The following release notes include information for Amazon EMR release 7.11.0. - * **Missing step and container logs** \- From Amazon EMR 7.9 to Amazon EMR 7.12, there is a bug where step logs and/or container logs may not get uploaded to S3 if the step or application is running for longer than 3 hours. Please upgrade to Amazon EMR 7.13 to resolve this issue. @@ -65,0 +60 @@ The following release notes include information for Amazon EMR release 7.11.0. +### Application upgrades @@ -66,0 +62 @@ The following release notes include information for Amazon EMR release 7.11.0. +The following applications are upgraded in this release: @@ -67,0 +64,22 @@ The following release notes include information for Amazon EMR release 7.11.0. + * HBase 2.6.4-amzn-0 (upgraded from 2.6.2-amzn-3) + + * Hadoop 3.4.2-amzn-0 (upgraded from 3.4.1-amzn-4) + + * Phoenix 5.3.0 (upgraded from 5.2.1) + + * Hudi 1.0.2-amzn-2 (upgraded from 1.0.2-amzn-1) + + * Trino 479-amzn-1 (upgraded from 476-amzn-1) + + * AWS SDK v2 2.42.12 (upgraded from 2.35.5) + + * AWS SDK v1 1.12.797 (upgraded from 1.12.792) + + * Spark 3.5.6-amzn-2, Hive 3.1.3-amzn-22, Tez 0.10.2-amzn-20, Presto 0.287-amzn-7, Iceberg 1.10.0-amzn-1, Delta 3.3.2-amzn-2, Flink 1.20.0-amzn-7, ZooKeeper 3.9.3-amzn-5 (amzn patch bumps) + + + + +### Known issues and limitations + +There are no known issues in this release.