AWS glue documentation change
Summary
Updated component versions (Spark 3.5.2→3.5.4, Iceberg 1.6.1→1.7.1, Delta Lake 3.2.1→3.3.0) and expanded FGAC support to Iceberg/Delta/Hudi tables
Security assessment
The FGAC expansion adds documentation about security controls for additional table formats, but version updates lack explicit security vulnerability references
Diff
diff --git a/glue/latest/dg/migrating-version-50.md b/glue/latest/dg/migrating-version-50.md index 01f9b1460..ee7597e27 100644 --- a/glue/latest/dg/migrating-version-50.md +++ b/glue/latest/dg/migrating-version-50.md @@ -38 +38 @@ This section describes new features and advantages of AWS Glue version 5.0. - * Apache Spark update from 3.3.0 in AWS Glue 4.0 to 3.5.2 in AWS Glue 5.0. See Major enhancements from Spark 3.3.0 to Spark 3.5.2. + * Apache Spark update from 3.3.0 in AWS Glue 4.0 to 3.5.4 in AWS Glue 5.0. See Major enhancements from Spark 3.3.0 to Spark 3.5.4. @@ -40 +40 @@ This section describes new features and advantages of AWS Glue version 5.0. - * Spark-native fine-grained access control (FGAC) using Lake Formation. For more information, see [Using AWS Glue with AWS Lake Formation for fine-grained access control](https://docs.aws.amazon.com/glue/latest/dg/security-lf-enable.html). + * Spark-native fine-grained access control (FGAC) using Lake Formation. This includes FGAC for Iceberg, Delta and Hudi tables. For more information, see [ Using AWS Glue with AWS Lake Formation for fine-grained access control](https://docs.aws.amazon.com/glue/latest/dg/security-lf-enable.html). @@ -52 +52 @@ For a complete list of limitations and considerations when using Spark-native FG - * Open Table Formats (OTF) updated to Hudi 0.15.0, Iceberg 1.6.1, and Delta Lake 3.2.1 + * Open Table Formats (OTF) updated to Hudi 0.15.0, Iceberg 1.7.1, and Delta Lake 3.3.0 @@ -118 +118 @@ For more information, see [Using AWS Glue with AWS Lake Formation for fine-grain -### Major enhancements from Spark 3.3.0 to Spark 3.5.2 +### Major enhancements from Spark 3.3.0 to Spark 3.5.4 @@ -197 +197 @@ For existing jobs, change the `Glue version` from the previous version to `Glue - * In AWS Glue Studio, choose `Glue 5.0 - Supports Spark 3.5.2, Scala 2, Python 3` in `Glue version`. + * In AWS Glue Studio, choose `Glue 5.0 - Supports Spark 3.5.4, Scala 2, Python 3` in `Glue version`. @@ -206 +206 @@ For new jobs, choose `Glue 5.0` when you create a job. - * In the console, choose `Spark 3.5.2, Python 3 (Glue Version 5.0) or Spark 3.5.2, Scala 2 (Glue Version 5.0)` in `Glue version`. + * In the console, choose `Spark 3.5.4, Python 3 (Glue Version 5.0) or Spark 3.5.4, Scala 2 (Glue Version 5.0)` in `Glue version`. @@ -208 +208 @@ For new jobs, choose `Glue 5.0` when you create a job. - * In AWS Glue Studio, choose `Glue 5.0 - Supports Spark 3.5.2, Scala 2, Python 3` in `Glue version`. + * In AWS Glue Studio, choose `Glue 5.0 - Supports Spark 3.5.4, Scala 2, Python 3` in `Glue version`. @@ -452,2 +452,2 @@ Java | 17 | 8 | 8 | 8 | 8 -Spark | 3.5.2-amzn-1 | 3.3.0-amzn-1 | 3.1.1-amzn-0 | 2.4.3 | 2.4.3 -Hadoop | 3.4.0-amzn-1 | 3.3.3-amzn-0 | 3.2.1-amzn-3 | 2.8.5-amzn-5 | 2.8.5-amzn-1 +Spark | 3.5.4 | 3.3.0-amzn-1 | 3.1.1-amzn-0 | 2.4.3 | 2.4.3 +Hadoop | 3.4.1 | 3.3.3-amzn-0 | 3.2.1-amzn-3 | 2.8.5-amzn-5 | 2.8.5-amzn-1 @@ -457 +457 @@ Hive | 2.3.9-amzn-4 | 2.3.9-amzn-2 | 2.3.7-amzn-4 | 1.2 | 1.2 -EMRFS | 2.66.0 | 2.54.0 | 2.46.0 | 2.38.0 | 2.30.0 +EMRFS | 2.69.0 | 2.54.0 | 2.46.0 | 2.38.0 | 2.30.0 @@ -460,2 +460,2 @@ Arrow | 12.0.1 | 7.0.0 | 2.0.0 | 0.10.0 | 0.10.0 -AWS Glue Data Catalog client | 4.2.0 | 3.7.0 | 3.0.0 | 1.10.0 | N/A -AWS SDK for Java | 2.28.8 | 1.12 | 1.12 | | +AWS Glue Data Catalog client | 4.5.0 | 3.7.0 | 3.0.0 | 1.10.0 | N/A +AWS SDK for Java | 2.29.52 | 1.12 | 1.12 | | @@ -487 +487 @@ EMR DynamoDB connector | 5.6.0 | 4.16.0 | -Amazon Redshift | 6.3.0 | 6.1.3 | +Amazon Redshift | 6.4.0 | 6.1.3 | @@ -503,2 +503,2 @@ Hudi | 0.15.0 | 0.12.1 | 0.10.1 -Delta Lake | 3.2.1 | 2.1.0 | 1.0.0 -Iceberg | 1.6.1 | 1.0.0 | 0.13.1 +Delta Lake | 3.3.0 | 2.1.0 | 1.0.0 +Iceberg | 1.7.1 | 1.0.0 | 0.13.1