AWS glue documentation change
Summary
Added detailed documentation about Lake Formation FGAC migration requirements, logging behavior changes, and deprecated logging arguments in AWS Glue 5.0
Security assessment
The changes document security-related access control migration requirements (FGAC/FTA) but do not indicate remediation of a specific vulnerability. The logging changes are operational improvements without explicit security context. FGAC documentation addition qualifies as security feature documentation.
Diff
diff --git a/glue/latest/dg/migrating-version-50.md b/glue/latest/dg/migrating-version-50.md index a834145d8..20d139ca5 100644 --- a//glue/latest/dg/migrating-version-50.md +++ b//glue/latest/dg/migrating-version-50.md @@ -5 +5 @@ -New featuresActions to migrate to AWS Glue 5.0Migration checklistAWS Glue 5.0 featuresMigrating from AWS Glue 4.0 to AWS Glue 5.0Migrating from AWS Glue 3.0 to AWS Glue 5.0Migrating from AWS Glue 2.0 to AWS Glue 5.0Connector and JDBC driver migration for AWS Glue 5.0 +New featuresActions to migrate to AWS Glue 5.0Migration checklistAWS Glue 5.0 featuresMigrating from AWS Glue 4.0 to AWS Glue 5.0Migrating from AWS Glue 3.0 to AWS Glue 5.0Migrating from AWS Glue 2.0 to AWS Glue 5.0 Logging behavior changes in AWS Glue 5.0 Connector and JDBC driver migration for AWS Glue 5.0 @@ -28,0 +29,2 @@ To use this feature with your AWS Glue ETL jobs, choose `5.0` for the `Glue vers + * Logging behavior changes in AWS Glue 5.0 + @@ -84 +86 @@ Alternatively, you can provide the following job parameter (provide your DataZon - + * @@ -93 +95,17 @@ Note the following breaking changes: - * `GlueContext`-based table-level access control with AWS Lake Formation permissions supported in Glue 4.0 or before is not supported in Glue 5.0. Use the new Spark native fine-grained access control (FGAC) in Glue 5.0. Note the following details: + * Lake Formation Fine-grained Access Control (FGAC) + + * AWS Glue 5.0 only supports the new Spark-native FGAC using Spark DataFrames. It does not support FGAC using AWS Glue DynamicFrames. + + * Use of FGAC in 5.0 requires migrating from AWS Glue DynamicFrames to Spark DataFrames + + * If you don't need FGAC, then it is not necessary to migrate to Spark DataFrame and GlueContext features, like job bookmarks and push down predicates, will continue to work. + + * Jobs with Spark-native FGAC require a minimum of 4 workers: one user driver, one system driver, one system executor, and one standby user executor. + + * For more information, see [ Using AWS Glue with AWS Lake Formation for fine-grained access control](https://docs.aws.amazon.com/glue/latest/dg/security-lf-enable.html). + + * Lake Formation Full Table Access (FTA) + + * AWS Glue 5.0 supports FTA with Spark-native DataFrames (new) and GlueContext DynamicFrames (legacy, with limitations) + + * Spark-native FTA @@ -95 +113 @@ Note the following breaking changes: - * If you need fine grained access control (FGAC) for row/column/cell access control, you will need to migrate from `GlueContext`/Glue `DynamicFrame` in Glue 4.0 and prior to Spark `DataFrame` in Glue 5.0. + * If 4.0 script uses GlueContext, migrate to using native spark. @@ -97 +115 @@ Note the following breaking changes: - * If you need database/table level access control, you can grant database/table permissions to your roles. This bypasses the need to migrate from `GlueContext` to Spark dataframes. + * This feature is limited to hive and iceberg tables @@ -99 +117 @@ Note the following breaking changes: - * If you don't need FGAC, then no migration to Spark dataframe is necessary and `GlueContext` features like job bookmarks, push down predicates will continue to work. + * For more info on configuring a 5.0 job to use Spark native FTA, see @@ -101 +119 @@ Note the following breaking changes: - * Jobs with FGAC require a minimum of 4 workers: one user driver, one system driver, one system executor, and one standby user executor. + * GlueContext DynamicFrame FTA @@ -103 +121,3 @@ Note the following breaking changes: -For more information, see [Using AWS Glue with AWS Lake Formation for fine-grained access control](https://docs.aws.amazon.com/glue/latest/dg/security-lf-enable.html). + * No code change necessary + + * This feature is limited to non-OTF tables - it will not work with Iceberg, Delta Lake, and Hudi. @@ -115 +135 @@ For more information, see [Using AWS Glue with AWS Lake Formation for fine-grain - * Custom log4j properties are not supported in Glue 5.0. + * Custom log4j properties are not supported in AWS Glue 5.0. @@ -311,0 +332,47 @@ Also note the following migration differences between AWS Glue versions 3.0 and +## Logging behavior changes in AWS Glue 5.0 + +The following are changes in logging behavior in AWS Glue 5.0. For more information, see [Logging for AWS Glue jobs](https://docs.aws.amazon.com/glue/latest/dg/monitor-continuous-logging.html). + + * All logs (system logs, Spark daemon logs, user logs, and Glue Logger logs) are now written to the `/aws-glue/jobs/error` log group by default. + + * The `/aws-glue/jobs/logs-v2` log group used for continuous logging in previous versions is no longer used. + + * You can no longer rename or customize the log group or log stream names using the removed continuous logging arguments. Instead, see the new job arguments in AWS Glue 5.0. + + + + +### Two new job arguments are introduced in AWS Glue 5.0 + + * `––custom-logGroup-prefix`: Allows you to specify a custom prefix for the `/aws-glue/jobs/error` and `/aws-glue/jobs/output` log groups. + + * `––custom-logStream-prefix`: Allows you to specify a custom prefix for the log stream names within the log groups. + +Validation rules and limitations for custom prefixes include: + + * The entire log stream name must be between 1 and 512 characters. + + * The custom prefix for log stream names is limited to 400 characters. + + * Allowed characters in prefixes include alphanumeric characters, underscores (`_`), hyphens (`-`), and forward slashes (`/`). + + + + +### Deprecated continuous logging arguments in AWS Glue 5.0 + +The following job arguments for continuous logging have been deprecated in AWS Glue 5.0 + + * `––enable-continuous-cloudwatch-log` + + * `––continuous-log-logGroup` + + * `––continuous-log-logStreamPrefix` + + * `––continuous-log-conversionPattern` + + * `––enable-continuous-log-filter` + + + +