AWS glue documentation change
Summary
Restructured documentation sections, updated descriptions of Full Table Access (FTA) integration with AWS Lake Formation, and moved migration instructions. Enhanced clarity about FTA capabilities in AWS Glue 5.0 including Spark features and access control mechanisms.
Security assessment
The changes primarily improve documentation about security controls (Lake Formation integration) and access management patterns. While it describes security-related features like FTA and compliance with security regulations, there is no evidence of addressing a specific vulnerability or security incident. The updates focus on clarifying existing security capabilities rather than patching issues.
Diff
diff --git a/glue/latest/dg/security-access-control-fta.md b/glue/latest/dg/security-access-control-fta.md index 7c9cb1314..4824d2074 100644 --- a//glue/latest/dg/security-access-control-fta.md +++ b//glue/latest/dg/security-access-control-fta.md @@ -5 +5 @@ -Introduction to Full Table AccessHow Full-Table Access (FTA) works on AWS GlueImplementing Full Table AccessMigrating from AWS Glue 4.0 to AWS Glue 5.0 FTAUsage PatternsAdditional ConfigurationConsiderations and Limitations +Introduction to Full Table AccessHow Full-Table Access (FTA) works on AWS GlueImplementing Full Table AccessUsage PatternsAdditional ConfigurationMigrating from AWS Glue 4.0 to AWS Glue 5.0 FTAConsiderations and Limitations @@ -7 +7 @@ Introduction to Full Table AccessHow Full-Table Access (FTA) works on AWS GlueIm -# AWS Lake Formation with full table access +# Using AWS Glue with AWS Lake Formation for Full Table Access @@ -11 +11 @@ Introduction to Full Table AccessHow Full-Table Access (FTA) works on AWS GlueIm -With AWS Glue, you can leverage AWS Lake Formation with AWS Glue Data Catalog where the job runtime role has full table permissions. This capability allows you to read and write to tables that are protected by AWS Lake Formation from your AWS Glue Spark jobs and Interactive Sessions. +AWS Glue 5.0 supports Full Table Access (FTA) control in Apache Spark based on your policies defined in AWS Lake Formation. This feature enables read and write operations from your AWS Glue Spark jobs on AWS Lake Formation registered tables when the job role has full table access. FTA is ideal for use cases that need to comply with security regulations at the table level and supports Spark capabilities including Resilient Distributed Datasets (RDDs), custom libraries, and User Defined Functions (UDFs) with AWS Lake Formation tables. @@ -13,3 +13 @@ With AWS Glue, you can leverage AWS Lake Formation with AWS Glue Data Catalog wh -You can access AWS Lake Formation protected AWS Glue Data catalog tables from AWS Glue Spark jobs or Interactive Sessions where the job's runtime role has full table access. You do not have to enable AWS Lake Formation on the AWS Glue job. When a AWS Glue Spark job is configured for Full Table Access (FTA), AWS Lake Formation credentials are used to read/write Amazon S3 data for AWS Lake Formation registered tables, while the job's runtime role credentials will be used to read/write tables not registered with AWS Lake Formation. - -AWS Glue 5.0 supports Full-Table Access (FTA) control in Apache Spark based on your policies defined in AWS Lake Formation. This feature enables read and write operations from your AWS Glue 5.0 AWS Glue Spark jobs on AWS Lake Formation registered tables when the job role has full table access. This level of control is ideal for use cases that need to comply with security regulations at the table level. In addition, you can now use Spark capabilities including Resilient Distributed Datasets (RDDs), custom libraries, and User Defined Functions (UDFs) with AWS Lake Formation tables. This capability enables Data Manipulation Language (DML) operations including CREATE, ALTER, DELETE, UPDATE, and MERGE INTO statements on Apache Hive and Iceberg tables from within the same Apache Spark application. +When a AWS Glue Spark job is configured for Full Table Access (FTA), AWS Lake Formation credentials are used to read/write Amazon S3 data for AWS Lake Formation registered tables, while the job's runtime role credentials will be used to read/write tables not registered with AWS Lake Formation. This capability enables Data Manipulation Language (DML) operations including CREATE, ALTER, DELETE, UPDATE, and MERGE INTO statements on Apache Hive and Iceberg tables. @@ -25 +23 @@ AWS Lake Formation offers two approaches for data access control: Fine-Grained A -In AWS Glue 4.0, AWS Lake Formation based data access worked through GlueContext class, the utility class provided by AWS Glue. In AWS Glue 5.0, AWS Lake Formation based data access is available through native Spark SQL, Spark DataFrames, and continues to be supported through GlueContext class. On AWS Glue 5.0, before this launch, you needed to enable fine grained access mode per AWS Glue job or session through the `--enable-lakeformation-fine-grained-access` parameter to enable AWS Lake Formation based data access. This was needed even when you only needed full table access control instead of fine grained access control with specific column, row, and cell filters. +In AWS Glue 4.0, AWS Lake Formation based data access worked through GlueContext class, the utility class provided by AWS Glue. In AWS Glue 5.0, AWS Lake Formation based data access is available through native Spark SQL, Spark DataFrames, and continues to be supported through GlueContext class. @@ -168,24 +165,0 @@ Key configurations: -## Migrating from AWS Glue 4.0 to AWS Glue 5.0 FTA - -When migrating from AWS Glue 4.0 GlueContext FTA to AWS Glue 5.0 native Spark FTA: - - 1. Allow third-party query engines to access data without the IAM session tag validation in AWS Lake Formation. Follow Step 1: Enable Full Table Access in AWS Lake Formation. - - 2. You shouldn't need to change the job runtime role if you have AWS Glue 4.0 FTA working. The main IAM permission to verify is that the AWS Glue job execution role has lakeformation:GetDataAccess. - - 3. Modify spark session configurations in the script. Ensure the following spark configurations are present: - - --conf spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog - --conf spark.sql.catalog.spark_catalog.warehouse=s3://<bucket-name>/warehouse/ - --conf spark.sql.catalog.spark_catalog.client.region=<REGION> - --conf spark.sql.catalog.spark_catalog.glue.account-id=ACCOUNT_ID - --conf spark.sql.catalog.spark_catalog.glue.lakeformation-enabled=true - --conf spark.sql.catalog.dropDirectoryBeforeTable.enabled=true - - 4. Update script such that GlueContext DataFrames are changed to native spark DataFrames. - - 5. Update your AWS Glue job to use AWS Glue 5.0 - - - - @@ -401,0 +376,24 @@ Operations not listed above will continue to use IAM permissions to access table +## Migrating from AWS Glue 4.0 to AWS Glue 5.0 FTA + +When migrating from AWS Glue 4.0 GlueContext FTA to AWS Glue 5.0 native Spark FTA: + + 1. Allow third-party query engines to access data without the IAM session tag validation in AWS Lake Formation. Follow Step 1: Enable Full Table Access in AWS Lake Formation. + + 2. You do not need to change the job runtime role. However, verify that the AWS Glue job execution role has lakeformation:GetDataAccess IAM permission. + + 3. Modify spark session configurations in the script. Ensure the following spark configurations are present: + + --conf spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkSessionCatalog + --conf spark.sql.catalog.spark_catalog.warehouse=s3://<bucket-name>/warehouse/ + --conf spark.sql.catalog.spark_catalog.client.region=<REGION> + --conf spark.sql.catalog.spark_catalog.glue.account-id=ACCOUNT_ID + --conf spark.sql.catalog.spark_catalog.glue.lakeformation-enabled=true + --conf spark.sql.catalog.dropDirectoryBeforeTable.enabled=true + + 4. Update script such that GlueContext DataFrames are changed to native spark DataFrames. + + 5. Update your AWS Glue job to use AWS Glue 5.0 + + + +