AWS glue documentation change
Summary
Added detailed descriptions of compaction strategies (Binpack, Sort, Z-order) and updated CLI example to include compaction configuration parameters.
Security assessment
Changes focus on performance optimization strategies and configuration syntax. No security vulnerabilities, access controls, or encryption mechanisms are mentioned. The modifications are operational/feature documentation improvements.
Diff
diff --git a/glue/latest/dg/enable-compaction.md b/glue/latest/dg/enable-compaction.md index fc60038d7..97d0310cf 100644 --- a//glue/latest/dg/enable-compaction.md +++ b//glue/latest/dg/enable-compaction.md @@ -44 +44,13 @@ If you don't have an AWS Glue VPC connection set up already, create a new one by - 7. Choose **Enable optimization**. + 7. Choose a compaction strategy. The available options are: + + * **Binpack** – Binpack is the default compaction strategy in Apache Iceberg. It combines smaller data files into larger ones for optimal performance. + + * **Sort** – Sorting in Apache Iceberg is a data organization technique that clusters information within files based on specified columns, significantly improving query performance by reducing the number of files that need to be processed. You define the sort order in Iceberg's metadata using the sort-order field, and when multiple columns are specified, data is sorted in the sequence the columns appear in the sort order, ensuring records with similar values are stored together within files. The sorting compaction strategy takes the optimization further by sorting data across all files within a partition. + + * **Z-order** – Z-ordering is a way to organize data when you need to sort by multiple columns with equal importance. Unlike traditional sorting that prioritizes one column over others, Z-ordering gives balanced weight to each column, helping your query engine read fewer files when searching for data. + +The technique works by weaving together the binary digits of values from different columns. For example, if you have the numbers 3 and 4 from two columns, Z-ordering first converts them to binary (3 becomes 011 and 4 becomes 100), then interleaves these digits to create a new value: 011010. This interleaving creates a pattern that keeps related data physically close together. + +Z-ordering is particularly effective for multi-dimensional queries. For example, a customer table Z-ordered by income, state, and zip code can deliver superior performance compared to hierarchical sorting when querying across multiple dimensions. This organization allows queries targeting specific combinations of income and geographic location to quickly locate relevant data while minimizing unnecessary file scans. + + 8. Choose **Enable optimization**. @@ -52 +64,3 @@ AWS CLI -The following example shows how to enable compaction. Replace the account ID with a valid AWS account ID. Replace the database name and table name with actual Iceberg table name and the database name. Replace the `roleArn` with the AWS Resource Name (ARN) of the IAM role and name of the IAM role that has the required permissions to run compaction. +The following example shows how to enable compaction. Replace the account ID with a valid AWS account ID. Replace the database name and table name with actual Iceberg table name and the database name. Replace the `roleArn` with the AWS Resource Name (ARN) of the IAM role and name of the IAM role that has the required permissions to run compaction. You can replace compaction strategy `sort` with other supported strategies like `z-order` or `binpack`. + +order" depending on your requirements. @@ -59,2 +73,8 @@ The following example shows how to enable compaction. Replace the account ID wit - --table-optimizer-configuration '{"roleArn":"arn:aws:iam::123456789012:role/optimizer_role", "enabled":'true', - "vpcConfiguration":{"glueConnectionName":"glue_connection_name"}}' \ + --table-optimizer-configuration '{ + "roleArn": "arn:aws:iam::123456789012:role/optimizer_role", + "enabled": true, + "vpcConfiguration": {"glueConnectionName": "glue_connection_name"}, + "compactionConfiguration": { + "icebergConfiguration": {"strategy": "sort"} + } + }'\ @@ -69 +88 @@ Call [CreateTableOptimizer](https://docs.aws.amazon.com/glue/latest/dg/aws-glue- -After you enable compaction, **Table optimization** tab shows the following compaction details (after approximately 15-20 minutes): +After you enable compaction, **Table optimization** tab shows the following compaction details once the compaction run is complete: