AWS glue documentation change
Summary
Added documentation for CompactionConfiguration and IcebergCompactionConfiguration structures, including compaction strategies (binpack/sort/z-order) and their operational impacts
Security assessment
The changes focus on performance optimization strategies for Iceberg table compaction (file size management, query optimization) without mentioning security controls, vulnerability mitigation, or access management. While compaction strategies improve operational efficiency, there's no evidence of addressing security vulnerabilities or adding security features.
Diff
diff --git a/glue/latest/dg/aws-glue-api-table-optimizers.md b/glue/latest/dg/aws-glue-api-table-optimizers.md index 84c10cb2f..4506a0d52 100644 --- a//glue/latest/dg/aws-glue-api-table-optimizers.md +++ b//glue/latest/dg/aws-glue-api-table-optimizers.md @@ -5 +5 @@ -— data types —TableOptimizerTableOptimizerConfigurationTableOptimizerVpcConfigurationTableOptimizerRunBatchGetTableOptimizerEntryBatchTableOptimizerBatchGetTableOptimizerErrorRetentionConfigurationIcebergRetentionConfigurationOrphanFileDeletionConfigurationIcebergOrphanFileDeletionConfigurationCompactionMetricsRetentionMetricsOrphanFileDeletionMetricsIcebergCompactionMetricsIcebergRetentionMetricsIcebergOrphanFileDeletionMetricsRunMetrics — operations —GetTableOptimizer (get_table_optimizer)BatchGetTableOptimizer (batch_get_table_optimizer)ListTableOptimizerRuns (list_table_optimizer_runs)CreateTableOptimizer (create_table_optimizer)DeleteTableOptimizer (delete_table_optimizer)UpdateTableOptimizer (update_table_optimizer) +— data types —TableOptimizerTableOptimizerConfigurationTableOptimizerVpcConfigurationCompactionConfigurationIcebergCompactionConfigurationTableOptimizerRunBatchGetTableOptimizerEntryBatchTableOptimizerBatchGetTableOptimizerErrorRetentionConfigurationIcebergRetentionConfigurationOrphanFileDeletionConfigurationIcebergOrphanFileDeletionConfigurationCompactionMetricsRetentionMetricsOrphanFileDeletionMetricsIcebergCompactionMetricsIcebergRetentionMetricsIcebergOrphanFileDeletionMetricsRunMetrics — operations —GetTableOptimizer (get_table_optimizer)BatchGetTableOptimizer (batch_get_table_optimizer)ListTableOptimizerRuns (list_table_optimizer_runs)CreateTableOptimizer (create_table_optimizer)DeleteTableOptimizer (delete_table_optimizer)UpdateTableOptimizer (update_table_optimizer) @@ -18,0 +19,4 @@ The table optimizer API describes the AWS Glue API for enabling compaction to im + * CompactionConfiguration structure + + * IcebergCompactionConfiguration structure + @@ -98,0 +103,4 @@ This configuration is necessary to perform optimization on tables that are in a + * `compactionConfiguration` – A CompactionConfiguration object. + +The configuration for a compaction optimizer. This configuration defines how data files in your table will be compacted to improve query performance and reduce storage costs. + @@ -124,0 +133,34 @@ The name of the AWS Glue connection used for the VPC for the table optimizer. +## CompactionConfiguration structure + +The configuration for a compaction optimizer. This configuration defines how data files in your table will be compacted to improve query performance and reduce storage costs. + +###### Fields + + * `icebergConfiguration` – An IcebergCompactionConfiguration object. + +The configuration for an Iceberg compaction optimizer. + + + + +## IcebergCompactionConfiguration structure + +The configuration for an Iceberg compaction optimizer. This configuration defines parameters for optimizing the layout of data files in Iceberg tables. + +###### Fields + + * `strategy` – UTF-8 string (valid values: `binpack="BINPACK"` | `sort="SORT"` | `z-order="ZORDER"`). + +The strategy to use for compaction. Valid values are: + + * `binpack`: Combines small files into larger files, typically targeting sizes over 100MB, while applying any pending deletes. This is the recommended compaction strategy for most use cases. + + * `sort`: Organizes data based on specified columns which are sorted hierarchically during compaction, improving query performance for filtered operations. This strategy is recommended when your queries frequently filter on specific columns. To use this strategy, you must first define a sort order in your Iceberg table properties using the `sort_order` table property. + + * `z-order`: Optimizes data organization by blending multiple attributes into a single scalar value that can be used for sorting, allowing efficient querying across multiple dimensions. This strategy is recommended when you need to query data across multiple dimensions simultaneously. To use this strategy, you must first define a sort order in your Iceberg table properties using the `sort_order` table property. + +If an input is not provided, the default value 'binpack' will be used. + + + + @@ -156,0 +199,10 @@ A `CompactionMetrics` object containing metrics for the optimizer run. + * `compactionStrategy` – UTF-8 string (valid values: `binpack="BINPACK"` | `sort="SORT"` | `z-order="ZORDER"`). + +The strategy used for the compaction run. Indicates which algorithm was applied to determine how files were selected and combined during the compaction process. Valid values are: + + * `binpack`: Combines small files into larger files, typically targeting sizes over 100MB, while applying any pending deletes. This is the recommended compaction strategy for most use cases. + + * `sort`: Organizes data based on specified columns which are sorted hierarchically during compaction, improving query performance for filtered operations. This strategy is recommended when your queries frequently filter on specific columns. To use this strategy, you must first define a sort order in your Iceberg table properties using the `sort_order` table property. + + * `z-order`: Optimizes data organization by blending multiple attributes into a single scalar value that can be used for sorting, allowing efficient querying across multiple dimensions. This strategy is recommended when you need to query data across multiple dimensions simultaneously. To use this strategy, you must first define a sort order in your Iceberg table properties using the `sort_order` table property. +