AWS glue documentation change
Summary
Removed documentation for 'bucket' and 'truncate' partitioning functions, simplified title, and updated examples to remove references to bucket/truncate strategies
Security assessment
Changes involve removing deprecated partitioning strategies but contain no references to security vulnerabilities or security-related fixes. The modifications appear to be routine documentation updates for feature simplification.
Diff
diff --git a/glue/latest/dg/zero-etl-data-partitioning.md b/glue/latest/dg/zero-etl-data-partitioning.md index aafc7562b..269db9d7f 100644 --- a//glue/latest/dg/zero-etl-data-partitioning.md +++ b//glue/latest/dg/zero-etl-data-partitioning.md @@ -7 +7 @@ What is data partitioning?Partition specification API referencePartitioning stra -# Data partitioning for Zero-ETL integrations +# Data partitioning @@ -38 +38 @@ Partition functions -Transformations applied to partition column values to create the actual partition boundaries. Examples include identity (using the raw value), time-based functions (year, month, day, hour), and bucketing. +Transformations applied to partition column values to create the actual partition boundaries. Examples include identity (using the raw value) and time-based functions (year, month, day, hour). @@ -87,10 +86,0 @@ An array of partition specifications that defines how data is partitioned in the - }, - { - "fieldName": "user_id", - "functionSpec": "bucket", - "bucketCount": 16 - }, - { - "fieldName": "product_code", - "functionSpec": "truncate", - "width": 4 @@ -121,4 +110,0 @@ Specifies the partitioning function. Valid values: - * `bucket` \- Distributes values into a specified number of buckets using hash-based partitioning. Requires a `bucketCount` parameter. - - * `truncate` \- Truncates string values to a specified width. Requires a `width` parameter. - @@ -130 +116 @@ Specifies the partitioning function. Valid values: -Time-based functions (`year`, `month`, `day`, `hour`) require the `ConversionSpec` parameter to specify the source timestamp format. The `bucket` and `truncate` functions require additional parameters as specified in their descriptions. +Time-based functions (`year`, `month`, `day`, `hour`) require the `ConversionSpec` parameter to specify the source timestamp format. @@ -146,36 +131,0 @@ A UTF-8 string that specifies the timestamp format of the source data. Valid val -BucketCount - - -An integer that specifies the number of buckets to create when using the `bucket` function. This parameter is required when `functionSpec` is set to `bucket`. The value must be a positive integer. Hash-based bucketing distributes values evenly across the specified number of buckets. - -###### Example Bucket partitioning example - - - { - "partitionSpec": [ - { - "fieldName": "user_id", - "functionSpec": "bucket", - "bucketCount": 16 - } - ] - } - -Width - - -An integer that specifies the truncation width when using the `truncate` function. This parameter is required when `functionSpec` is set to `truncate`. The value must be a positive integer representing the number of characters to retain from the beginning of string values. - -###### Example Truncate partitioning example - - - { - "partitionSpec": [ - { - "fieldName": "product_code", - "functionSpec": "truncate", - "width": 4 - } - ] - } - @@ -266,56 +215,0 @@ The original column values remain unchanged in your source data. AWS Glue only t -**Bucket partitioning** uses hash-based distribution to evenly spread data across a fixed number of buckets. This strategy is useful for high-cardinality columns where you want to control the number of partitions and ensure even data distribution. - -###### Example Bucket partitioning example - - - { - "partitionSpec": [ - { - "fieldName": "user_id", - "functionSpec": "bucket", - "bucketCount": 16 - } - ] - } - -This creates 16 buckets based on hash values of the "user_id" column, ensuring even distribution regardless of the actual user ID values. - -Bucket partitioning is particularly effective for: - - * High-cardinality columns like user IDs, session IDs, or primary keys - - * Scenarios where you need predictable partition counts - - * Workloads that benefit from parallel processing across fixed partition boundaries - - - - -**Truncate partitioning** creates partitions based on truncated string values, keeping only the first N characters. This strategy is useful for string columns where you want to group similar values together. - -###### Example Truncate partitioning example - - - { - "partitionSpec": [ - { - "fieldName": "product_code", - "functionSpec": "truncate", - "width": 4 - } - ] - } - -This creates partitions based on the first 4 characters of the "product_code" column. For example, "ELEC12345" and "ELEC67890" would be grouped in the same "ELEC" partition. - -Truncate partitioning is particularly effective for: - - * Product codes, SKUs, or other structured identifiers with meaningful prefixes - - * Geographic codes or postal codes where regional grouping is beneficial - - * Any string column where prefix-based grouping aligns with query patterns - - - - @@ -337,5 +230,0 @@ Truncate partitioning is particularly effective for: - }, - { - "fieldName": "user_id", - "functionSpec": "bucket", - "bucketCount": 8 @@ -346 +235 @@ Truncate partitioning is particularly effective for: -This creates a three-level partitioning scheme: first by month (from the "created_at" column), then by region, and finally by user ID buckets. This enables efficient queries that filter by date ranges, specific regions, user segments, or any combination of these dimensions. +This creates a two-level partitioning scheme: first by month (from the "created_at" column), then by region. This enables efficient queries that filter by date ranges, specific regions, or a combination of these dimensions. @@ -391,6 +279,0 @@ When designing multi-level partitioning schemes, consider: - * When using `bucket` partitioning, choose bucket counts that align with your query parallelism requirements. Common bucket counts are powers of 2 (8, 16, 32) for optimal hash distribution. - - * For `truncate` partitioning, select width values that create meaningful groupings based on your data patterns. Ensure the truncation width captures the significant prefix portion of your string values. - - * Avoid using `bucket` or `truncate` functions as the only partition strategy for very small datasets, as this may create unnecessary partition overhead. -