AWS glue documentation change
Summary
Added guidance about avoiding high-cardinality columns for partitioning and clarified 'identity' partition function usage
Security assessment
Changes relate to performance optimization for data partitioning rather than security features or vulnerabilities
Diff
diff --git a/glue/latest/dg/aws-glue-api-integrations.md b/glue/latest/dg/aws-glue-api-integrations.md index de25c4b06..0c828cdf9 100644 --- a//glue/latest/dg/aws-glue-api-integrations.md +++ b//glue/latest/dg/aws-glue-api-integrations.md @@ -115 +115 @@ A structure that describes how data is partitioned on the target. -The field name used to partition data on the target. +The field name used to partition data on the target. Avoid using columns that have unique values for each row (for example, 'LastModifiedTimestamp', 'SystemModTimeStamp') as the partition column. These columns are not suitable for partitioning because they create a large number of small partitions, which can lead to performance issues. @@ -119 +119 @@ The field name used to partition data on the target. -Specifies a function used to partition data on the target. +Specifies the function used to partition data on the target. The only accepted value for this parameter is ''identity'' (string). The ''identity'' function ensures that the data partitioning on the target follows the same scheme as the source. In other words, the partitioning structure of the source data is preserved in the target destination.