AWS glue documentation change
Summary
Added documentation for ConversionSpec parameter specifying timestamp formats and warnings about high-cardinality columns with identity partitioning
Security assessment
The changes provide operational guidance about timestamp conversion and partitioning best practices. While they mention performance impacts, there is no evidence of addressing security vulnerabilities or describing security features.
Diff
diff --git a/glue/latest/dg/aws-glue-api-integrations.md b/glue/latest/dg/aws-glue-api-integrations.md index 97cc4a0d8..e2312ca48 100644 --- a//glue/latest/dg/aws-glue-api-integrations.md +++ b//glue/latest/dg/aws-glue-api-integrations.md @@ -139,0 +140,24 @@ Specifies the function used to partition data on the target. The only accepted v + * `ConversionSpec` – UTF-8 string, not less than 1 or more than 128 bytes long. + +Specifies the timestamp format of the source data. Valid values are: + + * `epoch_sec` \- Unix epoch timestamp in seconds + + * `epoch_milli` \- Unix epoch timestamp in milliseconds + + * `iso` \- ISO 8601 formatted timestamp + +###### Note + +Only specify `ConversionSpec` when using timestamp-based partition functions (year, month, day, or hour). AWS Glue Zero-ETL uses this parameter to correctly transform source data into timestamp format before partitioning. + +Do not use high-cardinality columns with the `identity` partition function. High-cardinality columns include: + + * Primary keys + + * Timestamp fields (such as `LastModifiedTimestamp`, `CreatedDate`) + + * System-generated timestamps + +Using high-cardinality columns with identity partitioning creates many small partitions, which can significantly degrade ingestion performance. + @@ -864 +888 @@ This API is used to provide optional override properties for the the tables that -The connection ARN of the source, or the database ARN of the target. +The Amazon Resource Name (ARN) of the target table for which to create integration table properties. Currently, this API only supports creating integration table properties for target tables, and the provided ARN should be the ARN of the target table in the AWS Glue Data Catalog. Support for creating integration table properties for source connections (using the connection ARN) is not yet implemented and will be added in a future release. @@ -968 +992 @@ This API is used to retrieve optional override properties for the tables that ne -The connection ARN of the source, or the database ARN of the target. +The Amazon Resource Name (ARN) of the target table for which to retrieve integration table properties. Currently, this API only supports retrieving properties for target tables, and the provided ARN should be the ARN of the target table in the AWS Glue Data Catalog. Support for retrieving integration table properties for source connections (using the connection ARN) is not yet implemented and will be added in a future release. @@ -981 +1005 @@ The name of the table to be replicated. -The connection ARN of the source, or the database ARN of the target. +The Amazon Resource Name (ARN) of the target table for which to retrieve integration table properties. Currently, this API only supports retrieving properties for target tables, and the provided ARN should be the ARN of the target table in the AWS Glue Data Catalog. Support for retrieving integration table properties for source connections (using the connection ARN) is not yet implemented and will be added in a future release. @@ -1231 +1255 @@ Removes the specified tags from an integration resource. - * `ResourceArn` – _Required:_ UTF-8 string, not less than 1 or more than 10240 bytes long, matching the [Custom string pattern #50](./aws-glue-api-common.html#regex_50). + * `ResourceArn` – _Required:_ UTF-8 string, not less than 1 or more than 10240 bytes long, matching the [Custom string pattern #49](./aws-glue-api-common.html#regex_49). @@ -1262 +1286 @@ Lists the metadata tags assigned to the specified resource. - * `ResourceARN` – _Required:_ UTF-8 string, not less than 1 or more than 10240 bytes long, matching the [Custom string pattern #50](./aws-glue-api-common.html#regex_50). + * `ResourceARN` – _Required:_ UTF-8 string, not less than 1 or more than 10240 bytes long, matching the [Custom string pattern #49](./aws-glue-api-common.html#regex_49).