AWS glue documentation change
Summary
Added documentation for new Iceberg table management structures and operations including schema definitions, partitioning specs, sort orders, and update operations. Expanded parameters for federated tables and table creation/update inputs.
Security assessment
The changes primarily add documentation for new Iceberg table format support and related configuration parameters. No security vulnerabilities, encryption settings, access controls, or authentication mechanisms are mentioned. The updates focus on data organization and table management features rather than security aspects.
Diff
diff --git a/glue/latest/dg/aws-glue-api-catalog-tables.md b/glue/latest/dg/aws-glue-api-catalog-tables.md index 42b4b4dc5..9080a1b1f 100644 --- a//glue/latest/dg/aws-glue-api-catalog-tables.md +++ b//glue/latest/dg/aws-glue-api-catalog-tables.md @@ -5 +5 @@ -— data types —TableTableInputFederatedTableColumnStorageDescriptorSchemaReferenceSerDeInfoOrderSkewedInfoTableVersionTableErrorTableVersionErrorSortCriterionTableIdentifierKeySchemaElementPartitionIndexPartitionIndexDescriptorBackfillErrorIcebergInputOpenTableFormatInputViewDefinitionViewDefinitionInputViewRepresentationViewRepresentationInput — operations —CreateTable (create_table)UpdateTable (update_table)DeleteTable (delete_table)BatchDeleteTable (batch_delete_table)GetTable (get_table)GetTables (get_tables)GetTableVersion (get_table_version)GetTableVersions (get_table_versions)DeleteTableVersion (delete_table_version)BatchDeleteTableVersion (batch_delete_table_version)SearchTables (search_tables)GetPartitionIndexes (get_partition_indexes)CreatePartitionIndex (create_partition_index)DeletePartitionIndex (delete_partition_index)GetColumnStatisticsForTable (get_column_statistics_for_table)UpdateColumnStatisticsForTable (update_column_statistics_for_table)DeleteColumnStatisticsForTable (delete_column_statistics_for_table) +— data types —TableTableInputFederatedTableColumnStorageDescriptorSchemaReferenceSerDeInfoOrderSkewedInfoTableVersionTableErrorTableVersionErrorSortCriterionTableIdentifierKeySchemaElementPartitionIndexPartitionIndexDescriptorBackfillErrorIcebergInputOpenTableFormatInputViewDefinitionViewDefinitionInputViewRepresentationViewRepresentationInputUpdateOpenTableFormatInputUpdateIcebergInputCreateIcebergTableInputUpdateIcebergTableInputIcebergSortOrderIcebergSortFieldIcebergPartitionSpecIcebergPartitionFieldIcebergSchemaIcebergStructFieldIcebergTableUpdate — operations —CreateTable (create_table)UpdateTable (update_table)DeleteTable (delete_table)BatchDeleteTable (batch_delete_table)GetTable (get_table)GetTables (get_tables)GetTableVersion (get_table_version)GetTableVersions (get_table_versions)DeleteTableVersion (delete_table_version)BatchDeleteTableVersion (batch_delete_table_version)SearchTables (search_tables)GetPartitionIndexes (get_partition_indexes)CreatePartitionIndex (create_partition_index)DeletePartitionIndex (delete_partition_index)GetColumnStatisticsForTable (get_column_statistics_for_table)UpdateColumnStatisticsForTable (update_column_statistics_for_table)DeleteColumnStatisticsForTable (delete_column_statistics_for_table) @@ -60,0 +61,22 @@ The Table API describes data types and operations associated with tables. + * UpdateOpenTableFormatInput structure + + * UpdateIcebergInput structure + + * CreateIcebergTableInput structure + + * UpdateIcebergTableInput structure + + * IcebergSortOrder structure + + * IcebergSortField structure + + * IcebergPartitionSpec structure + + * IcebergPartitionField structure + + * IcebergSchema structure + + * IcebergStructField structure + + * IcebergTableUpdate structure + @@ -287,0 +310,4 @@ The name of the connection to the external metastore. + * `ConnectionType` – UTF-8 string, not less than 1 or more than 255 bytes long, matching the [Single-line string pattern](./aws-glue-api-common.html#aws-glue-api-regex-oneLine). + +The type of connection used to access the federated table, specifying the protocol or method for connecting to the external data source. + @@ -690,0 +717,4 @@ The table version for the Iceberg table. Defaults to 2. + * `CreateIcebergTableInput` – A CreateIcebergTableInput object. + +The configuration parameters required to create a new Iceberg table in the AWS Glue Data Catalog, including table properties and metadata specifications. + @@ -820,0 +851,243 @@ A string that represents the SQL query that describes the view with expanded res +## UpdateOpenTableFormatInput structure + +Input parameters for updating open table format tables in AWS GlueData Catalog, serving as a wrapper for format-specific update operations such as Apache Iceberg. + +###### Fields + + * `UpdateIcebergInput` – An UpdateIcebergInput object. + +Apache Iceberg-specific update parameters that define the table modifications to be applied, including schema changes, partition specifications, and table properties. + + + + +## UpdateIcebergInput structure + +Input parameters specific to updating Apache Iceberg tables in AWS Glue Data Catalog, containing the update operations to be applied to an existing Iceberg table. + +###### Fields + + * `UpdateIcebergTableInput` – _Required:_ An UpdateIcebergTableInput object. + +The specific update operations to be applied to the Iceberg table, containing a list of updates that define the new state of the table including schema, partitions, and properties. + + + + +## CreateIcebergTableInput structure + +The configuration parameters required to create a new Iceberg table in the AWS Glue Data Catalog, including table properties and metadata specifications. + +###### Fields + + * `Location` – _Required:_ Location string, not more than 2056 bytes long, matching the [URI address multi-line string pattern](./aws-glue-api-common.html#aws-glue-api-regex-uri). + +The S3 location where the Iceberg table data will be stored. + + * `Schema` – _Required:_ An IcebergSchema object. + +The schema definition that specifies the structure, field types, and metadata for the Iceberg table. + + * `PartitionSpec` – An IcebergPartitionSpec object. + +The partitioning specification that defines how the Iceberg table data will be organized and partitioned for optimal query performance. + + * `WriteOrder` – An IcebergSortOrder object. + +The sort order specification that defines how data should be ordered within each partition to optimize query performance. + + * `Properties` – A map array of key-value pairs. + +Each key is a UTF-8 string. + +Each value is a UTF-8 string. + +Key-value pairs of additional table properties and configuration settings for the Iceberg table. + + + + +## UpdateIcebergTableInput structure + +Contains the update operations to be applied to an existing Iceberg table inAWS Glue Data Catalog, defining the new state of the table metadata. + +###### Fields + + * `Updates` – _Required:_ An array of IcebergTableUpdate objects. + +The list of table update operations that specify the changes to be made to the Iceberg table, including schema modifications, partition specifications, and table properties. + + + + +## IcebergSortOrder structure + +Defines the sort order specification for an Iceberg table, determining how data should be ordered within partitions to optimize query performance. + +###### Fields + + * `OrderId` – _Required:_ Number (integer). + +The unique identifier for this sort order specification within the Iceberg table's metadata. + + * `Fields` – _Required:_ An array of IcebergSortField objects. + +The list of fields and their sort directions that define the ordering criteria for the Iceberg table data. + + + + +## IcebergSortField structure + +Defines a single field within an Iceberg sort order specification, including the source field, transformation, sort direction, and null value ordering. + +###### Fields + + * `SourceId` – _Required:_ Number (integer). + +The identifier of the source field from the table schema that this sort field is based on. + + * `Transform` – _Required:_ UTF-8 string. + +The transformation function applied to the source field before sorting, such as identity, bucket, or truncate. + + * `Direction` – _Required:_ UTF-8 string (valid values: `asc="ASC"` | `desc="DESC"`). + +The sort direction for this field, either ascending or descending. + + * `NullOrder` – _Required:_ UTF-8 string (valid values: `nulls-first="NULLS_FIRST"` | `nulls-last="NULLS_LAST"`). + +The ordering behavior for null values in this field, specifying whether nulls should appear first or last in the sort order. + + + + +## IcebergPartitionSpec structure + +Defines the partitioning specification for an Iceberg table, determining how table data will be organized and partitioned for optimal query performance. + +###### Fields + + * `Fields` – _Required:_ An array of IcebergPartitionField objects. + +The list of partition fields that define how the table data should be partitioned, including source fields and their transformations. + + * `SpecId` – Number (integer). + +The unique identifier for this partition specification within the Iceberg table's metadata history. + + + + +## IcebergPartitionField structure + +Defines a single partition field within an Iceberg partition specification, including the source field, transformation function, partition name, and unique identifier. + +###### Fields + + * `SourceId` – _Required:_ Number (integer). + +The identifier of the source field from the table schema that this partition field is based on. + + * `Transform` – _Required:_ UTF-8 string. + +The transformation function applied to the source field to create the partition, such as identity, bucket, truncate, year, month, day, or hour. + + * `Name` – _Required:_ UTF-8 string, not less than 1 or more than 1024 bytes long, matching the [Single-line string pattern](./aws-glue-api-common.html#aws-glue-api-regex-oneLine). + +The name of the partition field as it will appear in the partitioned table structure. + + * `FieldId` – Number (integer). + +The unique identifier assigned to this partition field within the Iceberg table's partition specification. + + + + +## IcebergSchema structure + +Defines the schema structure for an Iceberg table, including field definitions, data types, and schema metadata.