AWS cli medium security documentation change
Summary
Added security-related connection options for DynamoDB ELT connector
Security assessment
Documents STS role ARN usage and cross-account access controls for secure data processing
Diff
diff --git a/cli/latest/reference/glue/create-job.md b/cli/latest/reference/glue/create-job.md index a3f190167..c06971423 100644 --- a//cli/latest/reference/glue/create-job.md +++ b//cli/latest/reference/glue/create-job.md @@ -15 +15 @@ - * [AWS CLI 2.27.63 Command Reference](../../index.html) » + * [AWS CLI 2.28.0 Command Reference](../../index.html) » @@ -400 +400 @@ JSON Syntax: -> * For the `G.4X` worker type, each worker maps to 4 DPU (16 vCPUs, 64 GB of memory) with 256GB disk, and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for Glue version 3.0 or later Spark ETL jobs in the following Amazon Web Services Regions: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), and Europe (Stockholm). +> * For the `G.4X` worker type, each worker maps to 4 DPU (16 vCPUs, 64 GB of memory) with 256GB disk, and provides 1 executor per worker. We recommend this worker type for jobs whose workloads contain your most demanding transforms, aggregations, joins, and queries. This worker type is available only for Glue version 3.0 or later Spark ETL jobs in the following Amazon Web Services Regions: US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Spain), Europe (Stockholm), and South America (São Paulo). @@ -479,0 +480,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -577,0 +582,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -629,0 +638,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -645,0 +658,32 @@ JSON Syntax: +>>> +>>> PartitionPredicate -> (string) +>>> +>>>> Partitions satisfying this predicate are deleted. Files within the retention period in these partitions are not deleted. +>>> +>>> OutputSchemas -> (list) +>>> +>>>> Specifies the data schema for the catalog source. +>>>> +>>>> (structure) +>>>> +>>>>> Specifies a user-defined schema when a schema cannot be determined by Glue. +>>>>> +>>>>> Columns -> (list) +>>>>> +>>>>>> Specifies the column definitions that make up a Glue schema. +>>>>>> +>>>>>> (structure) +>>>>>> +>>>>>>> Specifies a single column in a Glue schema definition. +>>>>>>> +>>>>>>> Name -> (string) +>>>>>>> +>>>>>>>> The name of the column in the Glue Studio schema. +>>>>>>> +>>>>>>> Type -> (string) +>>>>>>> +>>>>>>>> The hive type for this column in the Glue Studio schema. +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -822,92 +865,0 @@ JSON Syntax: ->> ->> S3ExcelSource -> (structure) ->> ->>> Defines configuration parameters for reading Excel files from Amazon S3. ->>> ->>> Name -> (string) ->>> ->>>> The name of the S3 Excel data source. ->>> ->>> Paths -> (list) ->>> ->>>> The S3 paths where the Excel files are located. ->>>> ->>>> (string) ->>> ->>> CompressionType -> (string) ->>> ->>>> The compression format used for the Excel files. ->>> ->>> Exclusions -> (list) ->>> ->>>> Patterns to exclude specific files or paths from processing. ->>>> ->>>> (string) ->>> ->>> GroupSize -> (string) ->>> ->>>> Defines the size of file groups for batch processing. ->>> ->>> GroupFiles -> (string) ->>> ->>>> Specifies how files should be grouped for processing. ->>> ->>> Recurse -> (boolean) ->>> ->>>> Indicates whether to recursively process subdirectories. ->>> ->>> MaxBand -> (integer) ->>> ->>>> The maximum number of processing bands to use. ->>> ->>> MaxFilesInBand -> (integer) ->>> ->>>> The maximum number of files to process in each band. ->>> ->>> AdditionalOptions -> (structure) ->>> ->>>> Additional configuration options for S3 direct source processing. ->>>> ->>>> BoundedSize -> (long) ->>>> ->>>>> Sets the upper limit for the target size of the dataset in bytes that will be processed. ->>>> ->>>> BoundedFiles -> (long) ->>>> ->>>>> Sets the upper limit for the target number of files that will be processed. ->>>> ->>>> EnableSamplePath -> (boolean) ->>>> ->>>>> Sets option to enable a sample path. ->>>> ->>>> SamplePath -> (string) ->>>> ->>>>> If enabled, specifies the sample path. ->>> ->>> NumberRows -> (long) ->>> ->>>> The number of rows to process from each Excel file. ->>> ->>> SkipFooter -> (integer) ->>> ->>>> The number of rows to skip at the end of each Excel file. ->>> ->>> OutputSchemas -> (list) ->>> ->>>> The AWS Glue schemas to apply to the processed data. ->>>> ->>>> (structure) ->>>> ->>>>> Specifies a user-defined schema when a schema cannot be determined by Glue. ->>>>> ->>>>> Columns -> (list) ->>>>> ->>>>>> Specifies the column definitions that make up a Glue schema. ->>>>>> ->>>>>> (structure) ->>>>>> ->>>>>>> Specifies a single column in a Glue schema definition. ->>>>>>> ->>>>>>> Name -> (string) ->>>>>>> ->>>>>>>> The name of the column in the Glue Studio schema. @@ -915 +867 @@ JSON Syntax: ->>>>>>> Type -> (string) +>>>>>>> GlueStudioType -> (string) @@ -917 +869 @@ JSON Syntax: ->>>>>>>> The hive type for this column in the Glue Studio schema. +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -1013,0 +966,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -1101,0 +1058,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -1133,0 +1094,16 @@ JSON Syntax: +>>> +>>> PitrEnabled -> (boolean) +>>> +>>>> Specifies whether Point-in-Time Recovery (PITR) is enabled for the DynamoDB table. When set to `true` , allows reading from a specific point in time. The default value is `false` . +>>> +>>> AdditionalOptions -> (structure) +>>> +>>>> Specifies additional connection options for the DynamoDB data source. +>>>> +>>>> DynamodbExport -> (string) +>>>> +>>>>> Specifies the DynamoDB export configuration for the ELT operation. +>>>> +>>>> DynamodbUnnestDDBJson -> (boolean) +>>>> +>>>>> Specifies whether to unnest DynamoDB JSON format. When set to `true` , nested JSON structures in DynamoDB items are flattened. @@ -1195,0 +1172,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -1253,0 +1234,4 @@ JSON Syntax: +>>>>>>> +>>>>>>> GlueStudioType -> (string) +>>>>>>> +>>>>>>>> The data type of the column as defined in Glue Studio. @@ -1373,0 +1358,12 @@ JSON Syntax: +>>> +>>> AutoDataQuality -> (structure) +>>> +>>>> Specifies whether to automatically enable data quality evaluation for the S3 catalog target. When set to `true` , data quality checks are performed automatically during the write operation. +>>>>