AWS glue medium security documentation change
Summary
Restructured documentation for zero-ETL integration targets with expanded configuration details for S3 Tables, Amazon Redshift managed storage, and SageMaker Lakehouse. Added IAM policy examples and security configuration requirements.
Security assessment
Added explicit IAM policy requirements including glue:AuthorizeInboundIntegration and cross-account permissions. Includes security-critical notes about avoiding explicit DENY statements in bucket policies and mandatory condition keys for source ARN validation. These changes directly address access control and permission delegation security aspects.
Diff
diff --git a/glue/latest/dg/zero-etl-target.md b/glue/latest/dg/zero-etl-target.md index 7d59e6041..a7034ab90 100644 --- a//glue/latest/dg/zero-etl-target.md +++ b//glue/latest/dg/zero-etl-target.md @@ -5 +5 @@ -Configuring the integration target +SageMaker Lakehouse with S3 storageS3 Tables targetSageMaker Lakehouse with Amazon Redshift storageAmazon Redshift data warehouse targetConfiguring the integration target @@ -15 +15 @@ The configuration options for a target in a zero-ETL integration include: - * An Amazon SageMaker Lakehouse catalog and database configured with Amazon S3 storage. See [Setting up an AWS Glue database](./zero-etl-prerequisites.html#zero-etl-setup-target-resources-glue-database). + * An Amazon SageMaker Lakehouse catalog and database configured with regular Amazon S3 storage. See Configuring an Amazon SageMaker Lakehouse catalog with regular S3 storage. @@ -17 +17 @@ The configuration options for a target in a zero-ETL integration include: - * An Amazon SageMaker Lakehouse catalog configured with Amazon Redshift managed storage. See Configuring the integration with your target. + * An Amazon SageMaker Lakehouse catalog configured with Amazon S3 Tables bucket. See Configuring Amazon S3 tables as a target. @@ -19 +19,3 @@ The configuration options for a target in a zero-ETL integration include: - * An Amazon Redshift data warehouse identified by a Redshift namespace. See Configuring the integration with your target. + * An Amazon SageMaker Lakehouse catalog configured with Amazon Redshift managed storage. See Configuring an Amazon SageMaker Lakehouse catalog with Amazon Redshift managed storage. + + * An Amazon Redshift data warehouse identified by a Redshift namespace. See Configuring an Amazon Redshift data warehouse target. @@ -28 +30 @@ You cannot modify the target of a zero-ETL integration after creation. -## Configuring the integration with your target +## Configuring an Amazon SageMaker Lakehouse catalog with regular S3 storage @@ -30 +32 @@ You cannot modify the target of a zero-ETL integration after creation. -After you have selected your connection and specified a source IAM role, follow these steps when specifying an Amazon Redshift data warehouse target: +This section describes the prerequisites and setup steps for configuring a regular Amazon S3 bucket as storage for your Amazon SageMaker Lakehouse catalog target in a zero-ETL integration. @@ -32 +34 @@ After you have selected your connection and specified a source IAM role, follow - 1. Specify the namespace of the Redshift cluster or Redshift Serverless workgroup, or **Create a new namespace**. +### Prerequisites for setting up an integration @@ -34 +36 @@ After you have selected your connection and specified a source IAM role, follow - 2. Select the AWS Glue **Fix it for me** option. For the Redshift target, this will: +Before creating a zero-ETL integration with an Amazon SageMaker Lakehouse catalog using regular S3 storage, you need to complete the following setup tasks: @@ -36 +38 @@ After you have selected your connection and specified a source IAM role, follow - * Apply an authorized service principal on the Redshift cluster or Serverless workgroup. + 1. Set up an AWS Glue database @@ -38 +40 @@ After you have selected your connection and specified a source IAM role, follow - * Apply an authorized glue source ARN to the Redshift cluster or Serverless workgroup. + 2. Provide Catalog RBAC policy @@ -40 +42 @@ After you have selected your connection and specified a source IAM role, follow - * Associate a new parameter group with `enable_case_sensitive_identifier = true`. + 3. Create target IAM role @@ -42 +43,0 @@ After you have selected your connection and specified a source IAM role, follow - @@ -44 +44,0 @@ After you have selected your connection and specified a source IAM role, follow - 3. Provide the integration name and choose **Create and Launch Integration**. @@ -46 +45,0 @@ After you have selected your connection and specified a source IAM role, follow - 4. Once your integration is in the active state, navigate to the integration details page and choose **Create a database from integration**. @@ -48 +47,15 @@ After you have selected your connection and specified a source IAM role, follow - 5. Finally, you can navigate to the Redshift query editor, and connect to your database to validate the snapshot and incremental data. +After configuring the Amazon SageMaker Lakehouse catalog with regular Amazon S3 storage, you can proceed to Configuring the integration with your target to complete the integration setup. + +## Configuring Amazon S3 tables as a target + +This section describes the prerequisites and setup steps for configuring Amazon S3 Tables as a target for your zero-ETL integration. + +### Prerequisites for setting up an integration + +Before creating a zero-ETL integration with Amazon S3 Tables as a target, you need to complete the following setup tasks: + + 1. Setup Amazon S3 tables bucket + + 2. Provide Catalog RBAC policy + + 3. Create target IAM role @@ -52,0 +66,15 @@ After you have selected your connection and specified a source IAM role, follow +### Setup Amazon S3 tables bucket + + 1. Create an S3 table bucket in your account by following the instructions at [Getting started with Amazon S3 Tables](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-getting-started.html). + + 2. Enable Analytics integrations with your S3-Table bucket by following these instructions: [Integrating AWS services with Amazon S3 Tables](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-tables-integrating-aws.html#table-integration-procedures). + + + + +### Provide Catalog RBAC Policy + +The following permissions must be added to the Catalog RBAC Policy to allow for integrations between source and Amazon S3 tables catalog target. + +Target AWS Glue Catalog resource policy needs to include Glue Service permissions to AuthorizeInboundIntegration. Additionally, CreateInboundIntegration permission is required either on the source principal creating the Integration or in the target AWS Glue resource policy. + @@ -55 +83,48 @@ After you have selected your connection and specified a source IAM role, follow -You can only use lowercase alphanumeric characters and underscores in the namespace or catalog name. This is different from what the AWS Glue Data Catalog allows to create a database with any name (including special characters). +For cross-account scenario, both source principal as well as target AWS Glue Catalog resource policy need to include glue:CreateInboundIntegration permissions on the resource. + + + { + "Version": "2012-10-17", + "Statement": [ + { + // Optional for same account but mandatory for cross account scenarios + // Allow Alice to create Integration on Target Catalog + "Principal": { + "AWS": [ + "arn:aws:iam::<source-account-id>:user/Alice" + ] + }, + "Effect": "Allow", + "Action": [ + "glue:CreateInboundIntegration" + ], + "Resource": [ + "arn:aws:glue:<region>:<Target-Account-Id>:catalog/<s3tablescatalog>/*" + ], + "Condition": { + "StringLike": { + "aws:SourceArn": "arn:aws:dynamodb:<region>:<Account>:table/<table-name>" + } + } + }, + { // Required: Allow Glue to Authorize the Inbound Integration on behalf of Bob + "Principal": { + "Service": [ + "glue.amazonaws.com" + ] + }, + "Effect": "Allow", + "Action": [ + "glue:AuthorizeInboundIntegration" + ], + "Resource": [ + "arn:aws:glue:<region>:<Target-Account-Id>:catalog/<s3tablescatalog>/*" + ], + "Condition": { + "StringEquals": { + "aws:SourceArn": "arn:aws:dynamodb:<region>:<account-id>:table/<table-name>" + } + } + } + ] + } @@ -57 +132 @@ You can only use lowercase alphanumeric characters and underscores in the namesp -Follow these steps when specifying an Amazon SageMaker Lakehouse catalog and database configured with an Amazon S3 storage target: +###### Note @@ -59 +134,71 @@ Follow these steps when specifying an Amazon SageMaker Lakehouse catalog and dat - 1. Register an integration from Redshift into the catalog in Lake Formation. See [Registering Amazon Redshift clusters and namespaces to the AWS Glue Data Catalog](https://docs.aws.amazon.com/redshift/latest/dg/iceberg-integration-register.html). +Replace `<s3tablescatalog>` with the catalog name of your S3 tables. + +### Create target IAM Role + +Create a target IAM role with the following permissions and trust relationships: + +Example IAM policy: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Action": [ + "s3tables:ListTableBuckets", + "s3tables:GetTableBucket", + "s3tables:GetTableBucketEncryption", + "s3tables:GetNamespace", + "s3tables:CreateNamespace", + "s3tables:ListNamespaces", + "s3tables:CreateTable", + "s3tables:GetTable", + "s3tables:GetTableEncryption", + "s3tables:ListTables", + "s3tables:GetTableMetadataLocation", + "s3tables:UpdateTableMetadataLocation", + "s3tables:GetTableData", + "s3tables:PutTableData" + ], + "Resource": "arn:aws:s3tables:<region>:<account-id>:bucket/*", + "Effect": "Allow" + }, + { + "Action": [ + "cloudwatch:PutMetricData" + ], + "Resource": "*", + "Condition": { + "StringEquals": { + "cloudwatch:namespace": "AWS/Glue/ZeroETL" + } + }, + "Effect": "Allow" + }, + { + "Action": [ + "logs:CreateLogGroup", + "logs:CreateLogStream", + "logs:PutLogEvents" + ], + "Resource": "*", + "Effect": "Allow" + } + ] + } + +Add the following trust policy in the Target IAM role to allow AWS Glue Service to assume it: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "glue.amazonaws.com" + },