AWS AmazonRDS medium security documentation change
Summary
Expanded zero-ETL integration documentation to include Amazon SageMaker lakehouse as a target alongside Amazon Redshift. Added new Step 3b with detailed IAM role configurations and permissions policies for AWS Glue catalog integration.
Security assessment
Added IAM role configurations with specific permissions (including kms:GenerateDataKey and kms:Decrypt) and trust relationships. These changes directly relate to access control and encryption requirements for secure data integration between services.
Diff
diff --git a/AmazonRDS/latest/UserGuide/zero-etl.setting-up.md b/AmazonRDS/latest/UserGuide/zero-etl.setting-up.md index 98a708e34..dc0b37796 100644 --- a//AmazonRDS/latest/UserGuide/zero-etl.setting-up.md +++ b//AmazonRDS/latest/UserGuide/zero-etl.setting-up.md @@ -5 +5 @@ -Step 1: Create a custom DB parameter groupStep 2: Select or create a source databaseStep 3: Create a target Amazon Redshift data warehouseSet up an integration using the AWS SDKsNext steps +Step 1: Create a custom DB parameter groupStep 2: Select or create a source databaseStep 3a: Create a target data warehouseSet up an integration using the AWS SDKsStep 3b: Create a target Amazon SageMaker lakehouseNext steps @@ -7 +7 @@ Step 1: Create a custom DB parameter groupStep 2: Select or create a source data -# Getting started with Amazon RDS zero-ETL integrations with Amazon Redshift +# Getting started with Amazon RDS zero-ETL integrations @@ -9 +9 @@ Step 1: Create a custom DB parameter groupStep 2: Select or create a source data -Before you create a zero-ETL integration with Amazon Redshift, configure your RDS database and your Amazon Redshift data warehouse with the required parameters and permissions. During setup, you'll complete the following steps: +Before you create a zero-ETL integration, configure your RDS database and your data warehouse with the required parameters and permissions. During setup, you'll complete the following steps: @@ -15 +15 @@ Before you create a zero-ETL integration with Amazon Redshift, configure your RD - 3. Create a target Amazon Redshift data warehouse. + 3. Create a target data warehouse for Amazon Redshift or Create a target Amazon SageMaker lakehouse. @@ -20 +20 @@ Before you create a zero-ETL integration with Amazon Redshift, configure your RD -After you complete these tasks, continue to [Creating Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.creating.html). +After you complete these tasks, continue to [Creating Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.creating.html) or [Creating Amazon RDS zero-ETL integrations with an Amazon SageMaker lakehouse](./zero-etl.creating-smlh.html). @@ -25,0 +26,9 @@ You can have RDS complete these setup steps for you while you're creating the in +For Step 3, you can choose to create either a target data warehouse (Step 3a) or a target lakehouse (Step 3b) depending on your needs: + + * Choose a data warehouse if you need traditional data warehousing capabilities with SQL-based analytics. + + * Choose a Amazon SageMaker lakehouse if you need machine learning capabilities and want to use lakehouse features for data science and ML workflows. + + + + @@ -28 +37 @@ You can have RDS complete these setup steps for you while you're creating the in -Amazon RDS zero-ETL integrations with Amazon Redshift require specific values for the DB parameters that control binary logging (binlog). To configure binary logging, you must first create a custom DB parameter group, and then associate it with the source database. Configure the following parameter values. For instructions to create a parameter group, see [DB parameter groups for Amazon RDS DB instances](./USER_WorkingWithDBInstanceParamGroups.html). We recommend that you configure all parameter values within the same request to avoid dependency issues. +Amazon RDS zero-ETL integrations require specific values for the DB parameters that control binary logging (binlog). To configure binary logging, you must first create a custom DB parameter group, and then associate it with the source database. Configure the following parameter values. For instructions to create a parameter group, see [DB parameter groups for Amazon RDS DB instances](./USER_WorkingWithDBInstanceParamGroups.html). We recommend that you configure all parameter values within the same request to avoid dependency issues. @@ -41 +50 @@ In addition, make sure that the `binlog_row_value_options` parameter is _not_ se -After you create a custom DB parameter group, choose or create an RDS for MySQL database. This database will be the source of data replication to Amazon Redshift. For instructions to create a Single-AZ or Multi-AZ DB instance, see [Creating an Amazon RDS DB instance](./USER_CreateDBInstance.html). For instructions to create a Multi-AZ DB cluster, see [Creating a Multi-AZ DB cluster for Amazon RDS](./create-multi-az-db-cluster.html). +After you create a custom DB parameter group, choose or create an RDS for MySQL database. This database will be the source of data replication to the target data warehouse. For instructions to create a Single-AZ or Multi-AZ DB instance, see [Creating an Amazon RDS DB instance](./USER_CreateDBInstance.html). For instructions to create a Multi-AZ DB cluster, see [Creating a Multi-AZ DB cluster for Amazon RDS](./create-multi-az-db-cluster.html). @@ -43 +52 @@ After you create a custom DB parameter group, choose or create an RDS for MySQL -The database must be running a supported DB engine version. For a list of supported versions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations with Amazon Redshift](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html). +The database must be running a supported DB engine version. For a list of supported versions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html). @@ -53 +62 @@ In addition, make sure that automated backups are enabled on the database. For m -## Step 3: Create a target Amazon Redshift data warehouse +## Step 3a: Create a target data warehouse @@ -55 +64 @@ In addition, make sure that automated backups are enabled on the database. For m -After you create your source database, you must create and configure a target data warehouse in Amazon Redshift. The data warehouse must meet the following requirements: +After you create your source database, you must create and configure a target data warehouse. The data warehouse must meet the following requirements: @@ -95 +104 @@ After you create a data warehouse, you must configure the source RDS database as -Rather than setting up each resource manually, you can run the following Python script to automatically set up the required resources for you. The code example uses the [AWS SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to create a source RDS for MySQL DB instance and target Amazon Redshift data warehouse, each with the required parameter values. It then waits for the databases to be available before creating a zero-ETL integration between them. You can comment out different functions depending on which resources you need to set up. +Rather than setting up each resource manually, you can run the following Python script to automatically set up the required resources for you. The code example uses the [AWS SDK for Python (Boto3)](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to create a source RDS for MySQL DB instance and target data warehouse, each with the required parameter values. It then waits for the databases to be available before creating a zero-ETL integration between them. You can comment out different functions depending on which resources you need to set up. @@ -278,0 +288,120 @@ Within the script, optionally modify the names of the source, target, and parame +## Step 3b: Create a target Amazon SageMaker lakehouse + +When creating a zero-ETL integration with an Amazon SageMaker lakehouse, you must target a AWS Glue catalog in AWS Lake Formation. + +### Configure permissions for the target AWS Glue catalog + +To enable zero-ETL integration for a catalog, you need to configure the following permissions: + + * AWS Lake Formation administrator role + + * Glue role for data transfer + + + + +The target creation role must be a Lake Formation administrator and requires the following permissions: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "VisualEditor0", + "Effect": "Allow", + "Action": "lakeformation:RegisterResource", + "Resource": "*" + }, + { + "Sid": "VisualEditor1", + "Effect": "Allow", + "Action": [ + "s3:PutEncryptionConfiguration", + "iam:PassRole", + "glue:CreateCatalog", + "glue:GetCatalog", + "s3:PutBucketTagging", + "s3:PutLifecycleConfiguration", + "s3:PutBucketPolicy", + "s3:CreateBucket", + "redshift-serverless:CreateNamespace", + "s3:DeleteBucket", + "s3:PutBucketVersioning", + "redshift-serverless:CreateWorkgroup" + ], + "Resource": [ + "arn:aws:glue:*:account-id:catalog", + "arn:aws:glue:*:account-id:catalog/*", + "arn:aws:s3:::*", + "arn:aws:redshift-serverless:*:account-id:workgroup/*", + "arn:aws:redshift-serverless:*:account-id:namespace/*", + "arn:aws:iam::account-id:role/GlueDataCatalogDataTransferRole" + ] + } + ] + } + +The target creation role must have the following trust relationship: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": "glue.amazonaws.com" + }, + "Action": "sts:AssumeRole" + }, + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:sts::account-id:assumed-role/Role" + }, + "Action": "sts:AssumeRole" + } + ] + } + +The Glue data transfer role (GlueDataCatalogDataTransferRole) is required for MySQL catalog operations and must have the following permissions: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "DataTransferRolePolicy", + "Effect": "Allow", + "Action": [ + "kms:GenerateDataKey", + "kms:Decrypt", + "glue:GetCatalog", + "glue:GetDatabase" + ], + "Resource": [ + "*" + ] + } + ] + } + +The Glue data transfer role must have the following trust relationship: + + + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "Service": [ + "glue.amazonaws.com", + "redshift.amazonaws.com" + ] + }, + "Action": "sts:AssumeRole" + } + ] + } + @@ -281 +410 @@ Within the script, optionally modify the names of the source, target, and parame -With a source RDS database and an Amazon Redshift target data warehouse, you can now create a zero-ETL integration and replicate data. For instructions, see [Creating Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.creating.html). +With a source RDS database and either an Amazon Redshift target data warehouse or Amazon SageMaker lakehouse, you can create a zero-ETL integration and replicate data. For instructions, see [Creating Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.creating.html). @@ -291 +420 @@ Zero-ETL integrations -Creating zero-ETL integrations +Creating zero-ETL integrations with Amazon Redshift