AWS AmazonRDS documentation change
Summary
Expanded documentation for RDS zero-ETL integrations to include Amazon SageMaker lakehouse as a target, updated terminology from 'Amazon Redshift' to more generic 'target data warehouse/lakehouse', added SageMaker-specific diagrams and limitations, and updated links/headings to reflect broader integration scope.
Security assessment
Changes focus on expanding supported targets (adding SageMaker lakehouse) and generalizing documentation language rather than addressing security vulnerabilities or describing security features. No security advisories, access controls, or vulnerability mitigations are mentioned.
Diff
diff --git a/AmazonRDS/latest/UserGuide/zero-etl.md b/AmazonRDS/latest/UserGuide/zero-etl.md index 4525b98f5..cef2bd29b 100644 --- a//AmazonRDS/latest/UserGuide/zero-etl.md +++ b//AmazonRDS/latest/UserGuide/zero-etl.md @@ -7 +7 @@ BenefitsKey conceptsLimitationsQuotasSupported Regions -# Amazon RDS zero-ETL integrations with Amazon Redshift +# Amazon RDS zero-ETL integrations @@ -9 +9 @@ BenefitsKey conceptsLimitationsQuotasSupported Regions -An Amazon RDS zero-ETL integration with Amazon Redshift enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from RDS. It's a fully managed solution for making transactional data available in Amazon Redshift after it is written to an RDS database. _Extract, transform,_ and _load_ (ETL) is the process of combining data from multiple sources into a large, central data warehouse. +An Amazon RDS zero-ETL integration with Amazon Redshift and Amazon SageMaker enables near real-time analytics and machine learning (ML) using data from RDS. It's a fully managed solution for making transactional data available in your analytics destination after it is written to an RDS database. _Extract, transform,_ and _load_ (ETL) is the process of combining data from multiple sources into a large, central data warehouse. @@ -11 +11 @@ An Amazon RDS zero-ETL integration with Amazon Redshift enables near real-time a -A zero-ETL integration makes the data in your RDS database available in Amazon Redshift in near real-time. Once that data is in Amazon Redshift, you can power your analytics, ML, and AI workloads using the built-in capabilities of Amazon Redshift, such as machine learning, materialized views, data sharing, federated access to multiple data stores and data lakes, and integrations with Amazon SageMaker AI, QuickSight, and other AWS services. +A zero-ETL integration makes the data in your RDS database available in Amazon Redshift or an Amazon SageMaker lakehouse in near real-time. Once that data is in the target data warehouse or data lake, you can power your analytics, ML, and AI workloads using the built-in capabilities, such as machine learning, materialized views, data sharing, federated access to multiple data stores and data lakes, and integrations with Amazon SageMaker AI, QuickSight, and other AWS services. @@ -13 +13 @@ A zero-ETL integration makes the data in your RDS database available in Amazon R -To create a zero-ETL integration, you specify an RDS database as the _source_ , and an Amazon Redshift data warehouse as the _target_. The integration replicates data from the source database into the target data warehouse. +To create a zero-ETL integration, you specify an RDS database as the _source_ , and a supported data warehouse or lakehouse as the _target_. The integration replicates data from the source database into the target data warehouse or lakehouse. @@ -15 +15 @@ To create a zero-ETL integration, you specify an RDS database as the _source_ , -The following diagram illustrates this functionality: +The following diagram illustrates this functionality for zero-ETL integration with Amazon Redshift: @@ -19 +19,5 @@ The following diagram illustrates this functionality: -The integration monitors the health of the data pipeline and recovers from issues when possible. You can create integrations from multiple RDS databases into a single Amazon Redshift namespace, enabling you to derive insights across multiple applications. +The following diagram illustrates this functionality for zero-ETL integration with an Amazon SageMaker lakehouse: + + + +The integration monitors the health of the data pipeline and recovers from issues when possible. You can create integrations from multiple RDS databases into a single target data warehouse or lakehouse enabling you to derive insights across multiple applications. @@ -33 +37 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Getting started with Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.setting-up.html) + * [Getting started with Amazon RDS zero-ETL integrations](./zero-etl.setting-up.html) @@ -37 +41,3 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Data filtering for Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.filtering.html) + * [Creating Amazon RDS zero-ETL integrations with an Amazon SageMaker lakehouse](./zero-etl.creating-smlh.html) + + * [Data filtering for Amazon RDS zero-ETL integrations](./zero-etl.filtering.html) @@ -39 +45 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Adding data to a source RDS database and querying it in Amazon Redshift](./zero-etl.querying.html) + * [Adding data to a source RDS database and querying it](./zero-etl.querying.html) @@ -41 +47 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Viewing and monitoring Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.describingmonitoring.html) + * [Viewing and monitoring Amazon RDS zero-ETL integrations](./zero-etl.describingmonitoring.html) @@ -43 +49 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Modifying Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.modifying.html) + * [Modifying Amazon RDS zero-ETL integrations](./zero-etl.modifying.html) @@ -45 +51 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Deleting Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.deleting.html) + * [Deleting Amazon RDS zero-ETL integrations](./zero-etl.deleting.html) @@ -47 +53 @@ The integration monitors the health of the data pipeline and recovers from issue - * [Troubleshooting Amazon RDS zero-ETL integrations with Amazon Redshift](./zero-etl.troubleshooting.html) + * [Troubleshooting Amazon RDS zero-ETL integrations](./zero-etl.troubleshooting.html) @@ -54 +60 @@ The integration monitors the health of the data pipeline and recovers from issue -RDS zero-ETL integrations with Amazon Redshift have the following benefits: +RDS zero-ETL integrations have the following benefits: @@ -62 +68 @@ RDS zero-ETL integrations with Amazon Redshift have the following benefits: - * Let you leverage Amazon Redshift's analytics and ML capabilities to derive insights from transactional and other data, to respond effectively to critical, time-sensitive events. + * Let you leverage the target destination's analytics and ML capabilities to derive insights from transactional and other data, to respond effectively to critical, time-sensitive events. @@ -74 +80 @@ As you get started with zero-ETL integrations, consider the following concepts: -A fully managed data pipeline that automatically replicates transactional data and schemas from an RDS database to an Amazon Redshift data warehouse. +A fully managed data pipeline that automatically replicates transactional data and schemas from an RDS database to a data warehouse or catalog. @@ -81 +87 @@ The RDS database where data is replicated from. You can specify a Single-AZ or M -**Target data warehouse** +**Target** @@ -84 +90,3 @@ The RDS database where data is replicated from. You can specify a Single-AZ or M -The Amazon Redshift data warehouse where the data is replicated to. There are two types of data warehouse: a [provisioned cluster](https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html) data warehouse and a [serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-workgroup-namespace.html) data warehouse. A provisioned cluster data warehouse is a collection of computing resources called nodes, which are organized into a group called a _cluster_. A serverless data warehouse is comprised of a workgroup that stores compute resources, and a namespace that houses the database objects and users. Both data warehouses run an Amazon Redshift engine and contain one or more databases. +The data warehouse or lakehouse where the data is replicated to. There are two types of data warehouse: a [provisioned cluster](https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html) data warehouse and a [serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-workgroup-namespace.html) data warehouse. A provisioned cluster data warehouse is a collection of computing resources called nodes, which are organized into a group called a _cluster_. A serverless data warehouse is comprised of a workgroup that stores compute resources, and a namespace that houses the database objects and users. Both data warehouses run an analytics engine and contain one or more databases. + +A target lakehouse consists of catalogs, databases, tables, and views. For more information about lakehouse architecture, see [Amazon SageMaker Lakehouse components](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/lakehouse-components.html) in the _Amazon SageMaker Unified Studio User Guide_. @@ -92 +100 @@ For more information, see [Data warehouse system architecture](https://docs.aws. -The following limitations apply to RDS zero-ETL integrations with Amazon Redshift. +The following limitations apply to RDS zero-ETL integrations. @@ -101,0 +110,2 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif + * Amazon SageMaker lakehouse limitations + @@ -107 +117 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif - * The source database must be in the same Region as the target Amazon Redshift data warehouse. + * The source database must be in the same Region as the target. @@ -115 +125 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif - * If you stop the source database, the last few transactions might not be replicated to the target data warehouse until you resume the database. + * If you stop the source database, the last few transactions might not be replicated to the target until you resume the database. @@ -127 +137 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif - * System tables, temporary tables, and views aren't replicated to Amazon Redshift. + * System tables, temporary tables, and views aren't replicated to target warehouses. @@ -134 +144 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif - * Your source database must be running a supported version of RDS for MySQL. For a list of supported versions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations with Amazon Redshift](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html). + * Your source database must be running a supported version of RDS for MySQL. For a list of supported versions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html). @@ -142 +152 @@ The following limitations apply to RDS zero-ETL integrations with Amazon Redshif - * `ALTER TABLE` partition operations cause your table to resynchronize in order to reload data from RDS to Amazon Redshift. The table will be unavailable for querying while it's resynchronizing. For more information, see [One or more of my Amazon Redshift tables requires a resync](./zero-etl.troubleshooting.html#zero-etl.troubleshooting.resync). + * `ALTER TABLE` partition operations cause your table to resynchronize in order to reload data from RDS to the target warehouse. The table will be unavailable for querying while it's resynchronizing. For more information, see [One or more of my Amazon Redshift tables requires a resync](./zero-etl.troubleshooting.html#zero-etl.troubleshooting.resync). @@ -150,0 +161,9 @@ For a list of Amazon Redshift limitations related to zero-ETL integrations, see +### Amazon SageMaker lakehouse limitations + +Following is a limitation for Amazon SageMaker lakehouse zero-ETL integrations. + + * Catalog names are limited to 19 characters in length. + + + + @@ -153 +172 @@ For a list of Amazon Redshift limitations related to zero-ETL integrations, see -Your account has the following quotas related to RDS zero-ETL integrations with Amazon Redshift. Each quota is per-Region unless otherwise specified. +Your account has the following quotas related to RDS zero-ETL integrations. Each quota is per-Region unless otherwise specified. @@ -158 +177 @@ Integrations | 100 | The total number of integrations within an AWS account. -Integrations per target data warehouse | 50 | The number of integrations sending data to a single target Amazon Redshift data warehouse. +Integrations per target | 50 | The number of integrations sending data to a single target data warehouse or lakehouse. @@ -161 +180 @@ Integrations per source instance | 5 | The number of integrations sending data f -In addition, Amazon Redshift places certain limits on the number of tables allowed in each DB instance or cluster node. For more information, see [Quotas and limits in Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html) in the _Amazon Redshift Management Guide_. +In addition, the target warehouse places certain limits on the number of tables allowed in each DB instance or cluster node. For more information about Amazon Redshift quotas and limits, see [Quotas and limits in Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html) in the _Amazon Redshift Management Guide_. @@ -165 +184 @@ In addition, Amazon Redshift places certain limits on the number of tables allow -RDS zero-ETL integrations with Amazon Redshift are available in a subset of AWS Regions. For a list of supported Regions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations with Amazon Redshift](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html). +RDS zero-ETL integrations are available in a subset of AWS Regions. For a list of supported Regions, see [Supported Regions and DB engines for Amazon RDS zero-ETL integrations](./Concepts.RDS_Fea_Regions_DB-eng.Feature.ZeroETL.html).