AWS AmazonRDS documentation change
Summary
Expanded zero-ETL integration documentation to include Amazon SageMaker lakehouse as a target alongside Amazon Redshift. Updated terminology from Redshift-specific references to generic 'target data warehouse/lakehouse', added new diagrams, documentation links, and limitations specific to SageMaker integrations.
Security assessment
Changes focus on expanding supported integration targets and general documentation updates rather than addressing security vulnerabilities. No security controls, vulnerabilities, or mitigations are mentioned in the diff. The updates relate to feature expansion rather than security improvements.
Diff
diff --git a/AmazonRDS/latest/AuroraUserGuide/zero-etl.md b/AmazonRDS/latest/AuroraUserGuide/zero-etl.md index d6c20b2ca..90cd3fd7d 100644 --- a//AmazonRDS/latest/AuroraUserGuide/zero-etl.md +++ b//AmazonRDS/latest/AuroraUserGuide/zero-etl.md @@ -7 +7 @@ BenefitsKey conceptsLimitationsQuotasSupported Regions -# Aurora zero-ETL integrations with Amazon Redshift +# Aurora zero-ETL integrations @@ -9 +9 @@ BenefitsKey conceptsLimitationsQuotasSupported Regions -An Aurora zero-ETL integration with Amazon Redshift enables near real-time analytics and machine learning (ML) using Amazon Redshift on petabytes of transactional data from Aurora. It's a fully managed solution for making transactional data available in Amazon Redshift after it is written to an Aurora DB cluster. _Extract, transform,_ and _load_ (ETL) is the process of combining data from multiple sources into a large, central data warehouse. +An Aurora zero-ETL integration with Amazon Redshift and Amazon SageMaker enables near real-time analytics and machine learning (ML) using data from Aurora. It's a fully managed solution for making transactional data available in your analytics destination after it is written to an Aurora DB cluster. _Extract, transform,_ and _load_ (ETL) is the process of combining data from multiple sources into a large, central data warehouse. @@ -11 +11 @@ An Aurora zero-ETL integration with Amazon Redshift enables near real-time analy -A zero-ETL integration makes the data in your Aurora DB cluster available in Amazon Redshift in near real-time. Once that data is in Amazon Redshift, you can power your analytics, ML, and AI workloads using the built-in capabilities of Amazon Redshift, such as machine learning, materialized views, data sharing, federated access to multiple data stores and data lakes, and integrations with Amazon SageMaker AI, QuickSight, and other AWS services. +A zero-ETL integration makes the data in your Aurora DB cluster available in Amazon Redshift or an Amazon SageMaker lakehouse in near real-time. Once that data is in the target data warehouse or data lake, you can power your analytics, ML, and AI workloads using the built-in capabilities, such as machine learning, materialized views, data sharing, federated access to multiple data stores and data lakes, and integrations with Amazon SageMaker AI, QuickSight, and other AWS services. @@ -13 +13 @@ A zero-ETL integration makes the data in your Aurora DB cluster available in Ama -To create a zero-ETL integration, you specify an Aurora DB cluster as the _source_ , and an Amazon Redshift data warehouse as the _target_. The integration replicates data from the source database into the target data warehouse. +To create a zero-ETL integration, you specify an Aurora DB cluster as the _source_ , and a supported data warehouse or lakehouse as the _target_. The integration replicates data from the source database into the target data warehouse or lakehouse. @@ -15 +15 @@ To create a zero-ETL integration, you specify an Aurora DB cluster as the _sourc -The following diagram illustrates this functionality: +The following diagram illustrates this functionality for zero-ETL integration with Amazon Redshift: @@ -19 +19,5 @@ The following diagram illustrates this functionality: -The integration monitors the health of the data pipeline and recovers from issues when possible. You can create integrations from multiple Aurora DB clusters into a single Amazon Redshift namespace, enabling you to derive insights across multiple applications. +The following diagram illustrates this functionality for zero-ETL integration with an Amazon SageMaker lakehouse: + + + +The integration monitors the health of the data pipeline and recovers from issues when possible. You can create integrations from multiple Aurora DB clusters into a single target data warehouse or lakehouse enabling you to derive insights across multiple applications. @@ -35 +39 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Getting started with Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.setting-up.html) + * [Getting started with Aurora zero-ETL integrations](./zero-etl.setting-up.html) @@ -39 +43,3 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Data filtering for Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.filtering.html) + * [Creating Aurora zero-ETL integrations with an Amazon SageMaker lakehouse](./zero-etl.creating-smlh.html) + + * [Data filtering for Aurora zero-ETL integrations](./zero-etl.filtering.html) @@ -41 +47 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Adding data to a source Aurora DB cluster and querying it in Amazon Redshift](./zero-etl.querying.html) + * [Adding data to a source Aurora DB cluster and querying it](./zero-etl.querying.html) @@ -43 +49 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Viewing and monitoring Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.describingmonitoring.html) + * [Viewing and monitoring Aurora zero-ETL integrations](./zero-etl.describingmonitoring.html) @@ -45 +51 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Modifying Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.modifying.html) + * [Modifying Aurora zero-ETL integrations](./zero-etl.modifying.html) @@ -47 +53 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Deleting Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.deleting.html) + * [Deleting Aurora zero-ETL integrations](./zero-etl.deleting.html) @@ -49 +55 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric - * [Troubleshooting Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.troubleshooting.html) + * [Troubleshooting Aurora zero-ETL integrations](./zero-etl.troubleshooting.html) @@ -56 +62 @@ For information about pricing for zero-ETL integrations, see [Amazon Aurora pric -Aurora zero-ETL integrations with Amazon Redshift have the following benefits: +Aurora zero-ETL integrations have the following benefits: @@ -64 +70 @@ Aurora zero-ETL integrations with Amazon Redshift have the following benefits: - * Let you leverage Amazon Redshift's analytics and ML capabilities to derive insights from transactional and other data, to respond effectively to critical, time-sensitive events. + * Let you leverage the target destination's analytics and ML capabilities to derive insights from transactional and other data, to respond effectively to critical, time-sensitive events. @@ -76 +82 @@ As you get started with zero-ETL integrations, consider the following concepts: -A fully managed data pipeline that automatically replicates transactional data and schemas from an Aurora DB cluster to an Amazon Redshift data warehouse. +A fully managed data pipeline that automatically replicates transactional data and schemas from an Aurora DB cluster to a data warehouse or catalog. @@ -83 +89 @@ The Aurora DB cluster where data is replicated from. You can specify a DB cluste -**Target data warehouse** +**Target** @@ -86 +92,3 @@ The Aurora DB cluster where data is replicated from. You can specify a DB cluste -The Amazon Redshift data warehouse where the data is replicated to. There are two types of data warehouse: a [provisioned cluster](https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html) data warehouse and a [serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-workgroup-namespace.html) data warehouse. A provisioned cluster data warehouse is a collection of computing resources called nodes, which are organized into a group called a _cluster_. A serverless data warehouse is comprised of a workgroup that stores compute resources, and a namespace that houses the database objects and users. Both data warehouses run an Amazon Redshift engine and contain one or more databases. +The data warehouse or lakehouse where the data is replicated to. There are two types of data warehouse: a [provisioned cluster](https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html) data warehouse and a [serverless](https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-workgroup-namespace.html) data warehouse. A provisioned cluster data warehouse is a collection of computing resources called nodes, which are organized into a group called a _cluster_. A serverless data warehouse is comprised of a workgroup that stores compute resources, and a namespace that houses the database objects and users. Both data warehouses run an analytics engine and contain one or more databases. + +A target lakehouse consists of catalogs, databases, tables, and views. For more information about lakehouse architecture, see [Amazon SageMaker Lakehouse components](https://docs.aws.amazon.com/sagemaker-unified-studio/latest/userguide/lakehouse-components.html) in the _Amazon SageMaker Unified Studio User Guide_. @@ -94 +102 @@ For more information, see [Data warehouse system architecture](https://docs.aws. -The following limitations apply to Aurora zero-ETL integrations with Amazon Redshift. +The following limitations apply to Aurora zero-ETL integrations. @@ -105,0 +114,2 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds + * Amazon SageMaker lakehouse limitations + @@ -111 +121 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * The source DB cluster must be in the same Region as the target Amazon Redshift data warehouse. + * The source DB cluster must be in the same Region as the target. @@ -119 +129 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * If you stop the source DB cluster, the last few transactions might not be replicated to the target data warehouse until you resume the cluster. + * If you stop the source DB cluster, the last few transactions might not be replicated to the target until you resume the cluster. @@ -135 +145 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * System tables, temporary tables, and views aren't replicated to Amazon Redshift. + * System tables, temporary tables, and views aren't replicated to target warehouses. @@ -137 +147 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * `ALTER TABLE` partition operations cause your table to resynchronize in order to reload data from Aurora to Amazon Redshift. The table will be unavailable for querying while it's resynchronizing. For more information, see [One or more of my Amazon Redshift tables requires a resync](./zero-etl.troubleshooting.html#zero-etl.troubleshooting.resync). + * `ALTER TABLE` partition operations cause your table to resynchronize in order to reload data from Aurora to the analytics destnation. The table will be unavailable for querying while it's resynchronizing. For more information, see [One or more of my Amazon Redshift tables requires a resync](./zero-etl.troubleshooting.html#zero-etl.troubleshooting.resync). @@ -144 +154 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * Your source DB cluster must be running a supported version of Aurora MySQL. For a list of supported versions, see [Supported Regions and Aurora DB engines for zero-ETL integrations with Amazon Redshift](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html). + * Your source DB cluster must be running a supported version of Aurora MySQL. For a list of supported versions, see [Supported Regions and Aurora DB engines for zero-ETL integrations](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html). @@ -159 +169 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * Your source DB cluster must be running a supported version of Aurora PostgreSQL. For a list of supported versions, see [Supported Regions and Aurora DB engines for zero-ETL integrations with Amazon Redshift](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html). + * Your source DB cluster must be running a supported version of Aurora PostgreSQL. For a list of supported versions, see [Supported Regions and Aurora DB engines for zero-ETL integrations](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html). @@ -161 +171 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * If you select an Aurora PostgreSQL source DB cluster, you must specify at least one data filter pattern. At minimum, the pattern must include a single database (``database-name`.*.*`) for replication to Amazon Redshift. For more information, see [Data filtering for Aurora zero-ETL integrations with Amazon Redshift](./zero-etl.filtering.html). + * If you select an Aurora PostgreSQL source DB cluster, you must specify at least one data filter pattern. At minimum, the pattern must include a single database (``database-name`.*.*`) for replication to the target warehouse. For more information, see [Data filtering for Aurora zero-ETL integrations](./zero-etl.filtering.html). @@ -165 +175 @@ The following limitations apply to Aurora zero-ETL integrations with Amazon Reds - * If you perform [declarative partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE) transactions on the source DB cluster, all affected tables enter a failed state and are no longer accessible in Amazon Redshift. + * If you perform [declarative partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE) transactions on the source DB cluster, all affected tables enter a failed state and are no longer accessible. @@ -179,0 +190,9 @@ For a list of Amazon Redshift limitations related to zero-ETL integrations, see +### Amazon SageMaker lakehouse limitations + +Following is a limitation for Amazon SageMaker lakehouse zero-ETL integrations. + + * Catalog names are limited to 19 characters in length. + + + + @@ -182 +201 @@ For a list of Amazon Redshift limitations related to zero-ETL integrations, see -Your account has the following quotas related to Aurora zero-ETL integrations with Amazon Redshift. Each quota is per-Region unless otherwise specified. +Your account has the following quotas related to Aurora zero-ETL integrations. Each quota is per-Region unless otherwise specified. @@ -187 +206 @@ Integrations | 100 | The total number of integrations within an AWS account. -Integrations per target data warehouse | 50 | The number of integrations sending data to a single target Amazon Redshift data warehouse. +Integrations per target | 50 | The number of integrations sending data to a single target data warehouse or lakehouse. @@ -190 +209 @@ Integrations per source cluster | 5 | The number of integrations sending data fr -In addition, Amazon Redshift places certain limits on the number of tables allowed in each DB instance or cluster node. For more information, see [Quotas and limits in Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html) in the _Amazon Redshift Management Guide_. +In addition, the target warehouse places certain limits on the number of tables allowed in each DB instance or cluster node. For more information about Amazon Redshift quotas and limits, see [Quotas and limits in Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/mgmt/amazon-redshift-limits.html) in the _Amazon Redshift Management Guide_. @@ -194 +213 @@ In addition, Amazon Redshift places certain limits on the number of tables allow -Aurora zero-ETL integrations with Amazon Redshift are available in a subset of AWS Regions. For a list of supported Regions, see [Supported Regions and Aurora DB engines for zero-ETL integrations with Amazon Redshift](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html). +Aurora zero-ETL integrations are available in a subset of AWS Regions. For a list of supported Regions, see [Supported Regions and Aurora DB engines for zero-ETL integrations](./Concepts.Aurora_Fea_Regions_DB-eng.Feature.Zero-ETL.html).