AWS msk: MSK S3 data delivery: restructure, add Pricing, and cross-Region/account limits
Summary
Reorganized the Amazon MSK data delivery to Amazon S3 page (new title, reordered Topics, moved Benefits/Integrations/Data flow sections, updated diagram caption) and added a new Pricing section plus new requirement text stating the destination bucket must be in the same AWS Region as the cluster and that cross-account delivery is only supported for the destination bucket, with the cluster and dead-letter-queue bucket required to be in the same AWS account as the Channel.
Security assessment
The diff is primarily an editorial restructure (title, section reordering, diagram, pricing), but the newly added requirements paragraph constrains cross-account and cross-Region data paths: the DLQ bucket and cluster must share the Channel's AWS account while only the destination bucket may be cross-account. This documents an account/Region boundary that limits where data and error records can flow, which is security-relevant configuration guidance rather than a fix for a specific vulnerability. The relocated AWS KMS ('optional customer-managed encryption at rest') and AWS CloudTrail ('API audit logging') bullets continue to document encryption and audit capabilities, but those lines are pre-existing content merely moved, so they are not new security controls.
Evidence
+ * Cross-account delivery is supported for the destination bucket only. Your Amazon MSK cluster and the dead-letter queue bucket must be in the same AWS account as the Channel; only the destination bucket can be in a different AWS account.
Diff
diff --git a/msk/latest/developerguide/msk-data-delivery-s3.md b/msk/latest/developerguide/msk-data-delivery-s3.md index e29b49327..418469396 100644 --- a//msk/latest/developerguide/msk-data-delivery-s3.md +++ b//msk/latest/developerguide/msk-data-delivery-s3.md @@ -7 +7 @@ -IntegrationsCommon use casesData flowBenefitsHow it worksRequirements and supported configurations +BenefitsPricingCommon use casesIntegrationsHow it worksRequirements and supported configurations @@ -9 +9 @@ IntegrationsCommon use casesData flowBenefitsHow it worksRequirements and suppor -# data delivery to Amazon S3 general purpose buckets +# Amazon MSK data delivery to Amazon S3 @@ -11 +11 @@ IntegrationsCommon use casesData flowBenefitsHow it worksRequirements and suppor -With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source format to Amazon S3 general purpose buckets for downstream processing, with end-to-end reliability for mission-critical workloads. Use it to land Kafka data in Amazon S3 for use cases such as log archival, compliance retention, Kafka replay, and training AI/ML models. This approach removes the need to build self-managed connector pipelines that grow costly and operationally complex as workloads scale. +Amazon MSK data delivery to Amazon S3 delivers your Apache Kafka data to general purpose Amazon S3 buckets in its source format. You choose a Kafka topic and a destination bucket, and Amazon MSK Express brokers deliver your records to Amazon S3 as a fully managed capability. @@ -13 +13 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f -###### Topics +Amazon MSK scales delivery to your workload automatically and handles retries and backpressure, supporting throughput of up to 10 GBps. Routine operations such as capacity scaling and version upgrades happen without delivery gaps. Because delivery is native to Express brokers rather than a connector fleet that you run, there is no additional broker egress throughput to provision, and you pay $8.00 per TB delivered. Together this can reduce ingestion and delivery costs by up to 60% compared to self-managed alternatives. @@ -15 +15 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * Integrations +Without this capability, delivering Apache Kafka data to Amazon S3 for log archival, compliance retention, Kafka replay, or machine learning training data means building pipelines from self-managed connectors. You source or build connector plugins, secure approvals to deploy them, scale worker capacity as throughput grows, and apply security updates across a connector fleet, and you size that capacity for peak rather than actual demand. Data delivery removes the connector fleet and the coordination it requires. @@ -17 +17 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * Common use cases +The following diagram shows how records flow from an Amazon MSK Express broker topic through a Data Delivery channel to a general purpose Amazon S3 bucket, shown as the dashed path. The solid path shows delivery to Apache Iceberg tables in Amazon S3 Tables. @@ -19 +19,3 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * Data flow + + +###### Topics @@ -22,0 +25,6 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f + * Pricing + + * Common use cases + + * Integrations + @@ -50 +58 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f -## Integrations +## Benefits @@ -52 +60 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * **Amazon MSK Express brokers** — the data source. + * **No infrastructure to manage** — No connectors or compute clusters. You configure a Channel and the service handles delivery, scaling, and fault tolerance. @@ -54 +62 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * **Amazon S3** — general-purpose object destination. + * **No broker impact** — A channel reads from the topic without consuming broker throughput or affecting producer and consumer workloads. @@ -56 +64 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * **Amazon CloudWatch** — metrics and operational logs. + * **Scales with your data** — Supports data delivery throughput of up to 10 GBps with no manual scaling required. @@ -58 +66 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * **AWS CloudTrail** — API audit logging. + * **Data freshness in minutes** — Delivered data is available for querying or processing within 5 to 15 minutes of being produced to the topic. @@ -60 +68 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f - * **AWS KMS** — optional customer-managed encryption at rest. + * **Built-in error handling** — Unprocessable records are routed to a dead-letter queue with error context, so delivery continues uninterrupted. @@ -64,0 +73,10 @@ With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source f +## Pricing + +You pay for the volume of data delivered from your Apache Kafka topics to the destination, billed at per-byte resolution, at $8.00 per TB. There are no setup fees, minimum commitments, or upfront costs. + +Standard Amazon S3 storage, request, and data transfer charges apply to the destination bucket. There is no additional charge for broker egress used by this capability, and there are no separate connector, worker, or MSK Connect Unit (MCU) fees. + +You are not charged separately for failed delivery attempts routed to the dead-letter queue. Only successfully delivered data is billed. + +Rates vary by destination type and are subject to change. For current pricing, see [Amazon MSK pricing](https://aws.amazon.com/msk/pricing/). + @@ -76,7 +94 @@ For the API specification, see `CreateChannel`, `DescribeChannel`, `UpdateChanne -## Data flow - -The following diagram shows how records flow from an Amazon MSK Express broker topic through a Data Delivery channel to your destination, with unprocessable records routed to a dead-letter queue. - - - -## Benefits +## Integrations @@ -84 +96 @@ The following diagram shows how records flow from an Amazon MSK Express broker t - * **No infrastructure to manage** — No connectors or compute clusters. You configure a Channel and the service handles delivery, scaling, and fault tolerance. + * **Amazon MSK Express brokers** — the data source. @@ -86 +98 @@ The following diagram shows how records flow from an Amazon MSK Express broker t - * **No broker impact** — A channel reads from the topic without consuming broker throughput or affecting producer and consumer workloads. + * **Amazon S3** — general-purpose object destination. @@ -88 +100 @@ The following diagram shows how records flow from an Amazon MSK Express broker t - * **Scales with your data** — Supports data delivery throughput of up to 10 GBps with no manual scaling required. + * **Amazon CloudWatch** — metrics and operational logs. @@ -90 +102 @@ The following diagram shows how records flow from an Amazon MSK Express broker t - * **Data freshness in minutes** — Delivered data is available for querying or processing within 5 to 15 minutes of being produced to the topic. + * **AWS CloudTrail** — API audit logging. @@ -92 +104 @@ The following diagram shows how records flow from an Amazon MSK Express broker t - * **Built-in error handling** — Unprocessable records are routed to a dead-letter queue with error context, so delivery continues uninterrupted. + * **AWS KMS** — optional customer-managed encryption at rest. @@ -124,0 +137,4 @@ A Channel does **not** backfill previously produced data — only data produced + * The destination bucket must be in the same AWS Region as your Amazon MSK cluster. Cross-Region delivery is not supported. + + * Cross-account delivery is supported for the destination bucket only. Your Amazon MSK cluster and the dead-letter queue bucket must be in the same AWS account as the Channel; only the destination bucket can be in a different AWS account. +