AWS msk: Updated Amazon MSK Data Delivery documentation
Summary
Rewrote content to introduce Amazon MSK Data Delivery capabilities and removed CLI/console instructions for describing channels.
Security assessment
The evidence shows a complete rewrite of the page to describe new data delivery features without addressing vulnerabilities or adding security guidance.
Evidence
+# Amazon MSK Data Delivery
Diff
diff --git a/msk/latest/developerguide/msk-data-delivery-describe.md b/msk/latest/developerguide/msk-data-delivery-describe.md index 1192e3569..61b71a0b5 100644 --- a//msk/latest/developerguide/msk-data-delivery-describe.md +++ b//msk/latest/developerguide/msk-data-delivery-describe.md @@ -1 +1 @@ -[View a markdown version of this page](msk-data-delivery-describe.md) +[View a markdown version of this page](msk-data-delivery.md) @@ -3 +3 @@ -[](/pdfs/msk/latest/developerguide/MSKDevGuide.pdf#msk-data-delivery-describe "Open PDF") +[](/pdfs/msk/latest/developerguide/MSKDevGuide.pdf#msk-data-delivery "Open PDF") @@ -7 +7 @@ -Using the AWS Management ConsoleUsing the AWS CLI +# Amazon MSK Data Delivery @@ -9 +9 @@ Using the AWS Management ConsoleUsing the AWS CLI -# Describe a Channel +With Amazon MSK data delivery, you can deliver Apache Kafka data from Amazon MSK Express brokers directly to Amazon S3, without connectors or additional infrastructure to manage. Amazon MSK Express automatically handles scaling, retries, and backpressure, and manages routine operations such as capacity scaling and version upgrades without introducing delivery gaps. Because these are native broker capabilities, they add no broker egress throughput, so you avoid the incremental infrastructure costs that scaling connector-based pipelines typically incurs and match capacity to actual workload demand rather than provisioning for peak. Each capability supports throughput of up to 10 GBps. @@ -11 +11 @@ Using the AWS Management ConsoleUsing the AWS CLI -## Using the AWS Management Console +The two capabilities are: @@ -13 +13 @@ Using the AWS Management ConsoleUsing the AWS CLI - 1. Open the Amazon MSK console at [https://console.aws.amazon.com/msk/home?region=us-east-1#/home/](https://console.aws.amazon.com/msk/home?region=us-east-1#/home/). + * **Data delivery to streaming tables for Apache Iceberg** — With Amazon MSK Data Delivery, you can continuously materialize Apache Kafka topics as Apache Iceberg tables on Amazon S3 Tables. Intelligent inline compaction eliminates the performance impact of small files and keeps query performance predictable without sacrificing data freshness. Built-in coordination resolves concurrent writer conflicts across high-throughput consumers. Amazon S3 Tables automatically handles ongoing table maintenance, including compaction, snapshot expiration, and unreferenced file cleanup. @@ -15 +15 @@ Using the AWS Management ConsoleUsing the AWS CLI - 2. In the navigation pane, choose **Clusters**. + * **Data delivery to Amazon S3 general purpose buckets** — With Amazon MSK Data Delivery, you can deliver Apache Kafka data in the source format to Amazon S3 general purpose buckets for downstream processing, with end-to-end reliability for mission-critical workloads. Use it to land Kafka data in Amazon S3 for use cases such as log archival, compliance retention, Kafka replay, and training AI/ML models. This approach removes the need to build self-managed connector pipelines that grow costly and operationally complex as workloads scale. @@ -17 +16,0 @@ Using the AWS Management ConsoleUsing the AWS CLI - 3. Choose the name of your Amazon MSK Provisioned cluster with Express brokers. @@ -19 +17,0 @@ Using the AWS Management ConsoleUsing the AWS CLI - 4. Choose the **Channel** tab. @@ -21 +18,0 @@ Using the AWS Management ConsoleUsing the AWS CLI - 5. Choose the name of the Channel to describe. @@ -22,0 +20 @@ Using the AWS Management ConsoleUsing the AWS CLI +###### Topics @@ -23,0 +22 @@ Using the AWS Management ConsoleUsing the AWS CLI + * [data delivery for streaming tables to Apache Iceberg](./msk-data-delivery-iceberg.html) @@ -24,0 +24 @@ Using the AWS Management ConsoleUsing the AWS CLI + * [data delivery to Amazon S3 general purpose buckets](./msk-data-delivery-s3.html) @@ -26 +25,0 @@ Using the AWS Management ConsoleUsing the AWS CLI -The detail page shows source topic, destination, delivery settings, current state, and recent Amazon CloudWatch metrics. @@ -28 +26,0 @@ The detail page shows source topic, destination, delivery settings, current stat -## Using the AWS CLI @@ -31,50 +28,0 @@ The detail page shows source topic, destination, delivery settings, current stat - aws kafka describe-channel \ - --cluster-arn "arn:aws:kafka:us-east-1:123456789012:cluster/my-express-cluster/abc123" \ - --channel-arn "arn:aws:kafka:us-east-1:123456789012:channel/my-express-cluster/abc123/orders-channel" - -The following is an example response for an S3 Tables destination. - - - { - "ChannelArn": "arn:aws:kafka:us-east-1:123456789012:channel/my-express-cluster/abc123/orders-channel", - "ChannelName": "orders-channel", - "TopicConfigurationList": [ - { - "TopicArn": "arn:aws:kafka:us-east-1:123456789012:topic/my-express-cluster/abc123/orders-topic", - "RecordConverter": { "ValueConverter": "JSON" }, - "RecordSchema": { "GsrArn": "arn:aws:glue:us-east-1:123456789012:schema/my-registry/orders-schema" } - } - ], - "IcebergDestinationConfiguration": { - "ServiceExecutionRoleArn": "arn:aws:iam::123456789012:role/MSKChannelRole", - "DataFreshnessInSeconds": 300, - "Catalog": { - "WarehouseLocation": "arn:aws:s3tables:us-east-1:123456789012:bucket/analytics-table-bucket" - }, - "DeadLetterQueueS3": { - "BucketArn": "arn:aws:s3:::my-dlq-bucket", - "ErrorOutputPrefix": "dlq/" - }, - "DestinationTableList": [ - { - "DestinationDatabaseName": "analytics_db", - "DestinationTableName": "orders_iceberg", - "PartitionSpec": { - "PartitionStrategy": "TIME_HOUR", - "SourceList": [ { "SourceName": "timestamp" } ] - } - } - ], - "CompressionType": "ZSTD" - }, - "Status": "ACTIVE", - "DestinationType": "ICEBERG", - "CreationTime": "2026-06-18T10:30:00.000Z" - } - -###### Note - -`Status` is one of `CREATING`, `ACTIVE`, `UPDATING`, `DELETING`, `FAILED`, `SUSPENDING`, `SUSPENDED`. When `Status` is `FAILED`, the response includes a `StateInfo` object with a `Code` and `Message` describing the cause. While an operation is in flight (`CREATING`, `UPDATING`, `DELETING`), the response also includes `ClusterOperationArn`. - -**API reference** — see `DescribeChannel` in the _Amazon MSK API Reference_. - @@ -87 +35 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Create a Channel +Troubleshooting @@ -89 +37 @@ Create a Channel -Update a Channel +data delivery for streaming tables to Apache Iceberg