AWS opensearch-service documentation change
Summary
Updated documentation for DynamoDB integration with OpenSearch, including restructuring of snapshot options, ARN placeholder standardization, example corrections, and clarification of requirements for DynamoDB Streams/PITR
Security assessment
Changes focus on documentation clarity and example accuracy. While KMS encryption is mentioned, the modifications only adjust placeholder formatting rather than introducing new security concepts. No evidence of addressing specific vulnerabilities.
Diff
diff --git a/opensearch-service/latest/developerguide/configure-client-ddb.md b/opensearch-service/latest/developerguide/configure-client-ddb.md index 826a032b7..57d037b83 100644 --- a/opensearch-service/latest/developerguide/configure-client-ddb.md +++ b/opensearch-service/latest/developerguide/configure-client-ddb.md @@ -9 +9 @@ PrerequisitesStep 1: Configure the pipeline roleStep 2: Create the pipelineData -You can use an OpenSearch Ingestion pipeline with DynamoDB to stream DynamoDB table events (such as create, update, and delete) to Amazon OpenSearch Service domains and collections. The OpenSearch Ingestion pipeline incorporates change data capture (CDC) infrastructure to provide a high-scale, low-latency way to continuously stream data from a DynamoDB table. +You can use the [DynamoDB](https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/sources/dynamo-db/) plugin to stream table events, such as creates, updates, and deletes, to Amazon OpenSearch Service domains and Amazon OpenSearch Serverless collections. The pipeline uses change data capture (CDC) for high-scale, low-latency streaming. @@ -11 +11 @@ You can use an OpenSearch Ingestion pipeline with DynamoDB to stream DynamoDB ta -There are two ways that you can use DynamoDB as a source to process data—with and without a _full initial snapshot_. +You can process DynamoDB data with or without a full initial snapshot. @@ -13 +13 @@ There are two ways that you can use DynamoDB as a source to process data—with -A full initial snapshot is a backup of a table that DynamoDB takes with the [point-in-time recovery](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery.html) (PITR) feature. DynamoDB uploads this snapshot to Amazon S3. From there, an OpenSearch Ingestion pipeline sends it to one index in a domain, or partitions it to multiple indexes in a domain. To keep the data in DynamoDB and OpenSearch consistent, the pipeline syncs all of the create, update, and delete events in the DynamoDB table with the documents saved in the OpenSearch index or indexes. + * **With a full snapshot** – DynamoDB uses [point-in-time recovery](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/PointInTimeRecovery.html) (PITR) to create a backup and uploads it to Amazon S3. OpenSearch Ingestion then indexes the snapshot in one or multiple OpenSearch indexes. To maintain consistency, the pipeline synchronizes all DynamoDB changes with OpenSearch. This option requires you to enable both PITR and [DynamoDB Streams](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.Streams). @@ -15 +15 @@ A full initial snapshot is a backup of a table that DynamoDB takes with the [poi -When you use a full initial snapshot, your OpenSearch Ingestion pipeline first ingests the snapshot and then starts reading data from [DynamoDB Streams](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.Streams). It eventually catches up and maintains near real-time data consistency between DynamoDB and OpenSearch. When you choose this option, you must enable both PITR and a DynamoDB stream on your table. + * **Without a snapshot** – OpenSearch Ingestion streams only new DynamoDB events. Choose this option if you already have a snapshot or need real-time streaming without historical data. This option requires you to enable only DynamoDB Streams. @@ -17 +16,0 @@ When you use a full initial snapshot, your OpenSearch Ingestion pipeline first i -You can also use the OpenSearch Ingestion integration with DynamoDB to stream events without a snapshot. Choose this option if you already have a full snapshot from some other mechanism, or if you just want to stream current events from a DynamoDB table with DynamoDB Streams. When you choose this option, you only need to enable a DynamoDB stream on your table. @@ -19 +18,3 @@ You can also use the OpenSearch Ingestion integration with DynamoDB to stream ev -For more information about this integration, see [DynamoDB zero-ETL integration with Amazon OpenSearch Service](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/OpenSearchIngestionForDynamoDB.html) in the _Amazon DynamoDB Developer Guide_. + + +For more information, see [DynamoDB zero-ETL integration with Amazon OpenSearch Service](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/OpenSearchIngestionForDynamoDB.html) in the _Amazon DynamoDB Developer Guide_. @@ -63 +64 @@ After you have your DynamoDB table set up, [set up the pipeline role](./pipeline - "arn:aws:dynamodb:us-east-1:{account-id}:table/my-table" + "arn:aws:dynamodb:region:account-id:table/my-table" @@ -73 +74 @@ After you have your DynamoDB table set up, [set up the pipeline role](./pipeline - "arn:aws:dynamodb:us-east-1:{account-id}:table/my-table/export/*" + "arn:aws:dynamodb:region:account-id:table/my-table/export/*" @@ -85 +86 @@ After you have your DynamoDB table set up, [set up the pipeline role](./pipeline - "arn:aws:dynamodb:us-east-1:{account-id}:table/my-table/stream/*" + "arn:aws:dynamodb:region:account-id:table/my-table/stream/*" @@ -98 +99 @@ After you have your DynamoDB table set up, [set up the pipeline role](./pipeline - "arn:aws:s3:::amzn-s3-demo- /{exportPath}/*" + "arn:aws:s3:::my-bucket/export-folder/*" @@ -104 +105 @@ After you have your DynamoDB table set up, [set up the pipeline role](./pipeline -You can also use an AWS KMS customer managed key to encrypt the export data files. To decrypt the exported objects, specify `s3_sse_kms_key_id` for the key ID in the export configuration of the pipeline with the following format: `arn:aws:kms:`us-west-2`:`{account-id}`:key/`my-key-id``. The following policy includes the required permissions for using a customer managed key: +You can also use an AWS KMS customer managed key to encrypt the export data files. To decrypt the exported objects, specify `s3_sse_kms_key_id` for the key ID in the export configuration of the pipeline with the following format: `arn:aws:kms:`region`:`account-id`:key/`my-key-id``. The following policy includes the required permissions for using a customer managed key: @@ -114 +115 @@ You can also use an AWS KMS customer managed key to encrypt the export data file - "Resource": arn:aws:kms:us-west-2:{account-id}:key/my-key-id + "Resource": arn:aws:kms:region:account-id:key/my-key-id @@ -127 +128 @@ You can then configure an OpenSearch Ingestion pipeline like the following, whic - - table_arn: "arn:aws:dynamodb:us-west-2:{account-id}:table/table-a" + - table_arn: "arn:aws:dynamodb:region:account-id:table/table-a" @@ -129 +130 @@ You can then configure an OpenSearch Ingestion pipeline like the following, whic - s3_bucket: "amzn-s3-demo-" + s3_bucket: "my-bucket" @@ -134,2 +135,2 @@ You can then configure an OpenSearch Ingestion pipeline like the following, whic - region: "us-west-2" - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + region: "us-east-1" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -138,2 +139,2 @@ You can then configure an OpenSearch Ingestion pipeline like the following, whic - hosts: ["https://search-mydomain.us-east-1.es.amazonaws.com"] - index: "${getMetadata(\"table_name\")}" + hosts: ["https://search-mydomain.region.es.amazonaws.com"] + index: "${getMetadata(\"table-name\")}"