AWS opensearch-service documentation change
Summary
Updated documentation for Kafka data migration, including authentication methods (mTLS/SASL/SCRAM/IAM), IAM role ARN formatting fixes, and revised permission references. Changed section headers from 'Connectivity' to 'Migrating data' terminology.
Security assessment
Added explicit documentation for security features like mTLS, SASL/SCRAM, and IAM authentication methods. Updated IAM policy examples and linked to security role setup documentation. No evidence of addressing a specific vulnerability.
Diff
diff --git a/opensearch-service/latest/developerguide/configure-client-self-managed-kafka.md b/opensearch-service/latest/developerguide/configure-client-self-managed-kafka.md index 1e01f6e1c..aa3076204 100644 --- a/opensearch-service/latest/developerguide/configure-client-self-managed-kafka.md +++ b/opensearch-service/latest/developerguide/configure-client-self-managed-kafka.md @@ -5 +5 @@ -Connectivity to public Kafka clusters Connectivity to Kafka clusters in a VPC +Migrating data from public Kafka clusters Migrating data from Kafka clusters in a VPC @@ -9 +9 @@ Connectivity to public Kafka clusters Connectivity to Kafka clusters in a VPC -You can use an OpenSearch Ingestion pipeline with self-managed Kafka to stream data to Amazon OpenSearch Service domains and OpenSearch Serverless collections. OpenSearch Ingestion supports both public and private network configurations for the streaming of data from self-managed Kafka to domains or collections managed by OpenSearch Service or OpenSearch Serverless. +You can use the [Kafka](https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/sources/kafka/) plugin to stream data from self-managed Kafka clusters to Amazon OpenSearch Service domains and OpenSearch Serverless collections. OpenSearch Ingestion supports connections from Kafka clusters configured with either public or private (VPC) networking. This topic outlines the prerequisites and steps to set up an ingestion pipeline, including configuring network settings and authentication methods such as mutual TLS (mTLS), SASL/SCRAM, or IAM. @@ -11 +11 @@ You can use an OpenSearch Ingestion pipeline with self-managed Kafka to stream d -## Connectivity to public Kafka clusters +## Migrating data from public Kafka clusters @@ -13 +13 @@ You can use an OpenSearch Ingestion pipeline with self-managed Kafka to stream d -You can use OpenSearch Ingestion pipelines to migrate data from a self-managed Kafka cluster with a public configuration, which means that the domain DNS name can be publicly resolved. To do so, set up an OpenSearch Ingestion pipeline with self-managed Kafka as the source and OpenSearch Service or OpenSearch Serverless as the destination. This processes your streaming data from a self-managed source cluster to an AWS-managed destination domain or collection. +You can use OpenSearch Ingestion pipelines to migrate data from a public self-managed Kafka cluster, which means that the domain DNS name can be publicly resolved. To do so, set up an OpenSearch Ingestion pipeline with self-managed Kafka as the source and OpenSearch Service or OpenSearch Serverless as the destination. This processes your streaming data from a self-managed source cluster to an AWS-managed destination domain or collection. @@ -35 +35 @@ The following sample domain access policy allows the pipeline role, which you cr - "AWS": "arn:aws:iam::{pipeline-account-id}:role/pipeline-role" + "AWS": "arn:aws:iam::pipeline-account-id:role/pipeline-role" @@ -42 +42 @@ The following sample domain access policy allows the pipeline role, which you cr - "arn:aws:es:{region}:{account-id}:domain/domain-name" + "arn:aws:es:region:account-id:domain/domain-name" @@ -48 +48 @@ The following sample domain access policy allows the pipeline role, which you cr -To create an IAM role with the correct permissions to access write data to the collection or domain, see [Required permissions for domains](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/osis-get-started.html#osis-get-started-permissions) and [Required permissions for collections](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/osis-serverless-get-started.html#osis-serverless-get-started-permissions). +To create an IAM role with the correct permissions to access write data to the collection or domain, see [Setting up roles and users in Amazon OpenSearch Ingestion](./pipeline-security-overview.html). @@ -53 +53 @@ To create an IAM role with the correct permissions to access write data to the c -### Step 1: Configure the pipeline role +### Step 1: rnfigure the pipeline role @@ -92 +92 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -101 +101 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -105 +105 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -109 +109 @@ You can use a preconfigured blueprint to create this pipeline. For more informat -## Connectivity to Kafka clusters in a VPC +## Migrating data from Kafka clusters in a VPC @@ -141 +141 @@ The following sample domain access policy allows the pipeline role, which you cr - "AWS": "arn:aws:iam::{pipeline-account-id}:role/pipeline-role" + "AWS": "arn:aws:iam::pipeline-account-id:role/pipeline-role" @@ -148 +148 @@ The following sample domain access policy allows the pipeline role, which you cr - "arn:aws:es:{region}:{account-id}:domain/domain-name" + "arn:aws:es:region:account-id:domain/domain-name" @@ -154 +154 @@ The following sample domain access policy allows the pipeline role, which you cr -To create an IAM role with the correct permissions to access write data to the collection or domain, see [Required permissions for domains](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/osis-get-started.html#osis-get-started-permissions) and [Required permissions for collections](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/osis-serverless-get-started.html#osis-serverless-get-started-permissions). +To create an IAM role with the correct permissions to access write data to the collection or domain, see [Setting up roles and users in Amazon OpenSearch Ingestion](./pipeline-security-overview.html). @@ -173 +173 @@ After you have your pipeline prerequisites set up, [configure the pipeline role] - "Resource": ["arn:aws:secretsmanager:{region}:{account-id}:secret:secret-name"] + "Resource": ["arn:aws:secretsmanager:region:account-id:secret:secret-name"] @@ -187,3 +187,3 @@ After you have your pipeline prerequisites set up, [configure the pipeline role] - "arn:aws:ec2:*:{account-id}:network-interface/*", - "arn:aws:ec2:*:{account-id}:subnet/*", - "arn:aws:ec2:*:{account-id}:security-group/*" + "arn:aws:ec2:*:account-id:network-interface/*", + "arn:aws:ec2:*:account-id:subnet/*", + "arn:aws:ec2:*:account-id:security-group/*" @@ -257 +257 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -266 +266 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role" @@ -270 +270 @@ You can also migrate data from a source Confluent Kafka cluster to an OpenSearch - sts_role_arn: "arn:aws:iam::{account-id}:role/pipeline-role" + sts_role_arn: "arn:aws:iam::account-id:role/pipeline-role"