AWS opensearch-service medium security documentation change
Summary
Restructured tutorial steps, updated IAM permissions to include CreateRole/AttachPolicy, changed role ARN references to 'OpenSearchIngestion-PipelineRole', simplified pipeline creation process with service-managed roles, and emphasized public access configuration
Security assessment
Added IAM permissions (iam:CreateRole, iam:Attach*) and automated role creation to enforce proper permissions. Changed role ARN to a service-managed naming convention ('OpenSearchIngestion-PipelineRole') to prevent manual misconfiguration. These changes reduce security risks from improper IAM setup but don't address a specific disclosed vulnerability.
Diff
diff --git a/opensearch-service/latest/developerguide/osis-get-started.md b/opensearch-service/latest/developerguide/osis-get-started.md index abfe82cb3..c0f4260fb 100644 --- a//opensearch-service/latest/developerguide/osis-get-started.md +++ b//opensearch-service/latest/developerguide/osis-get-started.md @@ -5 +5 @@ -Required permissionsStep 1: Create the pipeline roleStep 2: Create a domainStep 3: Create a pipelineStep 4: Ingest some sample dataFixing permissions issuesRelated resources +Required permissionsStep 1: Create the pipeline roleStep 1: Create a domainStep 2: Create a pipelineStep 3: Ingest some sample dataFixing permissions issuesRelated resources @@ -15 +15 @@ You'll complete the following steps in this tutorial: - 1. Create the pipeline role. + 1. Create a domain. @@ -17 +17 @@ You'll complete the following steps in this tutorial: - 2. Create a domain. + 2. Create a pipeline. @@ -19,3 +19 @@ You'll complete the following steps in this tutorial: - 3. Create a pipeline. - - 4. Ingest some sample data. + 3. Ingest some sample data. @@ -28,3 +26 @@ Within the tutorial, you'll create the following resources: - * A pipeline named `ingestion-pipeline` - - * A domain named `ingestion-domain` that the pipeline will write to + * A domain named `ingestion-domain` that the pipeline writes to @@ -32 +28 @@ Within the tutorial, you'll create the following resources: - * An IAM role named `PipelineRole` that the pipeline will assume in order to write to the domain + * A pipeline named `ingestion-pipeline` @@ -39 +35 @@ Within the tutorial, you'll create the following resources: -To complete this tutorial, your user or role must have an attached [identity-based policy](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/security-iam-serverless.html#security-iam-serverless-id-based-policies) with the following minimum permissions. These permissions allow you to create a pipeline role (`iam:Create`), create or modify a domain (`es:*`), and work with pipelines (`osis:*`). The `iam:PassRole` permission allows you to pass the pipeline role permissions to the pipeline so that it can write data to the domain. +To complete this tutorial, your user or role must have an attached [identity-based policy](./security-iam-serverless.html#security-iam-serverless-id-based-policies) with the following minimum permissions. These permissions allow you to create a pipeline role and attach a policy (`iam:Create*` and `iam:Attach*`), create or modify a domain (`es:*`), and work with pipelines (`osis:*`). @@ -50,0 +47 @@ To complete this tutorial, your user or role must have an attached [identity-bas + "iam:Attach*", @@ -56 +53 @@ To complete this tutorial, your user or role must have an attached [identity-bas - "arn:aws:iam::your-account-id:role/PipelineRole" + "arn:aws:iam::your-account-id:role/OpenSearchIngestion-PipelineRole" @@ -59,0 +57,2 @@ To complete this tutorial, your user or role must have an attached [identity-bas + "iam:CreateRole", + "iam:AttachPolicy", @@ -76 +75 @@ First, create a role that the pipeline will assume in order to access the OpenSe - 3. In this tutorial, you'll ingest data into a domain called `ingestion-domain`, which you'll create in the next step. Select **JSON** and paste the following policy into the editor. Replace `{your-account-id}` with your account ID, and modify the Region if necessary. + 3. In this tutorial, you'll ingest data into a domain called `ingestion-domain`, which you'll create in the next step. Select **JSON** and paste the following policy into the editor. Replace ``your-account-id`` with your account ID, and modify the Region if necessary. @@ -132 +131 @@ Remember the Amazon Resource Name (ARN) of the role (for example, `arn:aws:iam:: -## Step 2: Create a domain +## Step 1: Create a domain @@ -134 +133 @@ Remember the Amazon Resource Name (ARN) of the role (for example, `arn:aws:iam:: -Next, create a domain named `ingestion-domain` to ingest data into. +First, create a domain named `ingestion-domain` to ingest data into. @@ -151 +150 @@ These requirements are meant to ensure simplicity in this tutorial. In productio -The domain must have an access policy that grants permission to `PipelineRole`, which you created in the previous step. The pipeline will assume this role (named **sts_role_arn** in the pipeline configuration) in order to send data to the OpenSearch Service domain sink. +The domain must have an access policy that grants permission to the `OpenSearchIngestion-PipelineRole` IAM role, which OpenSearch Service will create for you in the next step. The pipeline will assume this role in order to send data to the domain sink. @@ -153 +152 @@ The domain must have an access policy that grants permission to `PipelineRole`, -Make sure that the domain has the following domain-level access policy, which grants `PipelineRole` access to the domain. Replace the Region and account ID with your own: +Make sure that the domain has the following domain-level access policy, which grants the pipeline role access to the domain. Replace the Region and account ID with your own: @@ -162 +161 @@ Make sure that the domain has the following domain-level access policy, which gr - "AWS": "arn:aws:iam::your-account-id:role/PipelineRole" + "AWS": "arn:aws:iam::your-account-id:role/OpenSearchIngestion-PipelineRole" @@ -170 +169 @@ Make sure that the domain has the following domain-level access policy, which gr -For more information about creating domain-level access policies, see [Resource-based access policies](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-types-resource). +For more information about creating domain-level access policies, see [Resource-based policies](./ac.html#ac-types-resource). @@ -172 +171 @@ For more information about creating domain-level access policies, see [Resource- -If you already have a domain created, modify its existing access policy to provide the above permissions to `PipelineRole`. +If you already have a domain created, modify its existing access policy to provide the above permissions to `OpenSearchIngestion-PipelineRole`. @@ -174 +173 @@ If you already have a domain created, modify its existing access policy to provi -###### Note +## Step 2: Create a pipeline @@ -176,5 +175 @@ If you already have a domain created, modify its existing access policy to provi -Remember the domain endpoint (for example, `https://search-`ingestion-domain`.us-east-1.es.amazonaws.com`). You'll use it in the next step to configure your pipeline. - -## Step 3: Create a pipeline - -Now that you have a domain and a role with the appropriate access rights, you can create a pipeline. +Now that you have a domain, you can create a pipeline. @@ -188 +183 @@ Now that you have a domain and a role with the appropriate access rights, you ca - 3. Select the blank pipeline, then choose **Select blueprint**. + 3. Select the **Blank** pipeline, then choose **Select blueprint**. @@ -190 +185 @@ Now that you have a domain and a role with the appropriate access rights, you ca - 4. Name the pipeline **ingestion-pipeline** and keep the capacity settings as their defaults. + 4. In this tutorial, we'll create a simple pipeline that uses the [HTTP source](https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/sources/http-source/) plugin. The plugin accepts log data in a JSON array format. We'll specify a single OpenSearch Service domain as the sink, and ingest all data into the `application_logs` index. @@ -192 +187 @@ Now that you have a domain and a role with the appropriate access rights, you ca - 5. In this tutorial, you'll create a simple sub-pipeline called `log-pipeline` that uses the [Http source](https://opensearch.org/docs/latest/data-prepper/pipelines/configuration/sources/http-source/) plugin. This plugin accepts log data in a JSON array format. You'll specify a single OpenSearch Service domain as the sink, and ingest all data into the `application_logs` index. +In the **Source** menu, choose **HTTP**. For the **Path** , enter **/logs**. @@ -194 +189 @@ Now that you have a domain and a role with the appropriate access rights, you ca -Under **Pipeline configuration** , paste the following YAML configuration into the editor: + 5. For simplicity in this tutorial, we'll configure public access for the pipeline. For **Source network options** , choose **Public access**. For information about configuring VPC access, see [Configuring VPC access for Amazon OpenSearch Ingestion pipelines](./pipeline-security.html). @@ -196,16 +191 @@ Under **Pipeline configuration** , paste the following YAML configuration into t - version: "2" - log-pipeline: - source: - http: - path: "/${pipelineName}/test_ingestion_path" - processor: - - date: - from_time_received: true - destination: "@timestamp" - sink: - - opensearch: - hosts: [ "https://search-ingestion-domain.us-east-1.es.amazonaws.com" ] - index: "application_logs" - aws: - sts_role_arn: "arn:aws:iam::your-account-id:role/PipelineRole" - region: "us-east-1" + 6. Choose **Next**. @@ -213 +193,3 @@ Under **Pipeline configuration** , paste the following YAML configuration into t -###### Note + 7. For **Processor** , enter **Date** and choose **Add**. + + 8. Enable **From time received**. Leave all other settings as their defaults. @@ -215 +197 @@ Under **Pipeline configuration** , paste the following YAML configuration into t -The `path` option specifies the URI path for ingestion. This option is required for pull-based sources. For more information, see [Specifying the ingestion path](./creating-pipeline.html#pipeline-path). + 9. Choose **Next**. @@ -217 +199 @@ The `path` option specifies the URI path for ingestion. This option is required - 6. Replace the `hosts` URL with the endpoint of the domain that you created (or modified) in the previous section. Replace the `sts_role_arn` parameter with the ARN of `PipelineRole`. + 10. Configure sink details. For **OpenSearch resource type** , choose **Managed cluster**. Then choose the OpenSearch Service domain that you created in the previous section. @@ -219 +201 @@ The `path` option specifies the URI path for ingestion. This option is required - 7. Choose **Validate pipeline** and make sure that the validation succeeds. +For **Index name** , enter **application_logs**. OpenSearch Ingestion automatically creates this index in the domain if it doesn't already exist. @@ -221 +203 @@ The `path` option specifies the URI path for ingestion. This option is required - 8. For simplicity in this tutorial, configure public access for the pipeline. Under **Network** , choose **Public access**. + 11. Choose **Next**. @@ -223 +205 @@ The `path` option specifies the URI path for ingestion. This option is required -For information about configuring VPC access, see [Configuring VPC access for Amazon OpenSearch Ingestion pipelines](./pipeline-security.html). + 12. Name the pipeline **ingestion-pipeline**. Leave the capacity settings as their defaults. @@ -225 +207 @@ For information about configuring VPC access, see [Configuring VPC access for Am - 9. Keep log publishing enabled in case you encounter any issues while completing this tutorial. For more information, see [Monitoring pipeline logs](./monitoring-pipeline-logs.html). + 13. For **Pipeline role** , select **Create and use a new service role**. The pipeline role provides the required permissions for a pipeline to write to the domain sink and read from pull-based sources. By selecting this option, you allow OpenSearch Ingestion to create the role for you, rather than manually creating it in IAM. For more information, see [Setting up roles and users in Amazon OpenSearch Ingestion](./pipeline-security-overview.html). @@ -227 +209 @@ For information about configuring VPC access, see [Configuring VPC access for Am -Specify the following log group name: `/aws/vendedlogs/OpenSearchIngestion/ingestion-pipeline/audit-logs` + 14. For **Service role name suffix** , enter **PipelineRole**. In IAM, the role will have the format `arn:aws:iam::`your-account-id`:role/OpenSearchIngestion-**PipelineRole**`. @@ -229 +211 @@ Specify the following log group name: `/aws/vendedlogs/OpenSearchIngestion/inges - 10. Choose **Next**. Review your pipeline configuration and choose **Create pipeline**. The pipeline takes 5–10 minutes to become active. + 15. Choose **Next**. Review your pipeline configuration and choose **Create pipeline**. The pipeline takes 5–10 minutes to become active. @@ -234 +216 @@ Specify the following log group name: `/aws/vendedlogs/OpenSearchIngestion/inges -## Step 4: Ingest some sample data +## Step 3: Ingest some sample data @@ -246 +228 @@ First, get the ingestion URL from the **Pipeline settings** page: -Then, ingest some sample data. The following request uses [awscurl](https://github.com/okigan/awscurl) to send a single log file to the `application_logs` index: +Then, ingest some sample data. The following request uses [awscurl](https://github.com/okigan/awscurl) to send a single log file to the pipeline: @@ -253 +235 @@ Then, ingest some sample data. The following request uses [awscurl](https://gith - https://pipeline-endpoint.us-east-1.osis.amazonaws.com/log-pipeline/test_ingestion_path + https://pipeline-endpoint.us-east-1.osis.amazonaws.com/logs