AWS lambda documentation change
Summary
Restructured documentation to outline setup steps, added IAM security context, removed idempotency warning, and added permissions-related sections
Security assessment
Added explicit references to IAM security configurations and execution role permissions, which are security features. The changes improve security documentation but do not address a specific disclosed vulnerability.
Diff
diff --git a/lambda/latest/dg/with-msk.md b/lambda/latest/dg/with-msk.md index 118168505..6da1fa8e4 100644 --- a//lambda/latest/dg/with-msk.md +++ b//lambda/latest/dg/with-msk.md @@ -9 +9 @@ Example event -###### Note +[Amazon Managed Streaming for Apache Kafka (Amazon MSK)](https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html) is a fully-managed service that you can use to build and run applications that use Apache Kafka to process streaming data. Amazon MSK simplifies the setup, scaling, and management of Kafka clusters. Amazon MSK also makes it easier to configure your application for multiple Availability Zones and for security with AWS Identity and Access Management (IAM). @@ -11 +11 @@ Example event -If you want to send data to a target other than a Lambda function or enrich the data before sending it, see [ Amazon EventBridge Pipes](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes.html). +This chapter explains how to use an Amazon MSK cluster as an event source for your Lambda function. The general process for integrating Amazon MSK with Lambda involves the following steps: @@ -13 +13 @@ If you want to send data to a target other than a Lambda function or enrich the -[Amazon Managed Streaming for Apache Kafka (Amazon MSK)](https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html) is a fully managed service that you can use to build and run applications that use Apache Kafka to process streaming data. Amazon MSK simplifies the setup, scaling, and management of clusters running Kafka. Amazon MSK also makes it easier to configure your application for multiple Availability Zones and for security with AWS Identity and Access Management (IAM). Amazon MSK supports multiple open-source versions of Kafka. + 1. **[Cluster and network setup](./with-msk-cluster-network.html)** – First, set up your [Amazon MSK cluster](https://docs.aws.amazon.com/msk/latest/developerguide/what-is-msk.html). This includes the correct networking configuration to allow Lambda to access your cluster. @@ -15 +15 @@ If you want to send data to a target other than a Lambda function or enrich the -Amazon MSK as an event source operates similarly to using Amazon Simple Queue Service (Amazon SQS) or Amazon Kinesis. Lambda internally polls for new messages from the event source and then synchronously invokes the target Lambda function. Lambda reads the messages in batches and provides these to your function as an event payload. The maximum batch size is configurable (the default is 100 messages). For more information, see [Batching behavior](./invocation-eventsourcemapping.html#invocation-eventsourcemapping-batching). + 2. **[Event source mapping setup](./with-msk-configure.html)** – Then, create the [event source mapping](./invocation-eventsourcemapping.html) resource that Lambda needs to securely connect your Amazon MSK cluster to your function. @@ -17 +17 @@ Amazon MSK as an event source operates similarly to using Amazon Simple Queue Se -By default, Lambda [autoscales the number of event pollers](./with-msk-process.html#services-msk-default-mode) for your Amazon MSK event source mapping. To optimize the throughput of your Amazon MSK event source mapping, configure provisioned mode. In provisioned mode, you can define the minimum and maximum number of event pollers allocated to your event source mapping. This can improve the ability of your event source mapping to handle unexpected message spikes. For more information, see [provisioned mode](./with-msk-process.html#services-msk-provisioned-mode). + 3. **[Function and permissions setup](./with-msk-permissions.html)** – Finally, ensure that your function is correctly set up, and has the necessary permissions in its [execution role](./lambda-intro-execution-role.html). @@ -19 +18,0 @@ By default, Lambda [autoscales the number of event pollers](./with-msk-process.h -###### Warning @@ -21 +19,0 @@ By default, Lambda [autoscales the number of event pollers](./with-msk-process.h -Lambda event source mappings process each event at least once, and duplicate processing of records can occur. To avoid potential issues related to duplicate events, we strongly recommend that you make your function code idempotent. To learn more, see [How do I make my Lambda function idempotent](https://repost.aws/knowledge-center/lambda-function-idempotent) in the AWS Knowledge Center. @@ -23 +21,2 @@ Lambda event source mappings process each event at least once, and duplicate pro -For an example of how to configure Amazon MSK as an event source, see [Using Amazon MSK as an event source for AWS Lambda](https://aws.amazon.com/blogs/compute/using-amazon-msk-as-an-event-source-for-aws-lambda/) on the AWS Compute Blog. For a complete tutorial, see [ Amazon MSK Lambda Integration](https://amazonmsk-labs.workshop.aws/en/msklambda.html) in the Amazon MSK Labs. + +For examples on how to set up a Lambda integration with an Amazon MSK cluster, see [Tutorial: Using an Amazon MSK event source mapping to invoke a Lambda function](./services-msk-tutorial.html), [Using Amazon MSK as an event source for AWS Lambda](https://aws.amazon.com/blogs/compute/using-amazon-msk-as-an-event-source-for-aws-lambda/) on the AWS Compute Blog, and [ Amazon MSK Lambda Integration](https://amazonmsk-labs.workshop.aws/en/msklambda.html) in the Amazon MSK Labs. @@ -28,0 +28,2 @@ For an example of how to configure Amazon MSK as an event source, see [Using Ama + * [Configuring your Amazon MSK cluster and Amazon VPC network for Lambda](./with-msk-cluster-network.html) + @@ -31 +32 @@ For an example of how to configure Amazon MSK as an event source, see [Using Ama - * [Processing Amazon MSK messages with Lambda](./with-msk-process.html) + * [Configuring Lambda execution role permissions](./with-msk-permissions.html) @@ -89 +90 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Troubleshoot +Apache Kafka @@ -91 +92 @@ Troubleshoot -Configure event source +Cluster and network setup