AWS IDR medium security documentation change
Summary
Restructured documentation to add two integration methods (CloudFormation and Manual), including detailed prerequisites, deployment steps, testing procedures, and event bus sharing instructions. Added payload transformation examples and security logging recommendations.
Security assessment
The change explicitly recommends removing logger.info() statements to prevent sensitive payloads from appearing in CloudWatch Logs ('After testing... prevent the payload from appearing in Amazon CloudWatch Logs'). This directly addresses a security best practice for avoiding unintended data exposure.
Diff
diff --git a/IDR/latest/userguide/idr-gs-ingest_alarms_from_apm_to_eventbridge.md b/IDR/latest/userguide/idr-gs-ingest_alarms_from_apm_to_eventbridge.md index 3bc0e6627..0b36950f0 100644 --- a//IDR/latest/userguide/idr-gs-ingest_alarms_from_apm_to_eventbridge.md +++ b//IDR/latest/userguide/idr-gs-ingest_alarms_from_apm_to_eventbridge.md @@ -7 +7 @@ -# Ingest alarms from APMs that have direct integration with Amazon EventBridge +Option 1: Using CloudFormationOption 2: Manual integration @@ -9 +9 @@ -The following illustration shows the process for sending notifications to AWS Incident Detection and Response from Application Performance Monitoring (APM) tools that have direct integration with Amazon EventBridge, such as Datadog and Splunk. For a complete list of APMs that have direct integration with EventBridge, see [Amazon EventBridge integrations](https://aws.amazon.com/eventbridge/integrations). +# Ingest Alarms from APMs with direct EventBridge integration @@ -11 +11 @@ The following illustration shows the process for sending notifications to AWS In -To learn more about using the Incident Detection and Response Command Line Interface (CLI) to help automate these steps, see [AWS Incident Detection and Response CLI](./idr-cli.html). +The following topic shows the process for sending alarms to AWS Incident Detection and Response from Application Performance Monitoring (APM) tools that have direct integration with Amazon EventBridge. For a complete list of APMs that have direct integration with Amazon EventBridge, see [Amazon EventBridge integrations](https://aws.amazon.com/eventbridge/integrations). @@ -13 +13 @@ To learn more about using the Incident Detection and Response Command Line Inter - +You can deploy the provided [CloudFormation template](https://dcl74d3hc5lj1.cloudfront.net/apms/ThirdPartyApmSaasEventBridgeIntegration.json) or manually setup this integration. Before setting up the integration, verify that the AWS service-linked role (SLR) `AWSServiceRoleForHealth_EventProcessor`, is [created](https://docs.aws.amazon.com/IDR/latest/userguide/idr-gs-access-prov.html) in your accounts. @@ -15 +15 @@ To learn more about using the Incident Detection and Response Command Line Inter -Use the following steps to set up integration with AWS Incident Detection and Response. Before performing these steps, verify that the AWS service-linked role (SLR) `AWSServiceRoleForHealth_EventProcessor`, is [installed](./idr-gs-access-prov.html) in your accounts. +## Option 1: Using CloudFormation @@ -17 +17 @@ Use the following steps to set up integration with AWS Incident Detection and Re -###### Set up integration with AWS Incident Detection and Response +A CloudFormation template is available to simplify the process of creating the integration infrastructure required to ingest alarms to AWS Incident Detection and Response from your APM with Amazon EventBridge integration. @@ -19 +19 @@ Use the following steps to set up integration with AWS Incident Detection and Re -You must complete the following steps for each AWS account and AWS Region. Alerts must come from the AWS account and AWS Region where the application resources reside. +###### Note @@ -21 +21 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 1. Set up each of your APMs as Amazon EventBridge partner event sources (for example, `aws.partner/my_apm/integrationName`). For guidelines on setting up your APM as an event source, see [Receiving events from a SaaS partner with Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-saas.html). This creates a partner event bus in your account. + * Additional costs are incurred for resources deployed through this CloudFormation template (eg: Lambda and EventBridge). For more information about the pricing of these services, see [AWS Pricing](https://aws.amazon.com/pricing/). @@ -23 +23 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 2. Do one of the following: + * Deploy this CloudFormation template in every AWS account and Region where AWS Incident Detection and Response needs to ingest alarms. Incidents and Support Cases are opened on the AWS Account where the APM alert was received from. @@ -25 +25 @@ You must complete the following steps for each AWS account and AWS Region. Alert - * (Recommended method) Create a custom EventBridge event bus. AWS Incident Detection and Response installs a managed rule (`AWSHealthEventProcessorEventSource-DO-NOT-DELETE`) bus through the `AWSServiceRoleForHealth_EventProcessor` SLR. The rule source is the custom event bus. The rule destination is AWS Incident Detection and Response. The rule matches the pattern for ingesting 3rd party APM events. + * This document uses New Relic as an example, however the CloudFormation template can be used for any APM that has [SaaS integration with Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-saas.html). @@ -27 +27 @@ You must complete the following steps for each AWS account and AWS Region. Alert - * (Alternative method) Use the default event bus instead of a custom event bus. The default event bus requires the managed rule to send APM alerts to AWS Incident Detection and Response. + * After testing the integration, remove logger.info() statements from the `TransformLambdaFunction` to prevent the payload from appearing in Amazon CloudWatch Logs. @@ -29 +28,0 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 3. Create an [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) function (for example, `My_APM-AWSIncidentDetectionResponse-LambdaFunction`) to transform your partner event bus events. The transformed events matches the managed rule `AWSHealthEventProcessorEventSource-DO-NOT-DELETE`. @@ -31 +29,0 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 1. Transformed events include a unique AWS Incident Detection and Response identifier, and sets the source and detail type of the event to the required values. The pattern matches the managed rule. @@ -33 +30,0 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 2. Set the target of the Lambda function to either the custom event bus created in Step 2 (Recommended method) or to your default event bus. @@ -35 +32 @@ You must complete the following steps for each AWS account and AWS Region. Alert - 4. Create an EventBridge rule and define the event patterns that match the list of events that you want to push to AWS Incident Detection and Response. The source of the rule is the partner event bus that you define in step 1 (for example, aws.partner/my_apm/integrationName). The target of the rule is the Lambda function that you define in step 3 (for example, `My_APM-AWSIncidentDetectionResponse-LambdaFunction`). For guidlines on defining your EventBridge rule, see [Amazon EventBridge rules](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rules.html). +**Prerequisites for deploying this CloudFormation template:** @@ -36,0 +34 @@ You must complete the following steps for each AWS account and AWS Region. Alert + * A Partner Event source must be setup in Amazon EventBridge. For instructions on setting up your APM as an event source, see [Receiving events from a SaaS partner with Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-saas.html) in the _Amazon EventBridge User Guide_. @@ -37,0 +36 @@ You must complete the following steps for each AWS account and AWS Region. Alert + * The `TransformLambdaFunction` (Lambda function) in the template must be modified to set `["detail"]["incident-detection-response-identifier"]` to the desired value based on the JSON path of the alert name in the APM payload. @@ -40 +39,138 @@ You must complete the following steps for each AWS account and AWS Region. Alert -For examples on how to set up a partner event bus integration for use with AWS Incident Detection and Response, see [Example: Integrate notifications from Datadog and Splunk](./example_integrating_notifications.html). + + +**Prerequisite Steps:** + + 1. Open the EventBridge Console. Under the **Integration** menu, select **Partner event sources**. + + * Search for your APM in the Amazon EventBridge partners box. + + * Choose **Setup** , then follow the instructions provided. + + * **Note:** the last step is to choose **Associate with Event Bus** in the console for the Partner event source. Selecting this option automatically create a Partner Event Bus with the same name as the Partner event source (the names must match). + + * Copy the name of the Partner Event Bus or source. The Event Bus or source is used as a parameter, named `PartnerEventBusNameParameter`, when deploying the CloudFormation template. + + * **Example** for New Relic: `aws.partner/newrelic.com/1234567/source_name` + + * Copy the first part of the Partner Event Bus or source to input into the `PartnerEventBusPrefixParameter` when deploying the CloudFormation template. + + * Example for New Relic is `aws.partner/newrelic.com` + + 2. Download and edit the [CloudFormation template](https://dcl74d3hc5lj1.cloudfront.net/apms/ThirdPartyApmSaasEventBridgeIntegration.json). + + * Locate the `TransformLambdaFunction` in the template + + * Under `def lambda_handler(event, context)` set `event["detail"]["incident-detection-response-identifier"]` to the json path where alarm name appears in the JSON payload of the APM alarm. Every APM will have a different path. Some examples can be seen below, however your specific payloads may differ. + + * **New Relic Example:** `event["detail"]["incident-detection-response-identifier"] = event["detail"]["workflowName"]`. + + * **Datadog Example:** `event["detail"]["incident-detection-response-identifier"] = event["detail"]["meta"]["monitor"]["name"]` + + * **Splunk Example:** `event["detail"]["incident-detection-response-identifier"] = event["detail"]["ruleName"]` + + * Save the CloudFormation template. + + + + +**Deploying the CloudFormation Template:** + + 1. Open the CloudFormation console in your target account and Region. + + 2. Choose Create stack, With new resources (standard) + + * Select **Choose an existing template** , **Upload a template file** , **Choose file** , then upload the CloudFormation template you saved locally. + + 3. Specify stack details: + + * Enter a stack name (_Example:_ `NewRelicIntegrationForIDR`). + + * Specify the **Parameter values** obtained during **Prerequisite completion**. + + * APMNameParameter (_Example_ : `NewRelic`) + + * PartnerEventBusNameParameter (_Example_ : `aws.partner/newrelic.com/1234567/source_name`) + + * PartnerEventBusPrefixParameter (_Example_ : `aws.partner/newrelic.com`) + + * Choose **Next**. + + 4. Configure stack options: + + * Scroll to the bottom of the page and check the box to allow CloudFormation to create IAM resources with custom names. + + 5. Review and create: + + * Validate the parameter values are configured correctly and choose **Submit**. + + 6. The CloudFormation stack deploys the resources necessary to integrate your APM events to AWS Incident Detection and Response. Wait for the stack status to show `CREATE_COMPLETE`. + + 7. The CloudFormation stack creates the following resources, assuming the example values were input into the parameters for New Relic and was run in the US-EAST-1 Region. + + * CustomEventBus: NewRelic-AWSIncidentDetectionResponse-EventBus + + * EventBridgeRule: aws.partner/newrelic.com/1234567/source_name|NewRelic-AWSIncidentDetectionResponse-EventBridgeRule + + * TransformLambdaExecutionRole: IDR-TransformLambdaExecutionRole-us-east-1 + + * TransformLambdaFunction: NewRelic-AWSIncidentDetectionResponse-Lambda-Transform + + * TransformLambdaPermission: NewRelicIntegrationForIDR-TransformLambdaPermission-[random_string] + + + + +**Integration testing** + +After deploying the stack, test the integration by sending a test payload from your APM: + + 1. Navigate to the Lambda Console and select the `APMNameParameter-AWSIncidentDetectionResponse-Lambda-Transform` function. Choose the **Monitor** tab. + + 2. Look for a successful invocation in the metric graphs. + + 3. Choose **View Amazon CloudWatch Logs** to check Log streams for your test payload or any errors. + + + + +**Sharing Your Event Bus ARN to AWS Incident Detection and Response** + + 1. Open the Amazon EventBridge Console. Select **Event buses**. + + 2. Copy the ARN of the **Custom event bus** created as part of the CloudFormation stack, (_example:`arn:aws:events:us-east-1:123456789123:event-bus/NewRelic-AWSIncidentDetectionResponse-EventBus`_.) + + * Add this ARN to the "EventBridge Event Bus ARN" field in the "Third-Party APM Alarms" section of your [Alarm ingestion questionnaire - Overview](./idr-gs-questionnaire.html#idr-gs-alarm-questionnaire). + + 3. During the onboarding process, AWS Incident Detection and Response creates a managed EventBridge rule on this custom event bus to ingest your APM alarms. + + + + +## Option 2: Manual integration + + + +Complete the following steps for each AWS account and AWS Region where AWS Incident Detection and Response needs to ingest alarms from. AWS Incident Detection and Response recommends to set up alarms in the same AWS account and Region as your application resources to make it quicker to identify and investigate impacted resources. Incidents and Support Cases are opened on the AWS Account where the APM alert was received from. + + 1. Create an EventBridge partner event bus by setting up your APM as an Amazon EventBridge partner event source (for example, `aws.partner/apm_name/integrationName`). For guidelines on setting up your APM as an event source, see [Receiving events from a SaaS partner with Amazon EventBridge](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-saas.html). + + 2. Perform one of the following: + + * (Recommended) Create an EventBridge custom event bus named `$YourApmName-AWSIncidentDetectionResponse-EventBus`. + + * (Alternative) Use the default EventBridge event bus instead of a custom event bus. + +AWS Incident Detection and Response will install a managed rule (`AWSHealthEventProcessorEventSource-DO-NOT-DELETE`) on the custom or default event bus through the `AWSServiceRoleForHealth_EventProcessor` SLR. The rule source will be the custom or default event bus, the rule destination will be AWS Incident Detection and Response, and the rule will match the pattern for ingesting 3rd party APM events. + + 3. Create an [Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) function named `$YourApmName-AWSIncidentDetectionResponse-LambdaFunction` to transform your partner event bus events. The transformed events will match the managed rule `AWSHealthEventProcessorEventSource-DO-NOT-DELETE`. + + * Transformed events include a unique AWS Incident Detection and Response identifier, and sets the source and detail type of the event to the required values. This allows the transformed JSON payload structure to match the managed rule pattern. + + * Set the target of the Lambda function to either the custom event bus (Recommended) created in Step 2 or to your default event bus. + + 4. Create an EventBridge rule and define the event patterns that match the list of events that you want to push to AWS Incident Detection and Response. The source of the rule is the partner event bus you created in Step 1 (`aws.partner/apm_name/integrationName`). The target of the rule is the Lambda function you created in Step 3 (`[apm_name]-AWSIncidentDetectionResponse-LambdaFunction`). For guidelines on defining your EventBridge rule, see [Amazon EventBridge rules](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rules.html). + + + + +For a step by step example on how to set up partner event bus integrations manually with AWS Incident Detection and Response, see [Integrating notifications from Datadog and Splunk](https://docs.aws.amazon.com/IDR/latest/userguide/example_integrating_notifications.html). @@ -48 +184 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Integrate with CloudWatch +Payload requirements @@ -50 +186 @@ Integrate with CloudWatch -Example: Integrating notifications from Datadog and Splunk +Webhook integration