AWS solutions documentation change
Summary
Restructured troubleshooting guide with clearer section headers, added detailed resolution steps for StackSet deployment errors, ECS memory issues, Gremlin Lambda timeouts, and ECR access problems. Added new sections for VPC endpoint configuration requirements.
Security assessment
Changes focus on improving troubleshooting clarity and adding operational resolution steps. No explicit security vulnerabilities or weaknesses are addressed. The ECR VPC endpoint guidance relates to network configuration but does not introduce new security features or mitigate documented vulnerabilities.
Diff
diff --git a/solutions/latest/workload-discovery-on-aws/troubleshooting.md b/solutions/latest/workload-discovery-on-aws/troubleshooting.md index a96f8d941..46d785ade 100644 --- a//solutions/latest/workload-discovery-on-aws/troubleshooting.md +++ b//solutions/latest/workload-discovery-on-aws/troubleshooting.md @@ -59 +59 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f -**Reason:** The most likely reasons are as follows, +#### Global resources template not deployed @@ -61 +61 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 1. When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the global resources CloudFormation template has not been deployed. +**Reason:** When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the global resources CloudFormation template has not been deployed. @@ -63 +63 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 2. When the **CrossAccountDiscovery** CloudFormation parameter is set to `AWS_ORGANIZATIONS`: one or more accounts is not discovered and the **Role Status** column has **Not Deployed** entries. This means there has been a problem with the automated deployment of the global resources template using StackSets. +**Resolution:** Deploy the global resources template in the required accounts, as per the [documentation](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/import-a-region.html#deploy-the-aws-cloudformation-templates). @@ -65 +65 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 3. The discovery process ECS task is running out of memory. This is happens when importing a large number of accounts or resources. The **Last Discovered** column in the UI will have a value larger than the one specified in the **DiscoveryTaskFrequency** CloudFormation parameter (the default value is 15 minutes) and there will be an out of memory error in the ECS console. +#### StackSet deployment error @@ -66,0 +67 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f +**Reason:** When the **CrossAccountDiscovery** CloudFormation parameter is set to `AWS_ORGANIZATIONS`: one or more accounts is not discovered and the **Role Status** column has **Not Deployed** entries. This means there has been a problem with the automated deployment of the global resources template using StackSets. @@ -67,0 +69,15 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f +**Resolution:** Go to the **WdGlobalResources** StackSet in the region that Workload Discovery has been deployed to and check the errors in the stack instances that have failed to deploy: + + 1. Sign in to the [AWS CloudFormation console](https://console.aws.amazon.com/cloudformation/home?). + + 2. From the navigation menu, select **StackSets**. + + 3. Select the **Service-managed** tab. + + 4. In the search bar, search for `WdGlobalResources`. + + 5. Choose `WdGlobalResources` from the search results. + + 6. Select the **Stack Instances** tab. + + 7. Inspect the **Detailed status** column for any errors. @@ -70 +85,0 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f -**Resolution:** @@ -72 +86,0 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 1. Deploy the global resources template in the required accounts, as per the [documentation](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/import-a-region.html#deploy-the-aws-cloudformation-templates). @@ -74 +88 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 2. Go to **WdGlobalResources** StackSet in the region that Workload Discovery has been deployed to and check the errors in the stack instances that have failed to deploy. +#### Discovery ECS task out of memory @@ -76 +90 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f - 3. Update the **Memory** CloudFormation parameter to a larger value: start with double and keep increasing until the error stops. +**Reason:** The discovery process ECS task is running out of memory. This can happen when importing a large number of accounts or resources. The **Last Discovered** column in the UI will display **Not Discovered** or have a value larger than the one specified in the **DiscoveryTaskFrequency** CloudFormation parameter (the default value is 15 minutes). There will be an out of memory error in the ECS console. To verify, follow these steps: @@ -77,0 +92 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f + 1. Sign in to the [Amazon Elastic Container Service console](https://console.aws.amazon.com/ecs/home). @@ -78,0 +94 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f + 2. Select the cluster named workload-discovery-cluster. @@ -79,0 +96,10 @@ To help make it easier to troubleshoot, we recommend disabling the rollback on f + 3. Choose the Tasks tab. + + 4. Select the Stopped button in the Desired task status panel. + + 5. In the **Last Status** column check for the error message `OutOfMemoryError: Container killed due to memory usage`. + + + + +**Resolution:** Update the **Memory** CloudFormation parameter to a larger value: start with double and keep increasing until the error stops. @@ -85 +111 @@ Only certain combination of CPU units and memory values are valid so you may hav -### Only Non-AWS Config Resources Are Being Discovered In Specific Accounts +### Gremlin Lambda times out when connecting to AWS Neptune @@ -87 +113 @@ Only certain combination of CPU units and memory values are valid so you may hav -**Issue:** The only resource types that the solution discovers are the ones listed in the table on the [Supported resources](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/supported-resources-1.html) section. +**Issue:** GraphQL queries backed by the `<stack-name>-GremlinResol-GremlinAppSyncFunction-<ID-string>` lambda function timeout when attempting to connect to the AWS Neptune database. @@ -89 +115 @@ Only certain combination of CPU units and memory values are valid so you may hav -**Reason:** The most common causes of this issue are, +**Reason:** The VPC that the database is running has a custom DNS configuration. @@ -91 +117 @@ Only certain combination of CPU units and memory values are valid so you may hav - 1. When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the regional resources CloudFormation template has not been deployed in the regions of each account to be discover. +**Resolution:** Update the security group associated with the `<stack-name>-GremlinResol-GremlinAppSyncFunction-<ID-string>` lambda function to open port 53 for the UDP protocol. @@ -93 +119 @@ Only certain combination of CPU units and memory values are valid so you may hav - 2. When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the regional resources CloudFormation template has been deployed in the regions of a number of accounts that did not have Config enabled but the CloudFormation parameter **AlreadyHaveConfigSetup** was erroneously set to `Yes`. +### Unable to access Elastic Container Registry @@ -95 +121 @@ Only certain combination of CPU units and memory values are valid so you may hav - 3. When the **CrossAccountDiscovery** CloudFormation parameter is set to `AWS_ORGANIZATIONS`, AWS Config is not enabled in the regions of each account to be discovered. In `AWS_ORGANIZATIONS` mode, you are responsible for enabling Config as per your organization’s policies. +**Issue:** When the scheduled Amazon ECS task on Fargate is launched, the task fails with the following error: @@ -97,0 +124 @@ Only certain combination of CPU units and memory values are valid so you may hav + ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth @@ -98,0 +126 @@ Only certain combination of CPU units and memory values are valid so you may hav +**Reason:** The ECS task is is running in a VPC that does not have a route to the to the ECR API endpoint. @@ -100 +128,22 @@ Only certain combination of CPU units and memory values are valid so you may hav -**Resolution:** +**Resolution:** Add a VPC endpoint for the `com.amazonaws.<region>.ecr.api` route as per [the ECR documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html#ecr-setting-up-vpc-create). + +### Unable to pull container from Elastic Container Registry + +**Issue:** When the scheduled Amazon ECS task on Fargate is launched, the task fails with the following error: + + + CannotPullContainerFromRegistry: There is a connection issue between the task and Amazon ECR. Check your task network configuration + +**Reason:** The ECS task is is running in a VPC that does not have a route to the ECR Docker endpoint. + +**Resolution:** Add a VPC endpoint for the ``com.amazonaws.<region>.ecr.dkr` route as per [the ECR documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html#ecr-setting-up-vpc-create). + +### Only Non-AWS Config Resources Are Being Discovered In Specific Accounts + +**Issue:** The only resource types that the solution discovers are the ones listed in the table on the [Supported resources](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/supported-resources-1.html) section. + +#### Regional resources template not deployed + +**Reason:** When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the regional resources CloudFormation template has not been deployed in the regions of each account to be discovered. + +**Resolution:** Deploy the regional resources templates in the required accounts, as per the [documentation](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/import-a-region.html#deploy-the-aws-cloudformation-templates). @@ -102 +151 @@ Only certain combination of CPU units and memory values are valid so you may hav - 1. Deploy the regional resources templates in the required accounts, as per the [documentation](https://docs.aws.amazon.com/solutions/latest/workload-discovery-on-aws/import-a-region.html#deploy-the-aws-cloudformation-templates). +#### Regional resources template deployed incorrectly @@ -104 +153 @@ Only certain combination of CPU units and memory values are valid so you may hav - 2. Delete the previous deployed regional resources stack (AWS Config will be in an inconsistent state otherwise) and re-deploy with the CloudFormation parameter **AlreadyHaveConfigSetup** set to `No`. +**Reason:** When the **CrossAccountDiscovery** CloudFormation parameter is set to `SELF_MANAGED`, the regional resources CloudFormation template has been deployed in the regions of a number of accounts that did not have Config enabled but the CloudFormation parameter **AlreadyHaveConfigSetup** was erroneously set to `Yes`. @@ -106 +155 @@ Only certain combination of CPU units and memory values are valid so you may hav - 3. Enable AWS Config in the regions of each account to be discovered. +**Resolution:** Delete the previous deployed regional resources stack (AWS Config will be in an inconsistent state otherwise) and re-deploy with the CloudFormation parameter **AlreadyHaveConfigSetup** set to `No`. @@ -107,0 +157 @@ Only certain combination of CPU units and memory values are valid so you may hav +#### Config not enabled in required regions @@ -108,0 +159 @@ Only certain combination of CPU units and memory values are valid so you may hav +**Reason:** When the **CrossAccountDiscovery** CloudFormation parameter is set to `AWS_ORGANIZATIONS`, AWS Config is not enabled in the regions of each account to be discovered. In `AWS_ORGANIZATIONS` mode, you are responsible for enabling Config as per your organization’s policies. @@ -109,0 +161 @@ Only certain combination of CPU units and memory values are valid so you may hav +**Resolution:** Enable AWS Config in the regions of each account to be discovered.