AWS step-functions documentation change
Summary
Added troubleshooting section for Distributed Maps feature covering quota limits, deadlock risks in nested workflows, and monitoring solutions
Security assessment
The changes document operational issues like quota limits and deadlock risks in workflow design. No security vulnerabilities, exploits, or security features are mentioned. The content focuses on performance and scalability troubleshooting.
Diff
diff --git a/step-functions/latest/dg/troubleshooting.md b/step-functions/latest/dg/troubleshooting.md index a04e11994..7a9dd5364 100644 --- a//step-functions/latest/dg/troubleshooting.md +++ b//step-functions/latest/dg/troubleshooting.md @@ -7 +7 @@ -General issuesService integrationsActivitiesExpress workflows +General issuesService integrationsActivitiesExpress workflowsDistributed Maps @@ -26,0 +27,2 @@ For more troubleshooting advice and answers to common support questions, visit t + * Distributed Maps + @@ -188,0 +191,12 @@ To list failed and cancelled executions: +## Troubleshooting Distributed Maps + +### My map run is stuck. + +There is a service quota for the [maximum number of open map runs](https://docs.aws.amazon.com/step-functions/latest/dg/service-quotas.html#service-limits-accounts). Once this quota is exhausted, additional map runs wait for more quota to become available as other map runs complete. Backlogged map runs wait at the `MapRunStarted` event, before reading their input, so you will see the map run's pending items count remain at 0. Once a backlogged map run is able to acquire quota then it will begin reading its input. You will then see the pending items count increase and the map run will continue normally for the rest of its lifecycle. As quota becomes available, there is no particular order for how it is allocated among the backlogged map runs. + +There is a risk of deadlock when using nested map runs. In this situation you have a parent workflow execution containing a "parent" map run, which in turn starts a child workflow execution containing a "child" map run, which in turn starts a grandchild workflow execution. If no _maximum number of open map runs_ quota remains when the child workflow execution starts its child map run then that child map run will be backlogged. If the entire quota is being consumed by parent map runs, and all the child map runs are backlogged, then neither group will be able to make progress and they will remain deadlocked until quota becomes available again. A map run will abort and make its quota available again when its parent execution terminates, such as by timing out, failing due to an uncaught error, or due to a `StopExecution` API call. + +When considering using nested map runs in your workflow design, estimate if your expected number of open map runs could exceed the _maximum number of open map runs_ quota and put you at risk of delays or deadlock. If so, you can consider alternative designs. For example, instead of using a parent map run to iterate over a list of Amazon S3 objects and a child map run to read their contents, you could instead use the [`LOAD_AND_FLATTEN` transformation](https://docs.aws.amazon.com/step-functions/latest/dg/input-output-itemreader.html#itemreader-flatten) in your map run's `ReaderConfig` to perform both of those operations in a single map run. Another common pattern is to include a preprocessing step, such as a Lambda function, which iterates over your nested data and produces a file that can be used as the map run's input. + +You can [monitor your usage](https://docs.aws.amazon.com/step-functions/latest/dg/procedure-cw-metrics.html#resource-count-metrics-map-run) of the _maximum number of open map runs_ quota using the `ApproximateOpenMapRunCount`, `OpenMapRunLimit`, and `ApproximateMapRunBacklogSize` metrics in CloudWatch. +