AWS mwaa documentation change
Summary
Added documentation for 'Both public and private network access' mode, updated architecture diagrams, and clarified setup requirements for Apache Airflow version 3+ environments.
Security assessment
The changes introduce a new networking mode that enhances security by allowing worker-webserver communication through private VPC endpoints while maintaining public UI access. This addresses potential task failures in isolated networks without internet access, improving the security posture by reducing public exposure of internal communications.
Diff
diff --git a/mwaa/latest/userguide/configuring-networking.md b/mwaa/latest/userguide/configuring-networking.md index 5e7d9ab02..908707f78 100644 --- a//mwaa/latest/userguide/configuring-networking.md +++ b//mwaa/latest/userguide/configuring-networking.md @@ -7 +7 @@ -Apache Airflow access modesAccess modes overviewSetup for private and public access modesAccessing the VPC endpoint for your Apache Airflow webserver (private network access) +Apache Airflow access modesAccess modes overviewSetup for access modesAccessing the VPC endpoint for your Apache Airflow webserver (private network access) @@ -11 +11 @@ Apache Airflow access modesAccess modes overviewSetup for private and public acc -The Amazon Managed Workflows for Apache Airflow console contains built-in options to configure private or public routing to the Apache Airflow webserver on your environment. This guide describes the access modes available for the Apache Airflow webserver on your Amazon Managed Workflows for Apache Airflow environment, and the additional resources you'll need to configure in your Amazon VPC if you choose the private network option. +The Amazon Managed Workflows for Apache Airflow console contains built-in options to configure private, public, or both public and private routing to the Apache Airflow webserver on your environment. This guide describes the access modes available for the Apache Airflow webserver on your Amazon Managed Workflows for Apache Airflow environment, and the additional resources you'll need to configure in your Amazon VPC if you choose the private network option. @@ -20,0 +21,2 @@ The Amazon Managed Workflows for Apache Airflow console contains built-in option + * [Both public and private network access](./configuring-networking.html#webserver-options-public-and-private-network) + @@ -27 +29,3 @@ The Amazon Managed Workflows for Apache Airflow console contains built-in option - * [Setup for private and public access modes](./configuring-networking.html#access-network-choose) + * [Both public and private network access mode](./configuring-networking.html#access-overview-public-and-private) + + * [Setup for access modes](./configuring-networking.html#access-network-choose) @@ -32,0 +37,2 @@ The Amazon Managed Workflows for Apache Airflow console contains built-in option + * [Setup for both public and private network access](./configuring-networking.html#access-network-public-and-private) + @@ -40 +46 @@ The Amazon Managed Workflows for Apache Airflow console contains built-in option -You can choose private or public routing for your Apache Airflow webserver. To enable private routing, choose **Private network**. This limits user access to an Apache Airflow webserver within an Amazon VPC. To enable public routing, choose **Public network**. This allows users to access the Apache Airflow webserver over the internet. +You can choose private, public, or both public and private routing for your Apache Airflow webserver. To enable private routing, choose **Private network**. This limits user access to an Apache Airflow webserver within an Amazon VPC. To enable public routing, choose **Public network**. This allows users to access the Apache Airflow webserver over the internet. To enable both public and private routing, choose **Both public and private network access**. This allows users to access the Apache Airflow webserver over the internet while workers communicate with the webserver through a private VPC endpoint. @@ -46 +52 @@ The following architectural diagram depicts an Amazon MWAA environment with a pu - + @@ -49,0 +56,4 @@ The public network access mode allows the Apache Airflow UI to be accessed over +###### Important + +If your environment uses Apache Airflow version 3 or later with the **Public network** access mode, workers must be able to reach the webserver over the internet to communicate task state. If the subnets hosting your workers do not have internet access (for example, private subnets without a NAT gateway), DAG tasks will fail. To resolve this, upgrade to Apache Airflow version 3.2.1 or later and switch to **Both public and private network access** mode, which routes worker communication through a private VPC endpoint. + @@ -52 +62 @@ The following image depicts where to find the **Public network** option on the A - + @@ -58 +68 @@ The following architectural diagram depicts an Amazon MWAA environment with a pr - + @@ -66 +76,11 @@ The following image depicts where to find the **Private network** option on the - + + +### Both public and private network access + +Available for Apache Airflow version 3.2.1 and later. In Apache Airflow version 3 and later, workers communicate task state to the webserver through the Task API. If your Amazon VPC lacks internet access, workers cannot reach a public webserver, causing DAG tasks to fail. This mode creates both a public network load balancer for browser access to the Apache Airflow UI and a private VPC endpoint for worker-to-webserver communication, allowing workers to reach the webserver without internet access. Refer to the **Public network** and **Private network** architecture diagrams above for each component. + + + +###### Note + +With this mode, browser access to the Apache Airflow UI goes through the public URL. The private VPC endpoint is used by workers for internal communication and is not intended for browser access. @@ -70 +90 @@ The following image depicts where to find the **Private network** option on the -This section describes the VPC endpoints (AWS PrivateLink) created in your Amazon VPC when you choose the **Public network** or **Private network** access mode. +This section describes the VPC endpoints (AWS PrivateLink) created in your Amazon VPC when you choose the **Public network** , **Private network** , or **Both public and private network access** mode. @@ -93,0 +114,13 @@ If you chose the **Private network** access mode for your Apache Airflow webserv +### Both public and private network access mode + +If you chose the **Both public and private network access** mode for your Apache Airflow webserver, network traffic to the Apache Airflow UI is publicly routed over the internet, while worker-to-webserver communication is privately routed within your Amazon VPC. + + * Amazon MWAA creates a VPC interface endpoint for your Apache Airflow webserver (for worker connectivity), and an interface endpoint for your Amazon Aurora PostgreSQL metadata database. The endpoints are created in the Availability Zones mapped to your private subnets and are independent from other AWS accounts. + + * Amazon MWAA then binds an IP address from your private subnets to the interface endpoints. This is designed to support the best practice of binding a single IP from each Availability Zone of the Amazon VPC. + + * The Apache Airflow UI is accessible over the internet via a public network load balancer. Users access the UI the same way as with the **Public network** access mode. + + + + @@ -96 +129 @@ To learn more, refer to [Example use cases for an Amazon VPC and Apache Airflow -## Setup for private and public access modes +## Setup for access modes @@ -129,0 +163,15 @@ You can add permission to use additional actions for these services, or to use o +### Setup for both public and private network access + +If you choose the **Both public and private network access** option for your Apache Airflow webserver, you can begin using the Apache Airflow UI after you create your environment. No VPN or VPC endpoint access mechanism is required for browser access. The Apache Airflow UI is accessible over the internet. Workers connect to the webserver via the private VPC endpoint automatically. + +You'll need to take the following steps to configure access for your users, and permission for your environment to use other AWS services. + + 1. **Add permissions**. Amazon MWAA needs permission to use other AWS services. When you create an environment, Amazon MWAA creates a [service-linked role](./mwaa-slr.html) that allows it to use certain IAM actions for Amazon Elastic Container Registry (Amazon ECR), CloudWatch Logs, and Amazon EC2. + +You can add permission to use additional actions for these services, or to use other AWS services by adding permissions to your execution role. To learn more, refer to [Amazon MWAA execution role](./mwaa-create-role.html). + + 2. **Create user policies**. You might need to create multiple IAM policies for your users to configure access to your environment and Apache Airflow UI. To learn more, refer to [Accessing an Amazon MWAA environment](./access-policies.html). + + + + @@ -133,0 +182,2 @@ If you've chosen the **Private network** option, you'll need to create a mechani +If you've chosen **Both public and private network access** , you do not need to create a mechanism to access the Apache Airflow UI. It is accessible over the internet. The private VPC endpoint is used automatically by workers for internal communication. +