AWS redshift: Redshift enhanced VPC routing docs: required S3/Glue/Lake Formation VPC endpoints
Summary
Rewrites the Redshift Spectrum enhanced VPC routing page to cover both the integrated data lake query engine (RG clusters and Redshift Serverless) and Spectrum (RA3/DC2). Adds required VPC endpoint guidance (Amazon S3 gateway endpoint, AWS Glue interface endpoint, optional Lake Formation interface endpoint), splits engine-specific traffic routing behavior, recommends AWS PrivateLink interface endpoints for Glue over internet/NAT gateways, and clarifies bucket-policy restrictions on VPC endpoints and IAM-role-based authorization.
Security assessment
The added text documents network-boundary and access-control configuration for data lake queries: required S3 gateway and Glue/Lake Formation interface VPC endpoints, PrivateLink-preferred routing over internet/NAT gateways, and how S3 bucket policies restricting to VPC endpoints affect Spectrum versus the in-VPC integrated engine. It also reiterates that access is authorized by the attached IAM role and that traffic is SIGv4-signed and HTTPS-encrypted. This is security hardening/best-practice guidance for network and IAM posture, not a fix for a specific vulnerability or incident, so it is medium severity and not security_issue_related.
Evidence
+Redshift Spectrum on provisioned clusters can't access data stored in Amazon S3 buckets that use a bucket policy that restricts access to only specified VPC endpoints. Instead, use a bucket policy that restricts access to only specific principals, such as a specific AWS account or specific users. The integrated data lake query engine on RG provisioned clusters and Amazon Redshift Serverless can access such buckets when enhanced VPC routing is turned on, because its Amazon S3 access flows through your VPC endpoints.
Diff
diff --git a/redshift/latest/mgmt/spectrum-enhanced-vpc.md b/redshift/latest/mgmt/spectrum-enhanced-vpc.md index df055ca48..c7f448dda 100644 --- a//redshift/latest/mgmt/spectrum-enhanced-vpc.md +++ b//redshift/latest/mgmt/spectrum-enhanced-vpc.md @@ -7 +7 @@ -Permissions policy configuration when using Amazon Redshift Spectrum +Integrated data lake query engine for RG provisioned clusters and Amazon Redshift ServerlessAmazon Redshift Spectrum for RA3 and DC2 provisioned clustersPermissions policy configuration for data lake queries @@ -11 +11 @@ Amazon Redshift will no longer support the use of Python UDFs after June 30, 202 -# Accessing Amazon S3 buckets with Redshift Spectrum +# Querying data lake tables with enhanced VPC routing @@ -13 +13 @@ Amazon Redshift will no longer support the use of Python UDFs after June 30, 202 -In general, Amazon Redshift Spectrum doesn't support enhanced VPC routing with provisioned clusters, even though a provisioned cluster can query external tables from Amazon S3 when enhanced VPC routing is enabled. +When enhanced VPC routing is turned on, Amazon Redshift routes traffic through your VPC. Data lake tables store their data in Amazon S3 and their metadata in the AWS Glue Data Catalog, both of which are outside your VPC. For data lake queries to succeed, your cluster or workgroup must be able to reach Amazon S3 and AWS Glue. If your data lake tables are managed by AWS Lake Formation, it must also be able to reach Lake Formation. Enhanced VPC routing affects the way that Amazon Redshift accesses these external resources, so queries might fail unless you configure your VPC correctly. @@ -15 +15 @@ In general, Amazon Redshift Spectrum doesn't support enhanced VPC routing with p -Amazon Redshift enhanced VPC routing sends specific traffic through your VPC, which means that all traffic between your cluster and your Amazon S3 buckets is forced to pass through your Amazon VPC. Because Redshift Spectrum runs on AWS managed resources that are owned by Amazon Redshift but are outside your VPC, Redshift Spectrum doesn't use enhanced VPC routing. +Create the following VPC endpoints in the VPC and subnets where your cluster or workgroup runs, associating them with the appropriate route tables. These endpoints are required for both the integrated data lake query engine and Amazon Redshift Spectrum: @@ -17 +17 @@ Amazon Redshift enhanced VPC routing sends specific traffic through your VPC, wh -Traffic between Redshift Spectrum and Amazon S3 is securely routed through the AWS private network, outside of your VPC. In-flight traffic is signed using Amazon Signature Version 4 protocol (SIGv4) and encrypted using HTTPS. This traffic is authorized based on the IAM role that is attached to your Amazon Redshift cluster. To further manage Redshift Spectrum traffic, you can modify your cluster's IAM role and your policy attached to the Amazon S3 bucket. You might also need to configure your VPC to allow your cluster to access AWS Glue or Athena, as detailed following. + * **Amazon S3 gateway endpoint** – Gives your cluster or workgroup a route through the AWS network to Amazon S3 to read data lake data files. For steps, see [Gateway endpoints for Amazon S3](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html) in the _Amazon VPC User Guide_. @@ -19 +19 @@ Traffic between Redshift Spectrum and Amazon S3 is securely routed through the A -Note that because enhanced VPC routing affects the way that Amazon Redshift accesses other resources, queries might fail unless you configure your VPC correctly. For more information, see [Controlling network traffic with Redshift enhanced VPC routing](./enhanced-vpc-routing.html), which discusses in more detail creating a VPC endpoint, a NAT gateway, and other networking resources to direct traffic to your Amazon S3 buckets. + * **AWS Glue interface endpoint** – Gives your cluster or workgroup a route to the AWS Glue Data Catalog to resolve data lake schemas and tables. For steps, see [Creating an interface endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html) in the _Amazon VPC User Guide_. @@ -21 +21,6 @@ Note that because enhanced VPC routing affects the way that Amazon Redshift acce -###### Note + * **AWS Lake Formation interface endpoint** – Required only if your data lake tables are managed by AWS Lake Formation. Gives your cluster or workgroup a route to Lake Formation. For steps, see [Creating an interface endpoint](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html) in the _Amazon VPC User Guide_. + + + + +The endpoint configuration is the same regardless of how you query external tables, but the way traffic is routed differs depending on the query engine. The following sections describe the behavior for the integrated data lake query engine on RG provisioned clusters and Amazon Redshift Serverless, and for Redshift Spectrum on RA3 and DC2 provisioned clusters. @@ -23 +28 @@ Note that because enhanced VPC routing affects the way that Amazon Redshift acce -Amazon Redshift Serverless supports enhanced VPC routing for queries to external tables on Amazon S3. For more information about configuration, see [Loading in data from Amazon S3](https://docs.aws.amazon.com/redshift/latest/gsg/new-user-serverless.html#serverless-load-data-from-s3) in the Amazon Redshift Serverless Getting Started Guide. +## Integrated data lake query engine for RG provisioned clusters and Amazon Redshift Serverless @@ -25 +30 @@ Amazon Redshift Serverless supports enhanced VPC routing for queries to external -## Permissions policy configuration when using Amazon Redshift Spectrum +RG provisioned clusters and Amazon Redshift Serverless include an integrated data lake query engine. This engine runs on the cluster's or workgroup's own compute resources, within your VPC. When enhanced VPC routing is turned on and the required VPC endpoints are in place, access to Amazon S3 and AWS Glue originates from your in-VPC compute and flows through those VPC endpoints. This traffic stays within your VPC boundary and on the AWS network. This access is authorized based on the IAM role that is attached to your cluster or workgroup. To further manage this traffic, you can modify the IAM role and the policy attached to the Amazon S3 bucket. @@ -27 +32,9 @@ Amazon Redshift Serverless supports enhanced VPC routing for queries to external -Consider the following when using Redshift Spectrum: +## Amazon Redshift Spectrum for RA3 and DC2 provisioned clusters + +Redshift Spectrum runs on AWS managed resources that are owned by Amazon Redshift but are outside your VPC. As a result, even when enhanced VPC routing is turned on, the traffic that Redshift Spectrum sends to Amazon S3 does not pass through your VPC, but the cluster still requires the Amazon S3 and AWS Glue VPC endpoints described earlier to run data lake queries. + +Traffic between Redshift Spectrum and Amazon S3 is securely routed through the AWS private network, outside of your VPC. In-flight traffic is signed using Amazon Signature Version 4 protocol (SIGv4) and encrypted using HTTPS. This traffic is authorized based on the IAM role that is attached to your Amazon Redshift cluster. To further manage Redshift Spectrum traffic, you can modify your cluster's IAM role and your policy attached to the Amazon S3 bucket. + +## Permissions policy configuration for data lake queries + +Consider the following when querying data lake tables in Amazon S3: @@ -44 +57 @@ You can control access to data in your Amazon S3 buckets by using a bucket polic -Redshift Spectrum on provisioned clusters can't access data stored in Amazon S3 buckets that use a bucket policy that restricts access to only specified VPC endpoints. Instead, use a bucket policy that restricts access to only specific principals, such as a specific AWS account or specific users. +Redshift Spectrum on provisioned clusters can't access data stored in Amazon S3 buckets that use a bucket policy that restricts access to only specified VPC endpoints. Instead, use a bucket policy that restricts access to only specific principals, such as a specific AWS account or specific users. The integrated data lake query engine on RG provisioned clusters and Amazon Redshift Serverless can access such buckets when enhanced VPC routing is turned on, because its Amazon S3 access flows through your VPC endpoints. @@ -133 +146 @@ For more information, see the AWS Security blog post [How to Use Bucket Policies -Redshift Spectrum accesses your data catalog in AWS Glue or Athena. Another option is to use a dedicated Hive metastore for your data catalog. +Both Redshift Spectrum and the integrated data lake query engine access your data catalog in AWS Glue or Athena. Another option is to use a dedicated Hive metastore for your data catalog. @@ -135 +148 @@ Redshift Spectrum accesses your data catalog in AWS Glue or Athena. Another opti -To enable access to AWS Glue or Athena, configure your VPC with an internet gateway or NAT gateway. Configure your VPC security groups to allow outbound traffic to the public endpoints for AWS Glue and Athena. Alternatively, you can configure an interface VPC endpoint for AWS Glue to access your AWS Glue Data Catalog. When you use a VPC interface endpoint, communication between your VPC and AWS Glue is conducted within the AWS network. For more information, see [Creating an Interface Endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint). +The recommended way to reach the AWS Glue Data Catalog is to create an interface VPC endpoint (AWS PrivateLink) for AWS Glue. When you use a VPC interface endpoint, communication between your VPC and AWS Glue is routed within the AWS network. For more information, see [Creating an Interface Endpoint](https://docs.aws.amazon.com/vpc/latest/userguide/vpce-interface.html#create-interface-endpoint). @@ -137 +150 @@ To enable access to AWS Glue or Athena, configure your VPC with an internet gate -You can configure the following pathways in your VPC: +Alternatively, to reach the public endpoints for AWS Glue and Athena, configure your VPC with an internet gateway or NAT gateway, and configure your VPC security groups to allow outbound traffic to those public endpoints. You can configure the following pathways in your VPC: @@ -146,2 +158,0 @@ You can configure the following pathways in your VPC: -For more information, see [Controlling network traffic with Redshift enhanced VPC routing](./enhanced-vpc-routing.html). -