AWS athena documentation change
Summary
Updated Athena Federated Query documentation to introduce two connector types (AWS Glue Data Catalog federated connectors and Athena data catalog federated connectors), added a detailed table of connector type support by data source, removed the 'Permissions required' section, and added information about future changes where certain connectors won't require Lambda functions after April 21, 2026.
Security assessment
The changes are primarily organizational and informational, focusing on connector architecture and support matrix. While the documentation mentions that connectors can restrict access based on the user (a security feature), this is not new security documentation but rather a description of existing functionality. The removal of the 'Permissions required' section actually reduces security documentation rather than adding it. There is no evidence of addressing a specific security vulnerability or incident.
Diff
diff --git a/athena/latest/ug/federated-queries.md b/athena/latest/ug/federated-queries.md index 4f38be7fa..614084abc 100644 --- a//athena/latest/ug/federated-queries.md +++ b//athena/latest/ug/federated-queries.md @@ -7 +7 @@ -Considerations and limitationsPermissions requiredVideos +Considerations and limitationsConnector type support by data sourceVideos @@ -11 +11 @@ Considerations and limitationsPermissions requiredVideos -If you have data in sources other than Amazon S3, you can use Athena Federated Query to query the data in place or build pipelines that extract data from multiple data sources and store them in Amazon S3. With Athena Federated Query, you can run SQL queries across data stored in relational, non-relational, object, and custom data sources. +If you have data in sources other than Amazon S3, you can use Athena Federated Query to query the data in place or build pipelines that extract data from multiple data sources and store the data in Amazon S3. With Athena Federated Query, you can run SQL queries across data stored in relational, non-relational, object, and custom data sources. For a full list of supported data sources, see [Available data source connectors](./connectors-available.html). @@ -13 +13 @@ If you have data in sources other than Amazon S3, you can use Athena Federated Q -Athena uses _data source connectors_ that run on AWS Lambda to run federated queries. A data source connector is a piece of code that can translate between your target data source and Athena. You can think of a connector as an extension of Athena's query engine. Prebuilt Athena data source connectors exist for data sources like Amazon CloudWatch Logs, Amazon DynamoDB, Amazon DocumentDB, and Amazon RDS, and JDBC-compliant relational data sources such MySQL, and PostgreSQL under the Apache 2.0 license. You can also use the Athena Query Federation SDK to write custom connectors. To choose, configure, and deploy a data source connector to your account, you can use the Athena and Lambda consoles or the AWS Serverless Application Repository. After you deploy data source connectors, the connector is associated with a catalog that you can specify in SQL queries. You can combine SQL statements from multiple catalogs and span multiple data sources with a single query. +When you run a query against a data source, Athena invokes the connector to determine which data to read, manages parallelism, and pushes down filter predicates. Connectors can also restrict access to data based on the user who submits the query. @@ -15 +15 @@ Athena uses _data source connectors_ that run on AWS Lambda to run federated que -When a query is submitted against a data source, Athena invokes the corresponding connector to identify parts of the tables that need to be read, manages parallelism, and pushes down filter predicates. Based on the user submitting the query, connectors can provide or restrict access to specific data elements. Connectors use Apache Arrow as the format for returning data requested in a query, which enables connectors to be implemented in languages such as C, C++, Java, Python, and Rust. Since connectors are processed in Lambda, they can be used to access data from any data source on the cloud or on-premises that is accessible from Lambda. +Athena uses _data source connectors_ to run federated queries on underlying data. Athena supports two types of data source connectors with different capabilities: @@ -17 +17 @@ When a query is submitted against a data source, Athena invokes the correspondin -To write your own data source connector, you can use the Athena Query Federation SDK to customize one of the prebuilt connectors that Amazon Athena provides and maintains. You can modify a copy of the source code from the [GitHub repository](https://github.com/awslabs/aws-athena-query-federation/wiki/Available-Connectors) and then use the [Connector publish tool](https://github.com/awslabs/aws-athena-query-federation/wiki/Connector_Publish_Tool) to create your own AWS Serverless Application Repository package. + * AWS Glue Data Catalog federated connectors – These connectors use an AWS Glue connection to connect to the data source. They can be used with fine-grained data governance control support through Lake Formation. For more information, see [Federated catalog data connections](https://docs.aws.amazon.com/lake-formation/latest/dg/federated-catalog-data-connection.html) in the _AWS Lake Formation Developer Guide_. @@ -19 +19 @@ To write your own data source connector, you can use the Athena Query Federation -###### Note + * Connectors associated with a Lambda can optionally be manually registered as an AWS Glue Data Catalog to be used with Lake Formation for fine-grained data governance @@ -21 +21 @@ To write your own data source connector, you can use the Athena Query Federation -Third party developers may have used the Athena Query Federation SDK to write data source connectors. For support or licensing issues with these data source connectors, please work with your connector provider. These connectors are not tested or supported by AWS. + * Starting April 21, 2026, certain newly created connectors are automatically registered as Glue Data Catalogs and do not use a Lambda function in your AWS account @@ -23 +23 @@ Third party developers may have used the Athena Query Federation SDK to write da -For a list of data source connectors written and tested by Athena, see [Available data source connectors](./connectors-available.html). + * Athena data catalog federated connectors – These connectors are specific to Athena and cannot be registered as federated catalogs with AWS Glue Data Catalog. They require a Lambda function in your AWS account to query data. Custom connectors developed using the Athena Query Federation SDK are Athena data catalog connectors. For more information, see [Develop a data source connector using the Athena Query Federation SDK](./connect-data-source-federation-sdk.html). @@ -25 +24,0 @@ For a list of data source connectors written and tested by Athena, see [Availabl -For information about writing your own data source connector, see [Example Athena connector](https://github.com/awslabs/aws-athena-query-federation/tree/master/athena-example) on GitHub. @@ -27 +25,0 @@ For information about writing your own data source connector, see [Example Athen -## Considerations and limitations @@ -29 +27,8 @@ For information about writing your own data source connector, see [Example Athen - * Engine versions – Athena Federated Query is supported only on Athena engine version 2 and later. For information about Athena engine versions, see [Athena engine versioning](./engine-versions.html). + +For a list of data sources compatible with each type, see Connector type support by data source. + +###### Note + +Third party developers may have used the Athena Query Federation SDK to write data source connectors. For support or licensing issues with these data source connectors, please work with your connector provider. These connectors are not tested or supported by AWS. + +## Considerations and limitations @@ -43,17 +48,45 @@ For information about writing your own data source connector, see [Example Athen - - - -## Permissions required - -Data source connectors might require access to the following resources to function correctly. If you use a prebuilt connector, check the information for the connector to ensure that you have configured your VPC correctly. Also, ensure that IAM principals running queries and creating connectors have privileges to required actions. For more information, see [Allow access to Athena Federated Query: Example policies](./federated-query-iam-access.html). - - * Amazon S3 – In addition to writing query results to the Athena query results location in Amazon S3, data connectors also write to a spill bucket in Amazon S3. Connectivity and permissions to this Amazon S3 location are required. We recommend using spill to disk encryption for each connector and [S3 lifecycle configuration](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) to expire spilled data that is no longer needed. - - * Athena – Data sources need connectivity to Athena and vice versa for checking query status and preventing overscan. - - * AWS Glue Data Catalog – Connectivity and permissions are required if your connector uses Data Catalog for supplemental or primary metadata. - - * Amazon ECR – Data source connector Lambda functions use an Amazon ECR image from an Amazon ECR repository. The user that deploys the connector must have the permissions `ecr:BatchGetImage` and `ecr:GetDownloadUrlForLayer`. For more information, see [Amazon ECR permissions](https://docs.aws.amazon.com/lambda/latest/dg/images-create.html#gettingstarted-images-permissions) in the _AWS Lambda Developer Guide_. - - - + * Passthrough queries – Passthrough queries are not supported after a data source is registered as an AWS Glue Data Catalog. + + + + +## Connector type support by data source + +The following table shows the connector types that each data source supports. Certain AWS Glue Data Catalog federated catalog connectors that you create on or after April 21, 2026, do not require Lambda. + +Data source | AWS Glue Data Catalog federated connectors | Athena data catalog federated connectors +---|---|--- +| Without Lambda | With Lambda | +[Amazon CloudWatch Logs](./connectors-cloudwatch.html) | | Yes | Yes +[Amazon CloudWatch Metrics](./connectors-cwmetrics.html) | | Yes | Yes +[Amazon DocumentDB](./connectors-docdb.html) | Yes | Yes | Yes +[Amazon DynamoDB](./connectors-dynamodb.html) | Yes | Yes | Yes +[Amazon MSK](./connectors-msk.html) | | | Yes +[Amazon Neptune](./connectors-neptune.html) | | | Yes +[Amazon OpenSearch](./connectors-opensearch.html) | Yes | Yes | Yes +[Amazon Redshift](./connectors-redshift.html) | Yes | Yes | Yes +[Amazon Timestream](./connectors-timestream.html) | | Yes | Yes +[Azure Data Lake Storage](./connectors-adls-gen2.html) | | Yes | Yes +[Azure Synapse](./connectors-azure-synapse.html) | | Yes | Yes +[Cloudera Hive](./connectors-cloudera-hive.html) | | Yes | Yes +[Cloudera Impala](./connectors-cloudera-impala.html) | | Yes | Yes +[CMDB](./connectors-cmdb.html) | | Yes | Yes +[Confluent](./connectors-kafka.html) | | | Yes +[Custom](./connect-data-source-federation-sdk.html) | | | Yes +[Db2](./connectors-ibm-db2.html) | | Yes | Yes +[Db2 iSeries](./connectors-ibm-db2-as400.html) | | Yes | Yes +[Google BigQuery](./connectors-bigquery.html) | Yes | Yes | Yes +[Google Cloud Storage](./connectors-gcs.html) | | Yes | Yes +[HBase](./connectors-hbase.html) | | Yes | Yes +[Hortonworks (Hive)](./connectors-hortonworks.html) | | | Yes +[Kafka](./connectors-kafka.html) | | | Yes +[MySQL](./connectors-mysql.html) | Yes | Yes | Yes +[Oracle](./connectors-oracle.html) | Yes | Yes | Yes +[PostgreSQL](./connectors-postgresql.html) | Yes | Yes | Yes +[Redis OSS](./connectors-redis.html) | | | Yes +[SAP HANA](./connectors-sap-hana.html) | Yes | Yes | Yes +[Snowflake](./connectors-snowflake.html) | Yes | Yes | Yes +[SQL Server](./connectors-microsoft-sql-server.html) | Yes | Yes | Yes +[Teradata](./connectors-teradata.html) | Yes | Yes | Yes +[TPC-DS](./connectors-tpcds.html) | | Yes | Yes +[Vertica](./connectors-vertica.html) | | Yes | Yes @@ -67 +100 @@ Watch the following videos to learn more about using Athena Federated Query. -The following video demonstrates how to analyze results of an Athena federated query in Quick. +The following video demonstrates how to analyze results of an Athena Federated Query in Quick.