AWS glue documentation change
Summary
Major restructuring and expansion of AWS Glue connection properties documentation. Reorganized the table of contents and sections to add new connection types (Amazon DocumentDB, Amazon Redshift, MySQL, Oracle, PostgreSQL, SQL Server, etc.) with standardized property formats. Added detailed sections for each connection type including HOST, PORT, DATABASE, ENFORCE_SSL, ROLE_ARN, JDBC_PARAMS, CATALOG_CASING_FILTER properties and authentication methods. Moved older JDBC connection details to the end and restructured content flow.
Security assessment
The changes add extensive documentation about security features including ENFORCE_SSL property for multiple database connections, ROLE_ARN for IAM role-based access, authentication types (BASIC, IAM, OAUTH2, CUSTOM), and guidance on using AWS Secrets Manager for credential storage. However, there is no evidence in the diff that these changes address a specific security vulnerability or incident; they appear to be routine documentation improvements and expansion of security feature documentation.
Diff
diff --git a/glue/latest/dg/connection-properties.md b/glue/latest/dg/connection-properties.md index 0afc33e4c..06ac73938 100644 --- a//glue/latest/dg/connection-properties.md +++ b//glue/latest/dg/connection-properties.md @@ -7 +7 @@ -Required connection propertiesJDBC connection propertiesMongoDB and MongoDB Atlas connection propertiesSalesforce connection propertiesSnowflake connectionVertica connectionSAP HANA connectionAzure SQL connectionTeradata Vantage connectionOpenSearch Service connectionAzure Cosmos connectionSSL connection propertiesKafka connection properties for authenticationGoogle BigQuery connectionVertica connection +Required connection propertiesAmazon DocumentDB connectionOpenSearch Service connectionAmazon Redshift connectionGoogle BigQuery connectionMySQL connection propertiesOracle connection propertiesPostgreSQL connection propertiesSAP HANA connectionSnowflake connectionSQL Server connection propertiesTeradata Vantage connectionVertica connectionAzure Cosmos connectionAzure SQL connectionSalesforce connection propertiesJDBC connection propertiesMongoDB and MongoDB Atlas connection propertiesSSL connection propertiesKafka connection properties for authentication @@ -17 +17 @@ This topic includes information about properties for AWS Glue connections. - * AWS Glue JDBC connection properties + * Amazon DocumentDB connection @@ -19 +19 @@ This topic includes information about properties for AWS Glue connections. - * AWS Glue MongoDB and MongoDB Atlas connection properties + * OpenSearch Service connection @@ -21 +21 @@ This topic includes information about properties for AWS Glue connections. - * Salesforce connection properties + * Amazon Redshift connection @@ -23 +23 @@ This topic includes information about properties for AWS Glue connections. - * Snowflake connection + * Google BigQuery connection @@ -25 +25,5 @@ This topic includes information about properties for AWS Glue connections. - * Vertica connection + * MySQL connection properties + + * Oracle connection properties + + * PostgreSQL connection properties @@ -29 +33,3 @@ This topic includes information about properties for AWS Glue connections. - * Azure SQL connection + * Snowflake connection + + * SQL Server connection properties @@ -33 +39 @@ This topic includes information about properties for AWS Glue connections. - * OpenSearch Service connection + * Vertica connection @@ -37 +43 @@ This topic includes information about properties for AWS Glue connections. - * AWS Glue SSL connection properties + * Azure SQL connection @@ -39 +45 @@ This topic includes information about properties for AWS Glue connections. - * Apache Kafka connection properties for client authentication + * Salesforce connection properties @@ -41 +47 @@ This topic includes information about properties for AWS Glue connections. - * Google BigQuery connection + * AWS Glue JDBC connection properties @@ -43 +49,5 @@ This topic includes information about properties for AWS Glue connections. - * Vertica connection + * AWS Glue MongoDB and MongoDB Atlas connection properties + + * AWS Glue SSL connection properties + + * Apache Kafka connection properties for client authentication @@ -85 +95 @@ Specifies a comma-separated list of bootstrap server URLs. Include the port numb -## AWS Glue JDBC connection properties +## Amazon DocumentDB connection @@ -87 +97 @@ Specifies a comma-separated list of bootstrap server URLs. Include the port numb -AWS Glue Studio now creates unified connections for MySQL, Oracle, PostgresSQL, Redshift, and SQL Server data sources, which requires additional steps for accessing Secrets Manager and VPC resources, which may incur extra costs. You can access these connections in AWS Glue Studio by choosing the connection name for the respective connection. +The following properties are used to set up an Amazon DocumentDB connection. @@ -89 +99 @@ AWS Glue Studio now creates unified connections for MySQL, Oracle, PostgresSQL, -For more information, see [Considerations](./using-connectors-unified-connections.html#using-connectors-unified-connections-considerations). +**HOST** @@ -91 +100,0 @@ For more information, see [Considerations](./using-connectors-unified-connection -AWS Glue can connect to the following data stores through a JDBC connection: @@ -93 +102 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Amazon Redshift +(Required, STRING) The hostname of the Amazon DocumentDB cluster endpoint. @@ -95 +104 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Amazon Aurora +**PORT** @@ -97 +105,0 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Microsoft SQL Server @@ -99 +107 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * MySQL +(Required, STRING) The port number. The value must be a valid port number. @@ -101 +109 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Oracle +**ENFORCE_SSL** @@ -103 +110,0 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * PostgreSQL @@ -105 +112 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Snowflake, when using AWS Glue crawlers. +(Optional, BOOLEAN) Whether to enforce SSL for the connection. The default value is `false`. Allowed values: `true`, `false`. @@ -107 +114 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Aurora (supported if the native JDBC driver is being used. Not all driver features can be leveraged) +**ROLE_ARN** @@ -109 +115,0 @@ AWS Glue can connect to the following data stores through a JDBC connection: - * Amazon RDS for MariaDB @@ -110,0 +117 @@ AWS Glue can connect to the following data stores through a JDBC connection: +(Optional, STRING) The IAM role ARN used to access AWS Secrets Manager and to assign an IP address if a Amazon VPC is specified. @@ -111,0 +119 @@ AWS Glue can connect to the following data stores through a JDBC connection: +**JDBC_PARAMS** @@ -114 +122 @@ AWS Glue can connect to the following data stores through a JDBC connection: -###### Important +(Optional, STRING) Additional connection parameters. @@ -116 +124 @@ AWS Glue can connect to the following data stores through a JDBC connection: -Currently, an ETL job can use JDBC connections within only one subnet. If you have multiple data stores in a job, they must be on the same subnet, or accessible from the subnet. +**Authentication** @@ -118 +126 @@ Currently, an ETL job can use JDBC connections within only one subnet. If you ha -If you choose to bring in your own JDBC driver versions for AWS Glue crawlers, your crawlers will consume resources in AWS Glue jobs and Amazon S3 to ensure your provided drivers are run in your environment. The additional usage of resources will be reflected in your account. Additionally, providing your own JDBC driver does not mean that the crawler is able to leverage all of the driver’s features. Drivers are limited to the properties described in [Defining connections in the Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/glue-connections.html). +The Amazon DocumentDB connection supports the following authentication type: @@ -120 +128 @@ If you choose to bring in your own JDBC driver versions for AWS Glue crawlers, y -The following are additional properties for the JDBC connection type. + * **BASIC** – Authenticates with a username and password stored in AWS Secrets Manager. @@ -122 +129,0 @@ The following are additional properties for the JDBC connection type. -**JDBC URL** @@ -125 +131,0 @@ The following are additional properties for the JDBC connection type. -Enter the URL for your JDBC data store. For most database engines, this field is in the following format. In this format, replace `protocol`, `host`, `port`, and `db_name` with your own information. @@ -127 +133 @@ Enter the URL for your JDBC data store. For most database engines, this field is -` jdbc:`protocol`://`host`:`port`/`db_name` ` +Use the following properties when configuring a connection to Amazon DocumentDB through a Amazon VPC: @@ -129 +135 @@ Enter the URL for your JDBC data store. For most database engines, this field is -Depending on the database engine, a different JDBC URL format might be required. This format can have slightly different use of the colon (:) and slash (/) or different keywords to specify databases. +**VPC** @@ -131 +136,0 @@ Depending on the database engine, a different JDBC URL format might be required. -For JDBC to connect to the data store, a `db_name` in the data store is required. The `db_name` is used to establish a network connection with the supplied `username` and `password`. When connected, AWS Glue can access other databases in the data store to run a crawler or run an ETL job. @@ -133 +138 @@ For JDBC to connect to the data store, a `db_name` in the data store is required -The following JDBC URL examples show the syntax for several database engines. +Choose the name of the virtual private cloud (VPC) that contains your data store. The AWS Glue console lists all VPCs for the current Region. @@ -135 +140 @@ The following JDBC URL examples show the syntax for several database engines. - * To connect to an Amazon Redshift cluster data store with a `dev` database: +**Subnet** @@ -137 +141,0 @@ The following JDBC URL examples show the syntax for several database engines. -`jdbc:redshift://xxx.us-east-1.redshift.amazonaws.com:8192/dev` @@ -139 +143 @@ The following JDBC URL examples show the syntax for several database engines. - * To connect to an Amazon RDS for MySQL data store with an `employee` database: +Choose the subnet within the VPC that contains your data store. The AWS Glue console lists all subnets for the data store in your VPC. @@ -141 +145 @@ The following JDBC URL examples show the syntax for several database engines. -`jdbc:mysql://xxx-cluster.cluster-xxx.us-east-1.rds.amazonaws.com:3306/employee` +**Security groups** @@ -143 +146,0 @@ The following JDBC URL examples show the syntax for several database engines. - * To connect to an Amazon RDS for PostgreSQL data store with an `employee` database: @@ -145 +148 @@ The following JDBC URL examples show the syntax for several database engines. -`jdbc:postgresql://xxx-cluster.cluster-xxx.us-east-1.rds.amazonaws.com:5432/employee` +Choose the security groups that are associated with your data store. AWS Glue requires one or more security groups with an inbound source rule that allows AWS Glue to connect. The AWS Glue console lists all security groups that are granted inbound access to your VPC. AWS Glue associates these security groups with the elastic network interface that is attached to your VPC subnet. @@ -147 +150 @@ The following JDBC URL examples show the syntax for several database engines. - * To connect to an Amazon RDS for Oracle data store with an `employee` service name: +## OpenSearch Service connection @@ -149 +152 @@ The following JDBC URL examples show the syntax for several database engines. -`jdbc:oracle:thin://@xxx-cluster.cluster-xxx.us-east-1.rds.amazonaws.com:1521/employee` +Use the following properties to set up a OpenSearch Service connection for AWS Glue ETL jobs. @@ -151 +154 @@ The following JDBC URL examples show the syntax for several database engines. -The syntax for Amazon RDS for Oracle can follow the following patterns. In these patterns, replace `host`, `port`, `service_name`, and `SID` with your own information. +**Domain endpoint** @@ -153 +155,0 @@ The syntax for Amazon RDS for Oracle can follow the following patterns. In these - * `jdbc:oracle:thin://@`host`:`port`/`service_name`` @@ -155 +157 @@ The syntax for Amazon RDS for Oracle can follow the following patterns. In these - * `jdbc:oracle:thin://@`host`:`port`:`SID`` +An Amazon OpenSearch Service domain endpoint will have the following default form, https://search-`domainName`-`unstructuredIdContent`.`region`.es.amazonaws.com. For more information on identifying your domain endpoint, see [Creating and managing Amazon OpenSearch Service domains](https://docs.aws.amazon.com//opensearch-service/latest/developerguide/createupdatedomains.html) in the Amazon OpenSearch Service documentation. @@ -157 +159 @@ The syntax for Amazon RDS for Oracle can follow the following patterns. In these - * To connect to an Amazon RDS for Microsoft SQL Server data store with an `employee` database: +**Port** @@ -159 +160,0 @@ The syntax for Amazon RDS for Oracle can follow the following patterns. In these -`jdbc:sqlserver://xxx-cluster.cluster-xxx.us-east-1.rds.amazonaws.com:1433;databaseName=employee` @@ -161 +162 @@ The syntax for Amazon RDS for Oracle can follow the following patterns. In these -The syntax for Amazon RDS for SQL Server can follow the following patterns. In these patterns, replace `server_name`, `port`, and `db_name` with your own information. +The port open on the endpoint. @@ -163 +164 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t - * `jdbc:sqlserver://`server_name`:`port`;database=`db_name`` +**AWS Secret** @@ -165 +165,0 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t - * `jdbc:sqlserver://`server_name`:`port`;databaseName=`db_name`` @@ -167 +167 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t - * To connect to an Amazon Aurora PostgreSQL instance of the `employee` database, specify the endpoint for the database instance, the port, and the database name: +The **Secret name** of a secret in AWS Secrets Manager. AWS Glue will connect to OpenSearch Service using the keys of your secret. @@ -169 +169 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t -`jdbc:postgresql://employee_instance_1.`xxxxxxxxxxxx`.us-east-2.rds.amazonaws.com:5432/employee` +**ROLE_ARN** @@ -171 +170,0 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t - * To connect to an Amazon RDS for MariaDB data store with an `employee` database, specify the endpoint for the database instance, the port, and the database name: @@ -173 +172 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t -`jdbc:mysql://`xxx`-cluster.cluster-`xxx`.`aws-region`.rds.amazonaws.com:3306/employee` +(Optional, STRING) The IAM role ARN used to access AWS Secrets Manager and to assign an IP address if a Amazon VPC is specified. @@ -175 +174 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t - * ###### Warning +Use the following properties when configuring a connection to a OpenSearch Service endpoint hosted in Amazon VPC: @@ -177 +176 @@ The syntax for Amazon RDS for SQL Server can follow the following patterns. In t -Snowflake JDBC connections are supported only by AWS Glue crawlers. When using the Snowflake connector in AWS Glue jobs, use the Snowflake connection type. +**VPC** @@ -179 +177,0 @@ Snowflake JDBC connections are supported only by AWS Glue crawlers. When using t -To connect to a Snowflake instance of the `sample` database, specify the endpoint for the snowflake instance, the user, the database name, and the role name. You can optionally add the `warehouse` parameter. @@ -181 +179 @@ To connect to a Snowflake instance of the `sample` database, specify the endpoin -`jdbc:snowflake://`account_name`.snowflakecomputing.com/?user=`user_name`&db=sample&role=`role_name`&warehouse=`warehouse_name`` +Choose the name of the virtual private cloud (VPC) that contains your data store. The AWS Glue console lists all VPCs for the current Region. @@ -183 +181 @@ To connect to a Snowflake instance of the `sample` database, specify the endpoin -###### Important +**Subnet** @@ -185 +182,0 @@ To connect to a Snowflake instance of the `sample` database, specify the endpoin -For Snowflake connections over JDBC, the order of parameters in the URL is enforced and must be ordered as `user`, `db`, `role_name`, and `warehouse`. @@ -187 +184 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor - * To connect to a Snowflake instance of the `sample` database with AWS private link, specify the snowflake JDBC URL as follows: +Choose the subnet within the VPC that contains your data store. The AWS Glue console lists all subnets for the data store in your VPC. @@ -189 +186 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor -`jdbc:snowflake://`account_name`.`region`.privatelink.snowflakecomputing.com/?user=`user_name`&db=sample&role=`role_name`&warehouse=`warehouse_name`` +**Security groups** @@ -191,0 +189 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor +Choose the security groups that are associated with your data store. AWS Glue requires one or more security groups with an inbound source rule that allows AWS Glue to connect. The AWS Glue console lists all security groups that are granted inbound access to your VPC. AWS Glue associates these security groups with the elastic network interface that is attached to your VPC subnet. @@ -192,0 +191 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor +## Amazon Redshift connection @@ -194 +193 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor -**Username** +The following properties are used to set up an Amazon Redshift connection. @@ -195,0 +195 @@ For Snowflake connections over JDBC, the order of parameters in the URL is enfor