AWS glue documentation change
Summary
Updated Snowflake connection documentation to add support for OAuth and key-pair authentication methods, updated credential storage references from Data Catalog to AWS Glue connections, and restructured prerequisites and configuration steps.
Security assessment
The changes add documentation for two additional authentication methods (OAuth and key-pair) which are more secure alternatives to basic username/password authentication. This enhances security by providing options that avoid password storage and transmission. However, there is no evidence this change addresses a specific security vulnerability or incident - it appears to be a feature enhancement expanding authentication options.
Diff
diff --git a/glue/latest/dg/aws-glue-programming-etl-connect-snowflake-home.md b/glue/latest/dg/aws-glue-programming-etl-connect-snowflake-home.md index ae14207ad..5266c02fe 100644 --- a//glue/latest/dg/aws-glue-programming-etl-connect-snowflake-home.md +++ b//glue/latest/dg/aws-glue-programming-etl-connect-snowflake-home.md @@ -9 +9 @@ Configuring SnowflakeRead from SnowflakeWriting to SnowflakeSnowflake connection -You can use AWS Glue for Spark to read from and write to tables in Snowflake in AWS Glue 4.0 and later versions. You can read from Snowflake with a SQL query. You can connect to Snowflake using a user and password. You can refer to Snowflake credentials stored in AWS Secrets Manager through the AWS Glue Data Catalog. Data Catalog Snowflake credentials for AWS Glue for Spark are stored separately from Data Catalog Snowflake credentials for crawlers. You must choose a `SNOWFLAKE` type connection and not a `JDBC` type connection configured to connect to Snowflake. +You can use AWS Glue for Spark to read from and write to tables in Snowflake in AWS Glue 4.0 and later versions. You can read from Snowflake with a SQL query. You can connect to Snowflake using one of three methods - basic authentication (using username and password), OAuth authentication, or key-pair authentication. You can refer to Snowflake credentials stored in AWS Secrets Manager through the AWS Glue Data connections. Data connection Snowflake credentials for AWS Glue for Spark are stored separately from Data Catalog Snowflake credentials for crawlers. You must choose a `SNOWFLAKE` type connection and not a `JDBC` type connection configured to connect to Snowflake. @@ -21 +21 @@ Optionally, you can perform the following configuration to manage your connectio - 1. In Snowflake, generate a user, `snowflakeUser` and password, `snowflakePassword`. + 1. In AWS Secrets Manager, create a secret using your Snowflake credentials. To create a secret in Secrets Manager, follow the tutorial available in [ Create an AWS Secrets Manager secret ](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html#create_secret_cli) in the AWS Secrets Manager documentation. After creating the secret, keep the Secret name, `secretName` for the next step. @@ -23 +23 @@ Optionally, you can perform the following configuration to manage your connectio - 2. In AWS Secrets Manager, create a secret using your Snowflake credentials. To create a secret in Secrets Manager, follow the tutorial available in [ Create an AWS Secrets Manager secret ](https://docs.aws.amazon.com/secretsmanager/latest/userguide/create_secret.html#create_secret_cli) in the AWS Secrets Manager documentation. After creating the secret, keep the Secret name, `secretName` for the next step. + * For OAuth authentication: @@ -25 +25 @@ Optionally, you can perform the following configuration to manage your connectio - * When selecting **Key/value pairs** , create a pair for `snowflakeUser` with the key `USERNAME`. + * When selecting **Key/value pairs** , create a pair for `snowflakeUser` with the key `sfUser` @@ -27 +27,13 @@ Optionally, you can perform the following configuration to manage your connectio - * When selecting **Key/value pairs** , create a pair for `snowflakePassword` with the key `PASSWORD`. + * When selecting **Key/value pairs** , create a pair for `OAUTH_CLIENT_SECRET` with the key `USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET` + + * For Key-pair authentication: + + * When selecting **Key/value pairs** , create a pair for `snowflakeUser` with the key `sfUser` + + * When selecting **Key/value pairs** , create a pair for `private key` with the key `pem_private_key` + + * For basic authentication: + + * When selecting **Key/value pairs** , create a pair for `snowflakeUser` with the key `USERNAME` + + * When selecting **Key/value pairs** , create a pair for `snowflakePassword` with the key `PASSWORD` @@ -39,3 +51 @@ Optionally, you can perform the following configuration to manage your connectio - * `pem_private_key` \- Private key for key-pair authentication - - 3. In the AWS Glue Data Catalog, create a connection by choosing **Connections** , then **Create connection**. Following the steps in the connection wizard to complete the process: + 2. In the AWS Glue Studio Console, create a connection by choosing **Data Connections** , then **Create connection**. Following the steps in the connection wizard to complete the process: @@ -51,3 +61 @@ Optionally, you can perform the following configuration to manage your connectio - 4. In the next step in the wizard, set properties for your Snowflake connection. - - 5. In the final step in the wizard, review your settings and then complete the process to create your connection. + 3. In the next step in the wizard, set properties for your Snowflake connection. @@ -54,0 +63 @@ Optionally, you can perform the following configuration to manage your connectio + 4. In the final step in the wizard, review your settings and then complete the process to create your connection. @@ -58 +66,0 @@ Optionally, you can perform the following configuration to manage your connectio -In the following situations, you may require the following: @@ -60 +68 @@ In the following situations, you may require the following: - * For Snowflake hosted on AWS in an Amazon VPC +For Snowflake hosted on AWS in an Amazon VPC, you may require the following: @@ -75,5 +83 @@ In the following situations, you may require the following: -**Prerequisites:** A Snowflake table you would like to read from. You will need the Snowflake table name, `tableName`. You will need your Snowflake url `snowflakeUrl`, username `snowflakeUser` and password `snowflakePassword`. If your Snowflake user does not have a default namespace set, you will need the Snowflake database name, `databaseName` and the schema name `schemaName`. Additionally, if your Snowflake user does not have a default warehouse set, you will need a warehouse name `warehouseName`. - -For example: - -**Additional Prerequisites:** Complete the steps _To manage your connection credentials with AWS Glue_ to configure `snowflakeUrl`, `snowflakeUsername` and `snowflakePassword`. To review these steps, see Configuring Snowflake connections, the previous section. To select which **Additional network connection** to connect with, we will use the `connectionName` parameter. +**Prerequisites:** A Snowflake table you would like to read from. You will need the Snowflake table name, `tableName`. If your Snowflake user does not have a default namespace set, you will need the Snowflake database name, `databaseName` and the schema name `schemaName`. Additionally, if your Snowflake user does not have a default warehouse set, you will need a warehouse name `warehouseName`. To select which **Additional network connection** to connect with, the `connectionName` parameter will be used. @@ -110,5 +114 @@ Additionally, you can use the `autopushdown` and `query` parameters to read a po -**Prerequisites:** A Snowflake database you would like to write to. You will need a current or desired table name, `tableName`. You will need your Snowflake url `snowflakeUrl`, username `snowflakeUser` and password `snowflakePassword`. If your Snowflake user does not have a default namespace set, you will need the Snowflake database name, `databaseName` and the schema name `schemaName`. Additionally, if your Snowflake user does not have a default warehouse set, you will need a warehouse name `warehouseName`. - -For example: - -**Additional Prerequisites:** Complete the steps _To manage your connection credentials with AWS Glue_ to configure `snowflakeUrl`, `snowflakeUsername` and `snowflakePassword`. To review these steps, see Configuring Snowflake connections, the previous section. To select which **Additional network connection** to connect with, we will use the `connectionName` parameter. +**Prerequisites:** A Snowflake database you would like to write to. You will need a current or desired table name, `tableName`. If your Snowflake user does not have a default namespace set, you will need the Snowflake database name, `databaseName` and the schema name `schemaName`. Additionally, if your Snowflake user does not have a default warehouse set, you will need a warehouse name `warehouseName`. To select which **Additional network connection** to connect with, the `connectionName` parameter will be used. @@ -132 +132 @@ The Snowflake connection type takes the following connection options: -You can retrieve some of the parameters in this section from a Data Catalog connection (`sfUrl`, `sfUser`, `sfPassword`), in which case you are not required to provide them. You can do this by providing the parameter `connectionName`. +You can retrieve some of the parameters in this section from a AWS Glue connection (`sfUrl`, `sfUser`, `sfPassword`), in which case you are not required to provide them. You can do this by providing the parameter `connectionName`. @@ -138 +138 @@ You can retrieve connection parameters from AWS Secrets Manager secrets using th - * `sfPassword` (using key `PASSWORD` or `sfPassword`) + * `sfPassword` (using key `PASSWORD` or `sfPassword`, when using basic authentication) @@ -148 +148,3 @@ You can retrieve connection parameters from AWS Secrets Manager secrets using th - * `pem_private_key` (using key `pem_private_key`) + * `pem_private_key` (using key `pem_private_key`, when using key-pair authentication) + + * `USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET` (when using OAuth authentication) @@ -157 +159 @@ You can retrieve connection parameters from AWS Secrets Manager secrets using th - 2. Data Catalog connection properties + 2. Glue connection properties @@ -180 +182 @@ The following parameters are used generally when connecting to Snowflake. - * `sfPassword` — (Required unless `pem_private_key` provided) Used for Read/Write. Password for the Snowflake user. + * `sfPassword` — (Required when using basic authnetication) Used for Read/Write. Password for the Snowflake user. @@ -184 +186,3 @@ The following parameters are used generally when connecting to Snowflake. - * `pem_private_key` — Used for Read/Write. An unencrypted b64-encoded private key string. The private key for the Snowflake user. It is common to copy this out of a PEM file. For more information, see [Key-pair authentication and key-pair rotation](https://docs.snowflake.com/en/user-guide/key-pair-auth) in the Snowflake documentation. + * `pem_private_key` — (Required when using key-pair authentication) Used for Read/Write. An unencrypted b64-encoded private key string. The private key for the Snowflake user. It is common to copy this out of a PEM file. For more information, see [Key-pair authentication and key-pair rotation](https://docs.snowflake.com/en/user-guide/key-pair-auth) in the Snowflake documentation. + + * `USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET` — (Required when using OAuth Authentication) Used for both read and write operations. This value corresponds to the OAUTH_CLIENT_SECRET, which can be obtained from the Snowflake security integration configured to enable OAuth-based authentication for your account. For more details, refer to your Snowflake OAuth security integration setup documentation - [Configure Snowflake OAuth for custom clients](https://docs.snowflake.com/en/user-guide/oauth-custom). @@ -208 +212,37 @@ AWS Glue supports the following authentication methods for connecting to Snowfla - * **Username and Password Authentication:** Provide `sfUser` and `sfPassword` parameters. + * **Basic authentication:** Provide `sfUser` and `sfPassword` parameters. + + * **Key-pair authentication:** Provide `sfUser` and `pem_private_key` parameters. When using key-pair authentication, the `sfPassword` parameter is not required. + + * **OAuth authentication:** The Snowflake Connector supports the AUTHORIZATION_CODE grant type to request access to your Snowflake data. This grant type is referred to as “3-legged OAuth”, as it involves redirecting users to a third-party authorization server where they can authenticate and approve access. This method is used when creating a connection through the AWS Glue Console. + + * **Prerequisite:** To use this authentication method, ensure the following setup is complete: + + * **Configure Snowflake OAuth for a custom client** by following the official Snowflake documentation: [Configure Snowflake OAuth for custom clients.](https://docs.snowflake.com/en/user-guide/oauth-custom) + + * **Set the correct redirect URI** when creating the Snowflake security integration. For example: If you are creating the connection in the DUB (eu-west-1) region, your redirect URI should be: `https://eu-west-1.console.aws.amazon.com/gluestudio/oauth` + + * After creating the security integration, retain the following information for use when creating the Glue connection: + + * OAUTH_CLIENT_ID: This value should be provided as User Managed Client Application Client ID on the Glue connection creation page. + + * OAUTH_CLIENT_SECRET: This value should be stored in the AWS Secret used for the connection, under the key USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET. + + * OAuth Scopes — (Optional) Defines the specific permissions or levels of access requested from the Snowflake account. For example, a scope might limit access to a particular resource or operation. + + * This value can be specified in the following format: `session:role:Snowflake_Role_Name` + + * Example: `session:role:ANALYST_ROLE` + + * Authorization Code URL — (Required) The endpoint where the user is redirected to log in and grant authorization. + + * Example: `https://host/oauth/authorize` + + * Authorization Token URL — (Required) The endpoint used to exchange the authorization code for an access token. + + * Example: `https://host/oauth/token-request` + + * User Managed Client Application Client Id — (Required) The unique identifier for your registered OAuth client application in Snowflake + + * AWS Secret — (Required) Refers to an AWS Secrets Manager secret containing the following key-value pairs: + + * sfUser - The Snowflake username @@ -210 +250 @@ AWS Glue supports the following authentication methods for connecting to Snowfla - * **Key-Pair Authentication:** Provide `sfUser` and `pem_private_key` parameters. When using key-pair authentication, the `sfPassword` parameter is not required. + * USER_MANAGED_CLIENT_APPLICATION_CLIENT_SECRET - The client secret associated with the OAuth client application @@ -215 +255 @@ AWS Glue supports the following authentication methods for connecting to Snowfla -Both authentication methods are fully supported and can be configured using any combination of connection options, Data Catalog connections, or AWS Secrets Manager secrets. +All three authentication methods are fully supported and can be configured using any combination of connection options, Glue connections, or AWS Secrets Manager secrets. @@ -225 +265 @@ Connecting to Snowflake with AWS Glue for Spark is subject to the following limi - * This connector supports username/password authentication and key-pair authentication. Other authentication methods (such as OAuth or SAML) are not currently supported. + * This connector supports basic authentication, key-pair authentication, and OAuth authentication. Other authentication methods (such as SAML) are not currently supported.