AWS AmazonS3 documentation change
Summary
Updated the integration process for Amazon S3 Tables with AWS analytics services, shifting from mandatory AWS Lake Formation integration to using IAM access controls by default, with Lake Formation as an optional layer. The prerequisites and CLI setup steps were rewritten to remove Lake Formation-specific policies and steps, focusing on AWS Glue Data Catalog permissions and a simplified IAM-based setup.
Security assessment
The changes restructure the integration model to use IAM access controls as the default, which is a fundamental security and access management update. It clarifies and simplifies the permission model by removing the mandatory Lake Formation data lake administrator role and specific Lake Formation API permissions, replacing them with direct IAM and AWS Glue permissions. This change impacts how access is governed and delegated, but there is no evidence in the diff that it addresses a specific, disclosed security vulnerability or incident.
Diff
diff --git a/AmazonS3/latest/userguide/s3-tables-integrating-aws.md b/AmazonS3/latest/userguide/s3-tables-integrating-aws.md index 65f76a04a..0a3324cb0 100644 --- a//AmazonS3/latest/userguide/s3-tables-integrating-aws.md +++ b//AmazonS3/latest/userguide/s3-tables-integrating-aws.md @@ -13 +13 @@ This topic covers the prerequisites and procedures needed to integrate your Amaz -This integration uses the AWS Glue and AWS Lake Formation services and might incur AWS Glue request and storage costs. For more information, see [AWS Glue Pricing.](https://aws.amazon.com/glue/pricing/) +This integration uses the AWS Glue Data Catalog and might incur AWS Glue request and storage costs. For more information, see [AWS Glue Pricing.](https://aws.amazon.com/glue/pricing/) @@ -15 +15 @@ This integration uses the AWS Glue and AWS Lake Formation services and might inc -Additional pricing applies for running queries on your S3 tables. For more information, see pricing information for the query engine that you're using. +Additional pricing applies for running queries on S3 Tables. For more information, see pricing information for the query engine that you're using. @@ -23 +23 @@ The following prerequisites are required to integrate table buckets with AWS ana - * Attach the [AWSLakeFormationDataAdmin](https://docs.aws.amazon.com/aws-managed-policy/latest/reference/AWSLakeFormationDataAdmin.html) AWS managed policy to your AWS Identity and Access Management (IAM) principal to make that user a data lake administrator. For more information about how to create a data lake administrator, see [Create a data lake administrator](https://docs.aws.amazon.com//lake-formation/latest/dg/initial-lf-config.html#create-data-lake-admin) in the _AWS Lake Formation Developer Guide_. + * Add the following AWS Glue permissions to your AWS Identity and Access Management (IAM) principal: @@ -25 +25 @@ The following prerequisites are required to integrate table buckets with AWS ana - * Add permissions for the `glue:PassConnection` operation to your IAM principal. + * `glue:CreateCatalog` which is required to create `s3tablescatalog` federated catalog in the Data Catalog @@ -27 +27 @@ The following prerequisites are required to integrate table buckets with AWS ana - * Add permissions for the `lakeformation:RegisterResource` and `lakeformation:RegisterResourceWithPrivilegedAccess` operations to your IAM principal. + * `glue:PassConnection` grants the calling principal the right to delegate `aws:s3tables` connection creation to Amazon S3 service. @@ -42 +42 @@ If your table definition contains capital letters, you receive the following err -Amazon S3 Tables integrates with AWS Glue Data Catalog (Data Catalog) and registers the catalog as a Lake Formation data location. You can set up this registration from the Lake Formation console or by using the service APIs. When you register the location, you must specify an IAM role that grants read/write permissions to the Lake Formation registered role to access that location. Lake Formation assumes that role when supplying temporary credentials to integrated AWS services. +You can integrate table buckets with Data Catalog and AWS analytics services using IAM access controls by default, or optionally use Lake Formation access controls. @@ -44 +44 @@ Amazon S3 Tables integrates with AWS Glue Data Catalog (Data Catalog) and regist -If you have an IAM or S3 Tables resource-based policy that restricts IAM users and IAM roles based on principal tags, you must attach the same principal tags to the IAM role that Lake Formation uses to access your Amazon S3 data (for example, LakeFormationDataAccessRole) and grant this role the necessary permissions. This is required for your tag-based access control policy to work correctly with your S3 Tables analytics integration. +When you integrate using IAM access controls, you require IAM privileges to access Amazon S3 table buckets and tables, Data Catalog objects, and the query engine you're using. If you choose to integrate using Lake Formation, then both IAM access controls and Lake Formation grants determine the access to Data Catalog resources. Please refer to [_AWS Lake Formation Developer Guide_](https://docs.aws.amazon.com/lake-formation/latest/dg/create-s3-tables-catalog.html) to learn more about Lake Formation integration. @@ -46,7 +46 @@ If you have an IAM or S3 Tables resource-based policy that restricts IAM users a -This integration must be configured once per AWS Region. - -###### Important - -The AWS analytics services integration now uses the `WithPrivilegedAccess` option in the `registerResource` Lake Formation API operation to register S3 table buckets. The integration also now creates the `s3tablescatalog` catalog in the AWS Glue Data Catalog by using the `AllowFullTableExternalDataAccess` option in the `CreateCatalog` AWS Glue API operation. - -If you set up the integration with the preview release, you can continue to use your current integration. However, the updated integration process provides performance improvements, so we recommend migrating. To migrate to the updated integration, see Migrating to the updated integration process. +The following sections describe how you could use Amazon S3 management console or AWS CLI to configure the integration with IAM access controls. @@ -69,3 +63 @@ The **Create table bucket** page opens. -The first time that you integrate table buckets in any Region, Amazon S3 creates a new IAM service role on your behalf. This role allows Lake Formation to access all table buckets in your account and federate access to your tables in AWS Glue Data Catalog. - -###### To integrate table buckets using the AWS CLI +###### To integrate table buckets with IAM access controls using the AWS CLI @@ -81,9 +73 @@ The following steps show how to use the AWS CLI to integrate table buckets. To u - 2. Create an IAM service role that allows Lake Formation to access your table resources. - - 1. Create a file called `Role-Trust-Policy.json` that contains the following trust policy: - -JSON - - -**** - + 2. Create a file called `catalog.json` that contains the following catalog: @@ -92,2 +76,7 @@ JSON - "Version":"2012-10-17", - "Statement": [ + "Name": "s3tablescatalog", + "CatalogInput": { + "FederatedCatalog": { + "Identifier": "arn:aws:s3tables:us-east-1:111122223333:bucket/*", + "ConnectionName": "aws:s3tables" + }, + "CreateDatabaseDefaultPermissions":[ @@ -95,2 +83,0 @@ JSON - "Sid": "LakeFormationDataAccessPolicy", - "Effect": "Allow", @@ -98 +85 @@ JSON - "Service": "lakeformation.amazonaws.com" + "DataLakePrincipalIdentifier": "IAM_ALLOWED_PRINCIPALS" @@ -100,10 +87 @@ JSON - "Action": [ - "sts:AssumeRole", - "sts:SetContext", - "sts:SetSourceIdentity" - ], - "Condition": { - "StringEquals": { - "aws:SourceAccount": "111122223333" - } - } + "Permissions": ["ALL"] @@ -111,26 +88,0 @@ JSON - ] - } - - -Create the IAM service role by using the following command: - - aws iam create-role \ - --role-name S3TablesRoleForLakeFormation \ - --assume-role-policy-document file://Role-Trust-Policy.json - - 2. Create a file called `LF-GluePolicy.json` that contains the following policy: - -JSON - - -**** - - - { - "Version":"2012-10-17", - "Statement": [ - { - "Sid": "LakeFormationPermissionsForS3ListTableBucket", - "Effect": "Allow", - "Action": [ - "s3tables:ListTableBuckets" @@ -138,4 +90 @@ JSON - "Resource": [ - "*" - ] - }, + "CreateTableDefaultPermissions":[ @@ -143,59 +92,2 @@ JSON - "Sid": "LakeFormationDataAccessPermissionsForS3TableBucket", - "Effect": "Allow", - "Action": [ - "s3tables:CreateTableBucket", - "s3tables:GetTableBucket", - "s3tables:CreateNamespace", - "s3tables:GetNamespace", - "s3tables:ListNamespaces", - "s3tables:DeleteNamespace", - "s3tables:DeleteTableBucket", - "s3tables:CreateTable", - "s3tables:DeleteTable", - "s3tables:GetTable", - "s3tables:ListTables", - "s3tables:RenameTable", - "s3tables:UpdateTableMetadataLocation", - "s3tables:GetTableMetadataLocation", - "s3tables:GetTableData", - "s3tables:PutTableData" - ], - "Resource": [ - "arn:aws:s3tables:us-east-1:111122223333:bucket/*" - ] - } - ] - } - - -Attach the policy to the role by using the following command: - - aws iam put-role-policy \ - --role-name S3TablesRoleForLakeFormation \ - --policy-name LakeFormationDataAccessPermissionsForS3TableBucket \ - --policy-document file://LF-GluePolicy.json - - 3. Create a file called `input.json` that contains the following: - - { - "ResourceArn": "arn:aws:s3tables:us-east-1:111122223333:bucket/*", - - "WithFederation": true, - "RoleArn": "arn:aws:iam::111122223333:role/S3TablesRoleForLakeFormation" - } - -Register table buckets with Lake Formation by using the following command: - - aws lakeformation register-resource \ - --region us-east-1 \ - --with-privileged-access \ - --cli-input-json file://input.json - - 4. Create a file called `catalog.json` that contains the following catalog: - - { - "Name": "s3tablescatalog", - "CatalogInput": { - "FederatedCatalog": { - "Identifier": "arn:aws:s3tables:us-east-1:111122223333:bucket/*", - "ConnectionName": "aws:s3tables" + "Principal": { + "DataLakePrincipalIdentifier": "IAM_ALLOWED_PRINCIPALS" @@ -203,2 +95,3 @@ Register table buckets with Lake Formation by using the following command: - "CreateDatabaseDefaultPermissions":[], - "CreateTableDefaultPermissions":[], + "Permissions": ["ALL"] + } + ], @@ -215 +108 @@ Create the `s3tablescatalog` catalog by using the following command. Creating th - 5. Verify that the `s3tablescatalog` catalog was added in AWS Glue by using the following command: + 3. Verify that the `s3tablescatalog` catalog was added in AWS Glue by using the following command: @@ -222 +115 @@ Create the `s3tablescatalog` catalog by using the following command. Creating th -The AWS analytics services integration process has been updated. If you've set up the integration with the preview release, you can continue to use your current integration. However, the updated integration process provides performance improvements, so we recommend migrating by using the following steps. For more information about the migration or integration process, see [Creating an Amazon S3 Tables catalog in the AWS Glue Data Catalog](https://docs.aws.amazon.com/lake-formation/latest/dg/create-s3-tables-catalog.html) in the _AWS Lake Formation Developer Guide_. +The AWS analytics services integration process has been updated to use IAM permissions by default. If you've already set up the integration, you can continue to use your current integration. However, if you want to change your existing integration to use IAM permissions instead, see [_AWS Lake Formation Developer Guide_](https://docs.aws.amazon.com/lake-formation/latest/dg/create-s3-tables-catalog.html). You can also redo the integration to delete your existing setup in AWS Glue Data Catalog and AWS Lake Formation and re-run the integration. This will remove all existing Lake Formation grants and associated access permissions to the `s3tablescatalog`. @@ -251,2 +143,0 @@ If you want to work with SSE-KMS encrypted tables in integrated AWS analytics se -After you integrate your IAM principal is granted Lake Formation permissions to access your tables, if you want to allow other IAM principals to access tables, you need to grant Lake Formation permissions on your tables to those principals. For more information, see [Managing access to a table or database with Lake Formation](./grant-permissions-tables.html). -