AWS entityresolution documentation change
Summary
Added Step 4 for creating partitioned AWS Glue tables in ID mapping workflows, updated crawler creation instructions, and removed outdated limitations about partitioned table support
Security assessment
Changes focus on operational improvements for partitioned data processing efficiency and documentation updates. No security vulnerabilities, access controls, or encryption methods are mentioned. The partitioning feature relates to data processing optimization rather than security controls.
Diff
diff --git a/entityresolution/latest/userguide/prepare-input-data.md b/entityresolution/latest/userguide/prepare-input-data.md index a433a39d9..3b4c50415 100644 --- a/entityresolution/latest/userguide/prepare-input-data.md +++ b/entityresolution/latest/userguide/prepare-input-data.md @@ -5 +5 @@ -Step 1: Save your input data table in a supported data formatStep 2: Upload your input data table to Amazon S3Step 3: Create an AWS Glue table +Step 1: Save your input data table in a supported data formatStep 2: Upload your input data table to Amazon S3Step 3: Create an AWS Glue tableStep 4: Create a partitioned AWS Glue table @@ -49,2 +48,0 @@ You can select the folder to view the data table. -The input data in Amazon S3 must be cataloged in AWS Glue and represented as an AWS Glue table. For more information about how to create an AWS Glue table with Amazon S3 as the input, see [Working with crawlers on the AWS Glue console](https://docs.aws.amazon.com/glue/latest/dg/console-crawlers.html) in the _AWS Glue Developer Guide._ - @@ -53 +51,3 @@ The input data in Amazon S3 must be cataloged in AWS Glue and represented as an -AWS Entity Resolution doesn't support partitioned tables. +If you need partitioned AWS Glue tables, skip to Step 4: Create a partitioned AWS Glue table. + +The input data in Amazon S3 must be cataloged in AWS Glue and represented as an AWS Glue table. For more information about how to create an AWS Glue table with Amazon S3 as the input, see [Working with crawlers on the AWS Glue console](https://docs.aws.amazon.com/glue/latest/dg/console-crawlers.html) in the _AWS Glue Developer Guide._ @@ -67 +67 @@ AWS Entity Resolution doesn't currently support Amazon S3 locations registered w - 3. Select your S3 bucket from the list, and then choose **Add crawler**. + 3. Select your S3 bucket from the list, and then choose **Create crawler**. @@ -69 +69 @@ AWS Entity Resolution doesn't currently support Amazon S3 locations registered w - 4. On the **Add crawler** page, enter a **Crawler name** and then choose **Next**. + 4. On the **Set crawler properties** page, enter a crawler**Name** optional **Description** , and then choose **Next**. @@ -99,0 +100,72 @@ You are now ready to create a schema mapping. For more information, see [Creatin +## Step 4: Create a partitioned AWS Glue table + +###### Note + +The AWS Glue partitioning feature in AWS Entity Resolution is only supported in ID mapping workflows. This AWS Glue partitioning feature enables you to choose specific partitions for processing with AWS Entity Resolution. + +If you don't need partitioned AWS Glue tables, you can skip this step. + +A partitioned AWS Glue table automatically reflects new partitions in the AWS Glue table when you add new folders to the data structure (such as a new day folder under a month). + +When you create a partitioned AWS Glue table in AWS Entity Resolution, you can specify which partitions you want to process in an ID mapping workflow. Then, every time you run the ID mapping workflow, only the data in those partitions are processed, rather than processing all of the data in the entire AWS Glue table. This feature allows for more precise, efficient, and cost-effective data processing in AWS Entity Resolution, giving you greater control and flexibility in managing your entity resolution tasks. + +You can create a partitioned AWS Glue table for the source account in an ID mapping workflow. + +You must first catalog the input data in Amazon S3 in AWS Glue and represented it as an AWS Glue table. For more information about how to create an AWS Glue table with Amazon S3 as the input, see [Working with crawlers on the AWS Glue console](https://docs.aws.amazon.com/glue/latest/dg/console-crawlers.html) in the _AWS Glue Developer Guide._ + +In this step, you set up a crawler in AWS Glue that crawls all the files in your S3 bucket and then create a partitioned AWS Glue table. + +###### Note + +AWS Entity Resolution doesn't currently support Amazon S3 locations registered with AWS Lake Formation. + +###### To create a partitioned AWS Glue table + + 1. Sign in to the AWS Management Console and open the AWS Glue console at [https://console.aws.amazon.com/glue/](https://console.aws.amazon.com/glue/). + + 2. From the navigation bar, select **Crawlers**. + + 3. Select your S3 bucket from the list, and then choose **Create crawler**. + + 4. On the **Set crawler properties** page, enter a crawler **Name** , optional **Description** , and then choose **Next**. + + 5. Continue through the **Add crawler page** , specifying the details. + + 6. On the **Choose an IAM role** page, choose **Choose an existing IAM role** and then choose **Next**. + +You can also choose **Create an IAM role** or have your administrator create the IAM role if needed. + + 7. For **Create a schedule for this crawler** , keep the **Frequency** default (**Run on demand**) and then choose **Next**. + + 8. For **Configure the crawler’s output** , enter the AWS Glue database and then choose **Next**. + + 9. Review all the details, and then choose **Finish**. + + 10. On the **Crawlers** page, select the check box next to your S3 bucket and then choose **Run crawler**. + + 11. After the crawler is finished running, on the AWS Glue navigation bar, choose **Databases** , and then choose your database name. + + 12. On the **Database** page, under **Tables** , choose the table to be partitioned. + + 13. On the **Table overview** , select the **Actions** dropdown, and then choose **Edit table**. + + 1. Under **Table properties** , choose **Add**. + + 2. For the new **Key** , enter `aerPushDownPredicateString`. + + 3. For the new **Value** , enter `'<PartitionKey>=<PartitionValue'`. + + 4. Make a note of the AWS Glue database name and AWS Glue table name. + + + + +You are now ready to: + + * [Create a schema mapping](./create-schema-mapping.html) and then [create an ID mapping workflow for one AWS account](./creating-id-mapping-workflow-same-account.html). + + * [Create an ID namespace source](./create-id-namespace-source.html), [create an ID namespace target](./create-id-namespace-target.html), and then [create an ID mapping workflow across two AWS accounts](./creating-id-mapping-workflow-two-accounts.html). + + + +