AWS prescriptive-guidance documentation change
Summary
Updated documentation for bulk DNS migration to Route 53, including enhanced prerequisites, architecture diagrams, best practices section with security recommendations, detailed implementation steps, troubleshooting guide, and additional information.
Security assessment
The changes add security best practices documentation including least privilege principles and data validation, but there's no evidence of addressing a specific security vulnerability. The updates improve security posture through guidance rather than fixing a reported issue.
Diff
diff --git a/prescriptive-guidance/latest/patterns/migrate-dns-records-in-bulk-to-an-amazon-route-53-private-hosted-zone.md b/prescriptive-guidance/latest/patterns/migrate-dns-records-in-bulk-to-an-amazon-route-53-private-hosted-zone.md index ecbf66515..93cd3afab 100644 --- a//prescriptive-guidance/latest/patterns/migrate-dns-records-in-bulk-to-an-amazon-route-53-private-hosted-zone.md +++ b//prescriptive-guidance/latest/patterns/migrate-dns-records-in-bulk-to-an-amazon-route-53-private-hosted-zone.md @@ -7 +7 @@ -SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resources +SummaryPrerequisites and limitationsArchitectureToolsBest practicesEpicsTroubleshootingRelated resourcesAdditional informationAttachments @@ -15 +15 @@ SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resources -Network engineers and cloud administrators need an efficient and simple way to add Domain Name System (DNS) records to private hosted zones in Amazon Route 53. Using a manual approach to copy entries from a Microsoft Excel worksheet to appropriate locations in the Route 53 console is tedious and error prone. This pattern describes an automated approach that reduces the time and effort required to add multiple records. It also provides a repeatable set of steps for multiple hosted zone creation. +This pattern automates the bulk migration of DNS records into an Amazon Route 53 hosted zone. It uses a Python script with the AWS SDK for Python (Boto3) to read record data from a JSON file stored in Amazon Simple Storage Service (Amazon S3) and create the records programmatically by using the Route 53 API. @@ -17 +17 @@ Network engineers and cloud administrators need an efficient and simple way to a -This pattern uses Amazon Simple Storage Service (Amazon S3) to store records. To work with data efficiently, the pattern uses the JSON format because of its simplicity and its ability to support a Python dictionary (`dict` data type). +If your source system exports standard BIND zone files, the Route 53 native import feature or [cli53 ](https://github.com/barnybug/cli53)(a community-maintained, open-source CLI tool not affiliated with AWS) can handle the migration without custom scripting. However, in many enterprise environments, DNS records live in Excel worksheets rather than zone files. Excel exports often contain data quality issues such as non-breaking spaces and Unicode characters, that standard import tools do not handle. This pattern addresses that gap: it reads from Excel and cleans the data to create records end-to-end. @@ -19,3 +19 @@ This pattern uses Amazon Simple Storage Service (Amazon S3) to store records. To -###### Note - -If you can generate a zone file from your system, consider using the [Route 53 import feature](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resource-record-sets-creating-import.html) instead. +Use this pattern when you need to migrate a large number of DNS records from an existing system into a Route 53 hosted zone without creating each record manually through the console. @@ -27 +25,7 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * An Excel worksheet that contains private hosted zone records + * An active AWS account + + * An Excel worksheet that contains hosted zone records with columns for fully qualified domain name (FQDN), record type, Time to Live (TTL), and value + + * Python 3.12 or later (see [Download Python](https://www.python.org/downloads/)) + + * The pandas Python library installed (`pip3 install pandas`) @@ -29 +33 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * Familiarity with different types of DNS records such as A record, Name Authority Pointer (NAPTR) record, and SRV record (see [Supported DNS record types](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/ResourceRecordTypes.html)) + * Familiarity with DNS record types such as A, NAPTR, and SRV records (see [Supported DNS record types](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/ResourceRecordTypes.html)) @@ -38 +42 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * The pattern doesn’t provide extensive coverage for all use case scenarios. For example, the [change_resource_record_sets](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/route53.html#Route53.Client.change_resource_record_sets) call doesn’t use all the available properties of the API. + * This pattern does not use all available properties of the `change_resource_record_sets` API call. Extend the code as needed for your use case. @@ -40 +44 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * In the Excel worksheet, the value in each row is assumed to be unique. Multiple values for each fully qualified domain name (FQDN) are expected to appear in the same row. If that is not true, you should modify the code provided in this pattern to perform the necessary concatenation. + * In the Excel worksheet, the value in each row is assumed to be unique. Multiple values for each FQDN are expected to appear in the same row. If that is not the case, modify the code to concatenate multiple values for the same FQDN. @@ -42 +46,5 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * The pattern uses the AWS SDK for Python (Boto3) to call the Route 53 service directly. You can enhance the code to use an AWS CloudFormation wrapper for the `create_stack` and `update_stack` commands, and use the JSON values to populate template resources. + * This pattern calls the Route 53 API directly by using Boto3. You can enhance the code to use an AWS CloudFormation wrapper for the `create_stack` and `update_stack` commands and use the JSON values to populate template resources. + + * If your source system can export records in BIND zone file format consider using the Route 53 import feature instead. + + * Some AWS services aren't available in all AWS Regions. For Region availability, see AWS[ services by Region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). For specific endpoints, see the [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-service-information.html) page, and choose the link for the service. @@ -49 +57,3 @@ If you can generate a zone file from your system, consider using the [Route 53 i -**Technology stack** + + + 1. The user uploads an Excel worksheet with DNS record data to an Amazon S3 bucket. @@ -51 +61 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * Route 53 private hosted zones for routing traffic + 2. A Python script running locally (or in an IDE such as Kiro) reads the JSON file from the Amazon S3 bucket by using Boto3. @@ -53 +63 @@ If you can generate a zone file from your system, consider using the [Route 53 i - * Amazon S3 for storing the output JSON file + 3. The script processes the records and calls the Route 53 `change_resource_record_sets` API operation to create records in the hosted zone. @@ -58 +68 @@ If you can generate a zone file from your system, consider using the [Route 53 i - +## Tools @@ -60 +70 @@ If you can generate a zone file from your system, consider using the [Route 53 i -The workflow consists of these steps, as illustrated in the previous diagram and discussed in the _Epics_ section: + _AWS services_ @@ -62 +72 @@ The workflow consists of these steps, as illustrated in the previous diagram and - 1. Upload an Excel worksheet that has the record set information to an S3 bucket. + * [Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/Welcome.html) – Provides the hosted zone where DNS records are created. This pattern uses the `create_hosted_zone` and `change_resource_record_sets` API operations. @@ -64 +74 @@ The workflow consists of these steps, as illustrated in the previous diagram and - 2. Create and run a Python script that converts the Excel data to JSON format. + * [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) \- Stores the JSON file that contains the DNS record data to migrate. @@ -66 +75,0 @@ The workflow consists of these steps, as illustrated in the previous diagram and - 3. Read the records from the S3 bucket and clean the data. @@ -68 +76,0 @@ The workflow consists of these steps, as illustrated in the previous diagram and - 4. Create record sets in your private hosted zone. @@ -70,0 +79 @@ The workflow consists of these steps, as illustrated in the previous diagram and +_Other tools_ @@ -71,0 +81 @@ The workflow consists of these steps, as illustrated in the previous diagram and + * **Kiro** — An IDE with agentic AI capabilities, used to develop and run the Python migration script. For more information, see [Kiro](https://kiro.dev/). @@ -73 +83,8 @@ The workflow consists of these steps, as illustrated in the previous diagram and -## Tools + * **pandas** — A Python data analysis library, used to convert the Excel worksheet to JSON format. + + + + +## Best practices + + * Test with a small subset of 5–10 records before you run the script against all records to verify data formatting and correct record creation. @@ -75 +92 @@ The workflow consists of these steps, as illustrated in the previous diagram and - * [Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/Welcome.html) – Amazon Route 53 is a highly available and scalable DNS web service that handles domain registration, DNS routing, and health checking. + * Use the `UPSERT` action instead of `CREATE`. The `UPSERT` action creates a record if it doesn't exist or updates it if it does. This makes the script idempotent and safe to rerun. @@ -77 +94,9 @@ The workflow consists of these steps, as illustrated in the previous diagram and - * [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) – Amazon Simple Storage Service (Amazon S3) is an object storage service. You can use Amazon S3 to store and retrieve any amount of data at any time, from anywhere on the web. + * [Batch your changes](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/DNSLimitations.html#limits-api-requests-changeresourcerecordsets) to stay within the `ChangeResourceRecordSets` request limits. When you use the `UPSERT` action, each `ResourceRecord` element counts twice toward the 1,000-element maximum. + + * [Validate data before insertion](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/ResourceRecordTypes.html). Verify that FQDNs end with a period (`.`), that TTL values are positive integers, and that record types match the supported DNS record types. + + * [Back up existing records ](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/route53.html#Route53.Client.list_resource_record_sets) before you make bulk changes. Export the current hosted zone records by using `list_resource_record_sets` to provide a rollback point. + + * Implement retry logic with exponential backoff to handle Route 53 API rate limits. For more information, see [Error retries and exponential backoff in AWS](https://docs.aws.amazon.com/general/latest/gr/api-retries.html). + + * Follow the principle of least privilege and grant the minimum permissions required to perform a task. For more information, see [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege) and [Security best practices](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html) in the AWS Identity and Access Management (IAM) documentation. @@ -86,6 +111 @@ Task| Description| Skills required -Create an Excel file for your records.| Use the records you exported from your current system to create an Excel worksheet that has the required columns for a record, such as fully qualified domain name (FQDN), record type, Time to Live (TTL), and value. For NAPTR and SRV records, the value is a combination of multiple properties, so use Excel's `concat` method to combine these properties.| FqdnName| RecordType| Value| TTL ----|---|---|--- -something.example.org| A| 1.1.1.1| 900 - -Data engineer, Excel skills -Verify the working environment.| In your IDE, create a Python file to convert the Excel input worksheet to JSON format. (Instead of an IDE, you can also use an Amazon SageMaker notebook to work with Python code.)Verify that the Python version you’re using is version 3.7 or later. +Create an Excel file for your records.| @@ -92,0 +113,4 @@ Verify the working environment.| In your IDE, create a Python file to convert th + 1. Export records from your current DNS system. + 2. Create an Excel worksheet with these columns: `FqdnName`, `RecordType`, `Value`, `TTL`. + 3. For NAPTR and SRV records, use Excel's CONCAT function to combine multiple property values into the `Value` column. + 4. Verify that each FQDN ends with a period (`.`) and that TTL values are positive integers. @@ -94 +118,3 @@ Verify the working environment.| In your IDE, create a Python file to convert th - python3 --version +Example row: | FqdnName| RecordType| Value| TTL +---|---|---|--- +something.example.org| A| 192.0.2.1| 900 @@ -96 +122,2 @@ Verify the working environment.| In your IDE, create a Python file to convert th -Install the **pandas** package. +Data engineer +Convert the Excel worksheet data to JSON.| @@ -97,0 +125,3 @@ Install the **pandas** package. + 1. Verify that Python 3.12 or later is installed by running `python3 --version` in the terminal. + 2. Install the pandas library by running `pip3 install pandas --user` in the terminal. + 3. Create a Python file with the following code to convert the Excel worksheet to JSON: @@ -99 +128,0 @@ Install the **pandas** package. - pip3 install pandas --user @@ -101,2 +129,0 @@ Install the **pandas** package. -| General AWS -Convert the Excel worksheet data to JSON.| Create a Python file that contains the following code to convert from Excel to JSON. @@ -109,2 +137,9 @@ Convert the Excel worksheet data to JSON.| Create a Python file that contains th -where `Book1` is the name of the Excel worksheet and `my.json` is the name of the output JSON file.| Data engineer, Python skills -Upload the JSON file to an S3 bucket.| Upload the `my.json` file to an S3 bucket. For more information, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) in the Amazon S3 documentation.| App developer + 1. Run the script. Verify that the output file `my.json` contains the expected records. + +| Data engineer, Python developer +Upload the JSON file to an Amazon S3 bucket.| + + 1. Create an Amazon S3 bucket if one does not already exist. For more information, see [Creating a bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-overview.html) in the Amazon S3 documentation. + 2. Upload the `my.json` file to the bucket. + +| App developer @@ -114 +149,5 @@ Task| Description| Skills required -Create a private hosted zone.| Use the [create_hosted_zone](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/route53.html#Route53.Client.create_hosted_zone) API and the following Python sample code to create a private hosted zone. Replace the parameters `hostedZoneName`, `vpcRegion`, and `vpcId` with your own values. +Create a hosted zone.| + + 1. Create a Python file with the following code to create a hosted zone. Replace `<hosted-zone-name>`, `<vpc-region>`, and `<vpc-id>` with your values. + + @@ -126 +166 @@ Create a private hosted zone.| Use the [create_hosted_zone](https://boto3.amazon - 'VPCRegion: vpcRegion, + 'VPCRegion': vpcRegion, @@ -137,2 +178,9 @@ Create a private hosted zone.| Use the [create_hosted_zone](https://boto3.amazon -You can also use an infrastructure as code (IaC) tool such as AWS CloudFormation to replace these steps with a template that creates a stack with the appropriate resources and properties.| Cloud architect, Network administrator, Python skills -Retrieve details as a dictionary from Amazon S3.| Use the following code to read from the S3 bucket and to get the JSON values as a Python dictionary. + 1. Run the script and note the hosted zone ID from the response. + +You can also use an infrastructure as code (IaC) tool such as AWS CloudFormation to replace these steps with a template that creates a stack with the appropriate resources and properties. For more information, see [AWS Route 53::HostedZone](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-resource-route53-hostedzone.html) in the CloudFormation documentation.| Cloud architect, Network administrator, Python skills +Read the JSON data from Amazon S3.| + + 1. Use this code to read from the Amazon S3 bucket and load it as a Python dictionary. + + + @@ -139,0 +188,2 @@ Retrieve details as a dictionary from Amazon S3.| Use the following code to read + import json + import boto3 @@ -140,0 +191 @@ Retrieve details as a dictionary from Amazon S3.| Use the following code to read + s3_client = boto3.client('s3') @@ -142 +193 @@ Retrieve details as a dictionary from Amazon S3.| Use the following code to read - Bucket=bucket_name, + Bucket='<your-bucket-name>', @@ -148 +199,0 @@ Retrieve details as a dictionary from Amazon S3.| Use the following code to read - print(json_content) @@ -150,2 +201 @@ Retrieve details as a dictionary from Amazon S3.| Use the following code to read -where `json_content` contains the Python dictionary.| App developer, Python skills -Clean data values for spaces and Unicode characters.| As a safety measure to ensure the correctness of data, use the following code to perform a strip operation on the values in `json_content`. This code removes the space characters at the front and end of each string. It also uses the `replace` method to remove hard (non-breaking) spaces (the `\xa0` characters). + 1. Verify that `json_content` contains the expected records by printing the first few entries. @@ -152,0 +203,4 @@ Clean data values for spaces and Unicode characters.| As a safety measure to ens +| App developer, Python developer +Clean data values and insert records.| + + 1. Use this code to clean data values and insert records into the hosted zone. Replace `<hosted-zone-id>` with the ID from the previously configured hosted zone. @@ -154,6 +207,0 @@ Clean data values for spaces and Unicode characters.| As a safety measure to ens - for item in json_content: - fqn_name = unicodedata.normalize("NFKD",item["FqdnName"].replace("u'", "'").replace('\xa0', '').strip()) - rec_type = item["RecordType"].replace('\xa0', '').strip() - res_rec = { - 'Value': item["Value"].replace('\xa0', '').strip() - } @@ -161,2 +208,0 @@ Clean data values for spaces and Unicode characters.| As a safety measure to ens -| App developer, Python skills -Insert records.| Use the following code as part of the previous `for` loop. @@ -164,0 +211,11 @@ Insert records.| Use the following code as part of the previous `for` loop. + import unicodedata + import time + + for item in json_content: + fqn_name = unicodedata.normalize( + "NFKD", + item["FqdnName"].replace("u'", "'").replace('\xa0', '').strip()