AWS glue documentation change
Summary
Updated supported Glue versions (4.0 and 5.0), replaced 'sandbox environment' with 'environment', changed upgrade target from 4.0 to 5.0, and added guidance about using idempotent jobs during validation
Security assessment
Changes focus on version compatibility, terminology updates, and operational best practices. The idempotent jobs guidance addresses reliability during validation but does not directly mitigate a security vulnerability.
Diff
diff --git a/glue/latest/dg/upgrade-analysis.md b/glue/latest/dg/upgrade-analysis.md index 5dd33e03b..6765c981e 100644 --- a//glue/latest/dg/upgrade-analysis.md +++ b//glue/latest/dg/upgrade-analysis.md @@ -9 +9 @@ How it worksPrerequisitesRunning an upgrade analysis and applying the upgrade sc -The generative AI upgrades for Apache Spark preview is available for AWS Glue in the following AWS Regions: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and Asia Pacific (Sydney). Preview features are subject to change. +The generative AI upgrades for Apache Spark preview is available for AWS Glue versions 4.0 and 5.0 in the following AWS Regions: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Tokyo), and Asia Pacific (Sydney). Preview features are subject to change. @@ -18 +18 @@ Spark Upgrades in AWS Glue enables data engineers and developers to upgrade and -When you use upgrade analysis, AWS Glue identifies differences between versions and configurations in your job's code to generate an upgrade plan. The upgrade plan details all code changes, and required migration steps. Next, AWS Glue builds and runs the upgraded application in a sandbox environment to validate changes and generates a list of code changes for you to migrate your job. You can view the updated script along with the summary that details the proposed changes. After running your own tests, accept the changes and the AWS Glue job will be updated automatically to the latest version with the new script. +When you use upgrade analysis, AWS Glue identifies differences between versions and configurations in your job's code to generate an upgrade plan. The upgrade plan details all code changes, and required migration steps. Next, AWS Glue builds and runs the upgraded application in an environment to validate changes and generates a list of code changes for you to migrate your job. You can view the updated script along with the summary that details the proposed changes. After running your own tests, accept the changes and the AWS Glue job will be updated automatically to the latest version with the new script. @@ -26 +26 @@ The following prerequisites are required to use generative AI to upgrade jobs in - * AWS Glue 2 PySpark jobs – only AWS Glue 2 jobs can be upgraded to AWS Glue 4. + * AWS Glue 2 PySpark jobs – only AWS Glue 2 jobs can be upgraded to AWS Glue 5. @@ -461 +461 @@ As you begin using Spark Upgrades during the preview period, there are several i - * **Service Scope and Limitations** : The preview release focuses on PySpark code upgrades from AWS Glue versions 2.0 to version 4.0. At this time, the service handles PySpark code that doesn't rely on additional library dependencies. You can run automated upgrades for up to 10 jobs concurrently in an AWS account, allowing you to efficiently upgrade multiple jobs while maintaining system stability. + * **Service Scope and Limitations** : The preview release focuses on PySpark code upgrades from AWS Glue versions 2.0 to version 5.0. At this time, the service handles PySpark code that doesn't rely on additional library dependencies. You can run automated upgrades for up to 10 jobs concurrently in an AWS account, allowing you to efficiently upgrade multiple jobs while maintaining system stability. @@ -484,0 +485,2 @@ For example, if your production job processes terabytes of data with 20 G.2X wor + * **Use upgrade analysis with idempotent jobs** : Use upgrade analysis with idempotent jobs to ensure each subsequent validation job run attempt is similar to the previous one, and doesn't run into issues. Idempotent jobs are jobs that can be run multiple times with the same input data, and they will produce the same output each time. When using the Generative AI upgrades for Apache Spark in AWS Glue, the service will run multiple iterations of your job as part of the validation process. During each iteration, it will make changes to your Spark code and configurations to validate the upgrade plan. If your Spark job is not idempotent, running it multiple times with the same input data could lead to issues. +