AWS Security ChangesHomeSearch

AWS clean-rooms: Document PySpark 2.0 version, result formats, and service role permissions

Service: clean-rooms · 2026-10-31 · Documentation low

File: clean-rooms/latest/userguide/create-pyspark-analysis-template.md

Summary

Adds a PySpark 1.0 vs 2.0 feature comparison (result formats, CSV formatting, service role permissions), API/CLI guidance for the format and resultFormat parameters, reorders the parameters and tables-referenced steps, and adds a 'View policy document' option for service access permissions.

Security assessment

The change is primarily a feature/version documentation update for PySpark analysis templates. It mentions the IAM actions (s3:GetBucketLocation, s3:ListBucket) required by the service role and adds a 'View policy document' UI hint, but these are functional requirements rather than documentation of a new security feature, best practice, or a fixed vulnerability. No CVE or incident is referenced.

Evidence

+Permissions for the service role that writes job results |  The role must allow the `s3:GetBucketLocation` and `s3:ListBucket` actions on the result bucket. For more information, see [Create a service role to write results of a PySpark job](./setting-up-roles.html#create-role-pyspark-job). |  Same as PySpark 1.0

Diff

diff --git a/clean-rooms/latest/userguide/create-pyspark-analysis-template.md b/clean-rooms/latest/userguide/create-pyspark-analysis-template.md
index ff01db5..651d722 100644
--- a//clean-rooms/latest/userguide/create-pyspark-analysis-template.md
+++ b//clean-rooms/latest/userguide/create-pyspark-analysis-template.md
@@ -19,0 +20,15 @@ Parameters are user-provided strings that can contain arbitrary content.
+**PySpark versions**
+
+When you create a PySpark analysis template, you choose the PySpark version for the template. Use PySpark 2.0 for new analysis templates. PySpark 1.0 remains supported, and existing PySpark 1.0 analysis templates continue to work as before.
+
+The following table compares the features that each version supports.
+
+Feature | PySpark 1.0 | PySpark 2.0 (recommended)  
+---|---|---  
+Result output formats |  CSV only |  CSV or Parquet  
+CSV formatting |  Original CSV formatting |  Updated. For example, PySpark 2.0 writes Boolean values as `true` and `false`, includes milliseconds in `TIMESTAMP_NTZ` values, and writes empty strings as `""`. For the full list of changes, see [Differences between PySpark 1.0 and PySpark 2.0 analysis templates](./pyspark-template-versions.html).  
+Permissions for the service role that writes job results |  The role must allow the `s3:GetBucketLocation` and `s3:ListBucket` actions on the result bucket. For more information, see [Create a service role to write results of a PySpark job](./setting-up-roles.html#create-role-pyspark-job). |  Same as PySpark 1.0  
+User script and virtual environment |  See [Creating a user script](./create-user-script.html) and [Creating a virtual environment (optional)](./create-virtual-environment.html). |  Same as PySpark 1.0  
+  
+As the member who receives results, you choose the result format for your membership. To receive results in Parquet format, the analysis template must use PySpark 2.0. For how to set the result format, see the API and AWS CLI information later in this topic.
+
@@ -42,0 +58,13 @@ For information about creating the required service role, see [Create a service
+For information about how to create a PySpark analysis template using the AWS SDKs, see the [AWS Clean Rooms API Reference](https://docs.aws.amazon.com/clean-rooms/latest/apireference/Welcome.html).
+
+When you use the API or the AWS CLI, specify the PySpark version in the `format` parameter of the `CreateAnalysisTemplate` operation:
+
+  * `PYSPARK_2_0` – PySpark 2.0 (recommended)
+
+  * `PYSPARK_1_0` – PySpark 1.0
+
+
+
+
+To receive results in Parquet format, set `resultFormat` to `PARQUET` in your membership's job result configuration. This requires an analysis template with the `PYSPARK_2_0` format. If you set the result format to Parquet, jobs that use a `PYSPARK_1_0` analysis template fail validation.
+
@@ -49,2 +76,0 @@ The member who creates the PySpark analysis template must also be the member who
-For information about how to create a PySpark analysis template using the AWS SDKs, see the [AWS Clean Rooms API Reference](https://docs.aws.amazon.com/clean-rooms/latest/apireference/Welcome.html).
-
@@ -59 +85 @@ For information about how to create a PySpark analysis template using the AWS SD
-  4. On the **Templates** tab, go to the **Analysis templates created by you** section.
+  4. On the **Templates** tab, go to the **Analysis templates** table.
@@ -70,0 +97,2 @@ For information about how to create a PySpark analysis template using the AWS SD
+    4. Choose the PySpark version for the analysis template. AWS Clean Rooms recommends PySpark 2.0. For more information, see [Differences between PySpark 1.0 and PySpark 2.0 analysis templates](./pyspark-template-versions.html).
+
@@ -79 +107 @@ For information about how to create a PySpark analysis template using the AWS SD
-  8. For **Parameters – optional** , if you want to add parameters to make your analysis template reusable:
+  8. For **Tables referenced in the definition** , 
@@ -81 +109 @@ For information about how to create a PySpark analysis template using the AWS SD
-    1. Choose **Add parameter**.
+     * If all tables referenced in the definition have been associated to the collaboration:
@@ -83 +111 @@ For information about how to create a PySpark analysis template using the AWS SD
-    2. Enter a **Parameter name**.
+       * Leave the **All tables referenced in the definition have been associated to the collaboration** checkbox selected.
@@ -85 +113 @@ For information about how to create a PySpark analysis template using the AWS SD
-Parameter names must start with a letter or underscore, followed by alphanumeric characters or underscores.
+       * Under **Tables associated to the collaboration** , choose all associated tables that are referenced in the definition. 
@@ -87 +115 @@ Parameter names must start with a letter or underscore, followed by alphanumeric
-    3. For **Type** , **STRING** is automatically selected as the only supported type for PySpark analysis templates.
+     * If all tables referenced in the definition haven't been associated to the collaboration:
@@ -89 +117 @@ Parameter names must start with a letter or underscore, followed by alphanumeric
-    4. (Optional) Enter a **Default value** for the parameter.
+       * Clear the **All tables referenced in the definition have been associated to the collaboration** checkbox.
@@ -91 +119 @@ Parameter names must start with a letter or underscore, followed by alphanumeric
-If you provide a default value, job runners can use this value when running jobs without explicitly providing a parameter value.
+       * Under **Tables associated to the collaboration** , choose all associated tables that are referenced in the definition.
@@ -93 +121 @@ If you provide a default value, job runners can use this value when running jobs
-    5. To add more parameters, choose **Add another parameter** and repeat the previous steps.
+       * Under **Tables that will be associated later** , enter a table name. 
@@ -95 +123 @@ If you provide a default value, job runners can use this value when running jobs
-###### Note
+       * Choose **List another table** to list another table.
@@ -97 +125 @@ If you provide a default value, job runners can use this value when running jobs
-You can define up to 50 parameters per PySpark analysis template. Each parameter value can be up to 1,000 characters.
+  9. For **Parameters – optional** , if you want to add parameters to make your analysis template reusable:
@@ -99 +127 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-  9. For **Tables referenced in the definition** , 
+    1. Choose **Add parameter**.
@@ -101 +129 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-     * If all tables referenced in the definition have been associated to the collaboration:
+    2. Enter a **Parameter name**.
@@ -103 +131 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Leave the **All tables referenced in the definition have been associated to the collaboration** checkbox selected.
+Parameter names must start with a letter or underscore, followed by alphanumeric characters or underscores.
@@ -105 +133 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Under **Tables associated to the collaboration** , choose all associated tables that are referenced in the definition. 
+    3. For **Type** , **STRING** is automatically selected as the only supported type for PySpark analysis templates.
@@ -107 +135 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-     * If all tables referenced in the definition haven't been associated to the collaboration:
+    4. (Optional) Enter a **Default value** for the parameter.
@@ -109 +137 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Clear the **All tables referenced in the definition have been associated to the collaboration** checkbox.
+If you provide a default value, job runners can use this value when running jobs without explicitly providing a parameter value.
@@ -111 +139 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Under **Tables associated to the collaboration** , choose all associated tables that are referenced in the definition.
+    5. To add more parameters, choose **Add another parameter** and repeat the previous steps.
@@ -113 +141 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Under **Tables that will be associated later** , enter a table name. 
+###### Note
@@ -115 +143 @@ You can define up to 50 parameters per PySpark analysis template. Each parameter
-       * Choose **List another table** to list another table.
+You can define up to 50 parameters per PySpark analysis template. Each parameter value can be up to 1,000 characters.
@@ -127 +155 @@ When using **Detailed error messages** , all data provider members must approve
-  11. Specify the **Service access** permissions by selecting an **Existing service role name** from the dropdown list.
+  11. Specify the **Service access** permissions by selecting an **Existing service role name** from the dropdown list. To view the minimum permissions that the role needs, choose **View policy document**.