AWS deadline-cloud documentation change
Summary
Complete restructuring and expansion of the environment variables guide. Added sections on environment variable scope, setting variables in queue environments and job templates, and a detailed walkthrough of a sample job bundle. Replaced a simple example with comprehensive documentation including YAML examples, CLI commands, and log outputs.
Security assessment
The changes are purely instructional and focus on feature functionality (setting environment variables at different scopes). There is no mention of vulnerabilities, patches, security incidents, or specific security configurations. The update improves clarity and adds examples but does not address a security weakness or introduce security-specific guidance.
Diff
diff --git a/deadline-cloud/latest/developerguide/set-environment-variables.md b/deadline-cloud/latest/developerguide/set-environment-variables.md index d985b9a33..e80a91870 100644 --- a//deadline-cloud/latest/developerguide/set-environment-variables.md +++ b//deadline-cloud/latest/developerguide/set-environment-variables.md @@ -5 +5 @@ -PrerequisitesRun the environment variable sampleCompare the session actions with their definitions +Environment variable scopeSetting variables in a queue environmentSetting variables in a job templateTry it: Running the environment variable sample @@ -9 +9 @@ PrerequisitesRun the environment variable sampleCompare the session actions with -[Open Job Description (OpenJD) environments](https://github.com/OpenJobDescription/openjd-specifications/wiki/2023-09-Template-Schemas#4-environment) can set environment variables that every task command within their scope uses. Many applications and frameworks check for environment variables to control feature settings, logging level, and more. +Many applications and frameworks use environment variables to control feature settings, logging levels, and display configuration. You can use [Open Job Description (OpenJD) environments](https://github.com/OpenJobDescription/openjd-specifications/wiki/2023-09-Template-Schemas#4-environment) to set environment variables that every task command within their scope inherits. @@ -11 +11 @@ PrerequisitesRun the environment variable sampleCompare the session actions with -For example, the [Qt Framework](https://www.qt.io/product/framework) provides GUI functionality for many desktop applications. When you run these applications on a worker host without an interactive display, you may need to set the environment variable `QT_QPA_PLATFORM` to `offscreen` so the worker doesn’t look for a display. +## Environment variable scope @@ -13 +13 @@ For example, the [Qt Framework](https://www.qt.io/product/framework) provides GU -In this example, you'll use a sample job bundle from the Deadline Cloud samples directory to set and view the environment variables for a job. +AWS Deadline Cloud applies environment variables from queue environments that you attach to a queue. Within a job template, you can also define environment variables at the job and step levels using [OpenJD environments](https://github.com/OpenJobDescription/openjd-specifications/wiki/2023-09-Template-Schemas#4-environment). Variables defined at a narrower scope override variables with the same name from a broader scope. @@ -15 +15 @@ In this example, you'll use a sample job bundle from the Deadline Cloud samples -## Prerequisites + * **Queue environment** – A template that you attach to a queue in Deadline Cloud. Variables apply to all jobs submitted to the queue. You can set variables with a `variables` map for fixed values, or use scripts for dynamic values. @@ -17 +17,108 @@ In this example, you'll use a sample job bundle from the Deadline Cloud samples -Perform the following steps to run the [sample job bundle with environment variables](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/job_bundles/job_env_vars/template.yaml) from the Deadline Cloud samples github repository. + * **Job environment** – Defined under `jobEnvironments` in a job template. Variables apply to all steps and tasks in the job. A job-level variable overrides a queue-level variable with the same name. + + * **Step environment** – Defined under `stepEnvironments` in a job template. Variables apply only to the tasks in that step. A step-level variable overrides a job-level or queue-level variable with the same name. + + + + +## Setting variables in a queue environment + +You can set environment variables in a queue environment using a `variables` map for fixed values, or using a `script` with an `onEnter` action for dynamic values. + +The following queue environment template uses a `variables` map to set the `QT_QPA_PLATFORM` variable to `offscreen`, which allows applications that use the [Qt Framework](https://www.qt.io/product/framework) to run on worker hosts without an interactive display. + + + specificationVersion: 'environment-2023-09' + environment: + name: QtOffscreen + variables: + QT_QPA_PLATFORM: offscreen + +For dynamic values, such as modifying `PATH` or activating virtual environments, use a script that prints lines in the format `openjd_env: `VAR`=`value`` to stdout. The `openjd_env:` prefix is required. Using `echo`, `export`, or other shell mechanisms without the prefix does not propagate variables to jobs and tasks. + +The following queue environment template sets the `QT_QPA_PLATFORM` variable using a script. + + + specificationVersion: 'environment-2023-09' + environment: + name: QtOffscreen + script: + actions: + onEnter: + command: bash + args: + - "{{Env.File.Enter}}" + embeddedFiles: + - name: Enter + type: TEXT + data: | + #!/bin/env bash + set -euo pipefail + echo "openjd_env: QT_QPA_PLATFORM=offscreen" + +To attach a queue environment to your queue, use the Deadline Cloud console or the AWS CLI. For more information, see [Create a queue environment](https://docs.aws.amazon.com/deadline-cloud/latest/userguide/create-queue-environment.html) in the AWS Deadline Cloud User Guide. The following AWS CLI command creates a queue environment from a template file. + + + aws deadline create-queue-environment \ + --farm-id FARM_ID \ + --queue-id QUEUE_ID \ + --priority 1 \ + --template-type YAML \ + --template file://my-queue-env.yaml + +For more complex examples, such as creating and activating conda virtual environments, see the [Deadline Cloud queue environment samples](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/queue_environments) on GitHub. + +## Setting variables in a job template + +In a job template, add a `variables` map to a `jobEnvironments` or `stepEnvironments` entry. Each entry is a key-value pair where the key is the variable name and the value is the variable value. + +The following job template sets the `QT_QPA_PLATFORM` environment variable to `offscreen`, which allows applications that use the [Qt Framework](https://www.qt.io/product/framework) to run on worker hosts without an interactive display. + + + specificationVersion: 'jobtemplate-2023-09' + name: MyJob + jobEnvironments: + - name: JobEnv + variables: + QT_QPA_PLATFORM: offscreen + +You can set multiple variables in a single environment definition. + + + jobEnvironments: + - name: JobEnv + variables: + JOB_VERBOSITY: MEDIUM + JOB_PROJECT_ID: my-project-id + JOB_ENDPOINT_URL: https://my-host-name/my/path + QT_QPA_PLATFORM: offscreen + +You can reference job parameters in variable values by using the `{{Param.`ParameterName`}}` syntax. + + + jobEnvironments: + - name: JobEnv + variables: + JOB_EXAMPLE_PARAM: "{{Param.ExampleParam}}" + +To override a job-level variable for a specific step, define a `stepEnvironments` entry with the same variable name. The following example defines `JOB_PROJECT_ID` at the job level with the value `project-12`, and then overrides the value at the step level with `step-project-12`. Tasks in the step use the step-level value. + + + specificationVersion: 'jobtemplate-2023-09' + name: MyJob + jobEnvironments: + - name: JobEnv + variables: + JOB_PROJECT_ID: project-12 + steps: + - name: MyStep + stepEnvironments: + - name: StepEnv + variables: + JOB_PROJECT_ID: step-project-12 + +## Try it: Running the environment variable sample + +The Deadline Cloud samples repository includes a [job bundle that demonstrates setting and viewing environment variables](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/job_bundles/job_env_vars/template.yaml). The sample job template defines variables at both the job and step levels, then runs a task that prints the merged result. Use the following procedure to run the sample and inspect the results. + +### Prerequisites @@ -21 +128 @@ Perform the following steps to run the [sample job bundle with environment varia - 2. If you do not have the Deadline Cloud CLI and Deadline Cloud monitor on your workstation, follow the steps in [Set up Deadline Cloud submitters](https://docs.aws.amazon.com/deadline-cloud/latest/userguide/submitter.html) from the user guide. + 2. If you do not have the Deadline Cloud CLI and AWS Deadline Cloud monitor on your workstation, follow the steps in [Set up Deadline Cloud submitters](https://docs.aws.amazon.com/deadline-cloud/latest/userguide/submitter.html). @@ -26,2 +132,0 @@ Perform the following steps to run the [sample job bundle with environment varia - Cloning into 'deadline-cloud-samples'... - ... @@ -33 +138 @@ Perform the following steps to run the [sample job bundle with environment varia -## Run the environment variable sample +### Running the sample @@ -38,4 +142,0 @@ Perform the following steps to run the [sample job bundle with environment varia - Submitting to Queue: MySampleQueue - ... - - 2. In the Deadline Cloud monitor, you can see the new job and monitor its progress. After the Linux fleet associated with the queue has a worker available to run the job’s task, the job completes in a few seconds. Select the task, then choose the **View logs** option in the top right menu of the tasks panel. @@ -43 +144 @@ Perform the following steps to run the [sample job bundle with environment varia -On the right are three session actions, **Launch JobEnv** , **Launch StepEnv** , and **Task run**. The log view in the center of the window corresponds to the selected session action on the right. + 2. In the Deadline Cloud monitor, select the new job to monitor its progress. After the Linux fleet associated with the queue has a worker available, the job completes in a few seconds. Select the task, then choose **View logs** in the top right menu of the tasks panel. @@ -48 +149 @@ On the right are three session actions, **Launch JobEnv** , **Launch StepEnv** , -## Compare the session actions with their definitions +### Comparing session actions with their definitions @@ -50 +151 @@ On the right are three session actions, **Launch JobEnv** , **Launch StepEnv** , -In this section you use the Deadline Cloud monitor to compare the session actions with where they are defined in the job template. It continues from the previous section. +The log view shows three session actions. Open the file [job_env_vars/template.yaml](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/job_bundles/job_env_vars/template.yaml) in a text editor to compare each action with its definition in the job template. @@ -52 +153 @@ In this section you use the Deadline Cloud monitor to compare the session action -Open the file [job_env_vars/template.yaml](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/job_bundles/job_env_vars/template.yaml) in a text editor. This is the job template that defines the session actions. + 1. Select the **Launch JobEnv** session action. The log output shows the job-level environment variables being set. @@ -54 +155,5 @@ Open the file [job_env_vars/template.yaml](https://github.com/aws-deadline/deadl - 1. Select the **Launch JobEnv** session action in Deadline Cloud monitor. You will see the following log output. + Setting: JOB_VERBOSITY=MEDIUM + Setting: JOB_EXAMPLE_PARAM=An example parameter value + Setting: JOB_PROJECT_ID=project-12 + Setting: JOB_ENDPOINT_URL=https://internal-host-name/some/path + Setting: QT_QPA_PLATFORM=offscreen @@ -56,11 +161 @@ Open the file [job_env_vars/template.yaml](https://github.com/aws-deadline/deadl - 024/07/16 16:18:27-07:00 - 2024/07/16 16:18:27-07:00 ============================================== - 2024/07/16 16:18:27-07:00 --------- Entering Environment: JobEnv - 2024/07/16 16:18:27-07:00 ============================================== - 2024/07/16 16:18:27-07:00 Setting: JOB_VERBOSITY=MEDIUM - 2024/07/16 16:18:27-07:00 Setting: JOB_EXAMPLE_PARAM=An example parameter value - 2024/07/16 16:18:27-07:00 Setting: JOB_PROJECT_ID=project-12 - 2024/07/16 16:18:27-07:00 Setting: JOB_ENDPOINT_URL=https://internal-host-name/some/path - 2024/07/16 16:18:27-07:00 Setting: QT_QPA_PLATFORM=offscreen - -The following lines from the job template specified this action. +The following lines from the job template define this environment. @@ -70 +164,0 @@ The following lines from the job template specified this action. - description: Job environments apply to everything in the job. @@ -72 +165,0 @@ The following lines from the job template specified this action. - # When applications have options as environment variables, you can set them here. @@ -74 +166,0 @@ The following lines from the job template specified this action. - # You can use the value of job parameters when setting environment variables. @@ -76 +167,0 @@ The following lines from the job template specified this action. - # Some more ideas. @@ -79 +169,0 @@ The following lines from the job template specified this action. - # This variable lets applications using the Qt Framework run without a display @@ -82 +172 @@ The following lines from the job template specified this action. - 2. Select the **Launch StepEnv** session action in Deadline Cloud monitor. You will see the following log output. + 2. Select the **Launch StepEnv** session action. The log output shows the step-level variables, including the overridden `JOB_PROJECT_ID`. @@ -84,6 +174,2 @@ The following lines from the job template specified this action. - 2024/07/16 16:18:27-07:00 - 2024/07/16 16:18:27-07:00 ============================================== - 2024/07/16 16:18:27-07:00 --------- Entering Environment: StepEnv - 2024/07/16 16:18:27-07:00 ============================================== - 2024/07/16 16:18:27-07:00 Setting: STEP_VERBOSITY=HIGH - 2024/07/16 16:18:27-07:00 Setting: JOB_PROJECT_ID=step-project-12 + Setting: STEP_VERBOSITY=HIGH + Setting: JOB_PROJECT_ID=step-project-12 @@ -91 +177 @@ The following lines from the job template specified this action. -The following lines from the job template specified this action. +The following lines from the job template define this environment.