AWS deadline-cloud documentation change
Summary
Added comprehensive documentation about conda packages including structure, usage, and detailed guidance for packaging applications, plugins, and adaptor packages. Reorganized content with new sections and updated links to GitHub samples.
Security assessment
The changes are primarily instructional and organizational, focusing on how to create and use conda packages with Deadline Cloud. The only security-related content is a pre-existing recommendation to use a private Amazon S3 bucket for conda channels to control distribution and limit package access, which was already present in the original document and was not added by this change. There is no evidence of addressing a specific security vulnerability, weakness, or incident.
Diff
diff --git a/deadline-cloud/latest/developerguide/conda-package.md b/deadline-cloud/latest/developerguide/conda-package.md index 4d803eacb..9e81a0b35 100644 --- a//deadline-cloud/latest/developerguide/conda-package.md +++ b//deadline-cloud/latest/developerguide/conda-package.md @@ -4,0 +5,2 @@ +Package an applicationPackage a pluginAdaptor packages + @@ -6,0 +9,8 @@ +A conda package is a compressed archive of software written in any language. Conda supports a variety of operating system and architecture combinations, so you can package full applications like Blender, Maya, and Nuke alongside libraries for Python and other languages. For more information about conda packages, see [Packages](https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/packages.html) in the conda documentation. + +To use a conda package, you install it into a virtual environment. A conda virtual environment has a _prefix directory_ where packages are installed. Installing a package uses hardlinking or reflinking of files when supported, so creating multiple environments with the same packages does not use significant additional disk space. To use a virtual environment, you activate it to set environment variables. Activation runs scripts that packages provide, giving each package the opportunity to modify PATH or other environment variables. Conda packages typically contain applications or libraries, but the flexible activation means they can also point to applications installed on a shared filesystem. + +Making a custom package involves three stages: a _recipe_ contains the build instructions, a _package_ is the built artifact (`.conda` or `.tar.bz2` file), and a _channel_ hosts packages for installation. The `rattler-build publish` command handles all three steps—it can build a recipe into a package and publish to a channel, or it can take a package artifact directly to publish it. + +The [conda-forge](https://conda-forge.org/) community maintains package recipes for a broad set of open source software, and hosts package artifacts in the `conda-forge` channel. You can configure your queue to include `conda-forge` as a package source, and then build custom packages that depend on conda-forge packages to run. For Linux, conda-forge hosts a full compiler toolchain including CUDA support, with consistent compiling and linking options selected. You can use conda-forge packages as dependencies in your own recipes, or install them alongside your custom packages in the same environment. + @@ -8,0 +19,6 @@ You can combine an entire application, including dependencies, into a conda pack +###### Note + +Large applications can require tens of GB of free disk space for the source archive, extracted files, and build output. Make sure that you use a disk with enough available space for the package build output. + +## Package an application + @@ -20 +36 @@ When repackaging an application for conda, there are two goals: - 1. To repackage an application for conda, write conda build recipes that install the application into a subdirectory like `$CONDA_PREFIX/opt/`<application-name>``. This separates it from the standard prefix directories like `bin` and `lib`. + 1. Write conda build recipes that install the application into a subdirectory like `$CONDA_PREFIX/opt/`<application-name>``. This separates it from the standard prefix directories like `bin` and `lib`. @@ -22 +38 @@ When repackaging an application for conda, there are two goals: - 2. Then, add symlinks or launch scripts to `$CONDA_PREFIX/bin` to run the application binaries. + 2. Add symlinks or launch scripts to `$CONDA_PREFIX/bin` to run the application binaries. @@ -28 +44,3 @@ Alternatively, create activate.d scripts that the `conda activate` command will - 4. Ensure you follow the copyright and license agreements for the applications you package. We recommend using a private Amazon S3 bucket for your conda channel to control distribution and limit package access to your farm. + 4. If the application supports plugins, provide a clear convention that plugin packages should follow to integrate with the application in a virtual environment. For example, the [Maya 2026 sample recipe](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes/maya-2026#instructions-for-maya-plugin-packages) documents this convention for Maya plugins. + + 5. Ensure you follow the copyright and license agreements for the applications you package. We recommend using a private Amazon S3 bucket for your conda channel to control distribution and limit package access to your farm. @@ -33 +51 @@ Alternatively, create activate.d scripts that the `conda activate` command will -Sample recipes for all the packages in the deadline-cloud channel are available in the [Deadline Cloud Samples GitHub repository](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes). +Sample recipes for the packages in the `deadline-cloud` channel are available in the [Deadline Cloud samples](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes#readme) repository on GitHub. @@ -35 +53 @@ Sample recipes for all the packages in the deadline-cloud channel are available -###### To package a plugin for conda +## Package a plugin @@ -37 +55 @@ Sample recipes for all the packages in the deadline-cloud channel are available - * Application plugins can be packaged as their own conda packages. When creating a plugin package: +Application plugins can be packaged as their own conda packages. When creating a plugin package, follow these guidelines: @@ -39 +57 @@ Sample recipes for all the packages in the deadline-cloud channel are available - * Include the host application package as both a build and a run dependency in the build recipe `meta.yaml` and `recipe.yaml`. Use a version constraint so that the build recipe is only installed with compatible packages. + * Include the host application package as both a build and a run dependency in the build recipe `recipe.yaml`. Use a version constraint so that the build recipe is only installed with compatible packages. @@ -45,0 +64,8 @@ Sample recipes for all the packages in the deadline-cloud channel are available +## Adaptor packages + +Some Deadline Cloud application integrations use an _adaptor_ that extends the application interface to simplify [writing job templates](./building-jobs.html). An adaptor is a command-line interface with support for running a background daemon, reporting status, and applying path mapping. For more information, see the [Open Job Description Adaptor Runtime](https://github.com/OpenJobDescription/openjd-adaptor-runtime-for-python#readme) on GitHub. For example, [deadline-cloud-for-maya](https://github.com/aws-deadline/deadline-cloud-for-maya/) on GitHub includes an integrated job submission GUI and a Maya adaptor that is available as the `maya-openjd` package on service-managed fleets. + +Job submissions from Deadline Cloud submitter GUIs include a `CondaPackages` parameter value that specifies the conda packages to include in a virtual environment for running the job. The `CondaPackages` parameter value for Maya typically looks like `maya=2025.* maya-openjd=0.15.* maya-mtoa` and might contain alternative entries for plugin packages. When the queue environment sets up a conda virtual environment for running the job, it resolves these package names and version constraints to be compatible and adds all the dependency packages they need to run. Each adaptor and plugin package specifies what it is compatible with, including which versions of Maya, which versions of Python, and other dependencies. + +To build your own adaptor packages using our samples such as the [maya-openjd recipe](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes/maya-openjd) on GitHub, you can build on the packages for Python and other dependencies provided by [conda-forge](https://conda-forge.org/). You might need to build the [deadline](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes/deadline) and [openjd-adaptor-runtime](https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/conda_recipes/openjd-adaptor-runtime) recipes first to satisfy dependencies. +