AWS omics documentation change
Summary
Added guidance on using scratch storage with ephemeral volumes and scratch directive recommendations
Security assessment
Documents performance optimizations for temporary storage without discussing security aspects like data isolation, encryption, or access controls. Focuses on efficiency rather than security.
Diff
diff --git a/omics/latest/dev/workflow-definition-nextflow.md b/omics/latest/dev/workflow-definition-nextflow.md index 6988dd166..4e753c7e2 100644 --- a//omics/latest/dev/workflow-definition-nextflow.md +++ b//omics/latest/dev/workflow-definition-nextflow.md @@ -7 +7 @@ -Use nf-schema and nf-validation pluginsSpecify storage URIsNextflow directivesUse Nextflow profilesExport workflow-level contentExport task contentSpecify the Nextflow syntax versionNextflow v26.04 release notes +Use nf-schema and nf-validation pluginsSpecify storage URIsNextflow directivesUse Nextflow profilesExport workflow-level contentExport task contentSpecify the Nextflow syntax versionUsing scratch storage efficiently in NextflowNextflow v26.04 release notes @@ -30,0 +31,2 @@ Nextflow DSL2 is based on the Groovy programming language, so parameters are dyn + * Using scratch storage efficiently in Nextflow + @@ -440,0 +443,26 @@ For Nextflow v25.10.0 and earlier, HealthOmics does not support the v2 parser. +## Using scratch storage efficiently in Nextflow + +Nextflow's `scratch` directive controls where a process writes its temporary working files. When ephemeral storage is enabled (`scratchStorageMode: LOCAL`), use the `scratch` directive to direct scratch I/O to the fast local volume at `/tmp`. + +The following table describes the supported `scratch` directive values and their behavior in HealthOmics: + +Value | Behavior in HealthOmics | Recommendation +---|---|--- +`scratch true` | Uses `$TMPDIR`. Scratch I/O is directed to the local ephemeral volume when `scratchStorageMode` is `LOCAL`. | Recommended +`scratch '/some/path'` | Uses the specified literal path as the scratch directory. To use ephemeral storage, set the path to `/tmp` or a subdirectory of `/tmp`. The path must exist in the container and be writable. | Works when path is under `/tmp` +`scratch 'ram-disk'` | Attempts to use `/dev/shm` (tmpfs in RAM). This is not recommended for local scratch storage in HealthOmics. | Not recommended + +The recommended approach is to set `scratch true` in your process definition, which automatically uses `$TMPDIR` and requires no path configuration: + + + process my_process { + scratch true + disk '200 GB' + script: + """ + my-tool --input ${input} --output ${output} + """ + } + +For more information about ephemeral storage and the `disk` directive, see [Ephemeral storage for HealthOmics workflow tasks](./workflows-ephemeral-storage.html). +