AWS Security ChangesHomeSearch

AWS pcs: AWS PCS: base64-decode Slurm cluster secret before writing key file

Service: pcs · 2026-10-23 · Documentation medium

File: pcs/latest/userguide/working-with_clusters_secrets_get.md · Type: credentials

Summary

Rewrites the Secrets Manager retrieval instructions to clarify that the cluster secret is returned base64-encoded, adds an Important note that Slurm requires the raw decoded bytes (with a `base64 -d` example writing /etc/slurm/slurm.key), and adds a Note that managed compute node groups receive the secret automatically during bootstrap.

Security assessment

The change documents correct handling of a Slurm JWT signing secret retrieved from Secrets Manager, including decoding it before writing the key file and noting that managed node groups get the secret via bootstrap. It is secret-handling/authentication guidance but does not reference any specific vulnerability, CVE, or incident, so it is not security_issue_related.

Evidence

+Secrets Manager stores and returns the cluster secret as a base64-encoded string, but Slurm requires the secret as the raw (decoded) bytes. If you write the secret to a Slurm key file (for example, `/etc/slurm/slurm.key`), you must base64-decode the value first. Writing the base64-encoded string directly produces a key file that Slurm cannot use. `slurmd` then fails authentication with an error such as `decode_jwt: jwt_decode (with key) failure: Invalid argument`.

Diff

diff --git a/pcs/latest/userguide/working-with_clusters_secrets_get.md b/pcs/latest/userguide/working-with_clusters_secrets_get.md
index 7523425..3deb089 100644
--- a//pcs/latest/userguide/working-with_clusters_secrets_get.md
+++ b//pcs/latest/userguide/working-with_clusters_secrets_get.md
@@ -9 +9 @@
-You can use Secrets Manager to get the current base64-encoded version of a Slurm cluster secret The following example uses the AWS CLI. Make the following substitutions before running the command. 
+You can use Secrets Manager to get the current version of a Slurm cluster secret. Secrets Manager returns the secret as a base64-encoded string. The following example uses the AWS CLI. Make the following substitutions before running the command. 
@@ -25,0 +26,18 @@ You can use Secrets Manager to get the current base64-encoded version of a Slurm
+###### Important
+
+Secrets Manager stores and returns the cluster secret as a base64-encoded string, but Slurm requires the secret as the raw (decoded) bytes. If you write the secret to a Slurm key file (for example, `/etc/slurm/slurm.key`), you must base64-decode the value first. Writing the base64-encoded string directly produces a key file that Slurm cannot use. `slurmd` then fails authentication with an error such as `decode_jwt: jwt_decode (with key) failure: Invalid argument`.
+
+To decode the secret and write it to a key file, pipe the output through `base64 -d`:
+    
+    
+    aws secretsmanager get-secret-value \
+        --region region \
+        --secret-id 'secret-arn' \
+        --version-stage AWSCURRENT \
+        --query 'SecretString' \
+        --output text | base64 -d | sudo tee /etc/slurm/slurm.key
+
+###### Note
+
+You don't need to retrieve the cluster secret yourself on a AWS PCS-managed compute node group. AWS PCS delivers the secret to instances launched by the compute node group as part of the node bootstrap. Retrieve the secret manually only when you configure a Slurm client outside of a managed compute node group, such as a persistent login node.
+