AWS wellarchitected documentation change
Summary
Enhanced infrastructure optimization guidance: Added HyperPod configuration details, LoRA adaptation techniques, updated section headers, and added multiple reference links.
Security assessment
Changes focus on resource utilization and cost optimization techniques. While LoRA adaptation improves efficiency, it's not presented as a security feature. No vulnerabilities or security controls are discussed in the added content.
Diff
diff --git a/wellarchitected/latest/generative-ai-lens/gencost02-bp02.md b/wellarchitected/latest/generative-ai-lens/gencost02-bp02.md index b4b51537c..191580951 100644 --- a//wellarchitected/latest/generative-ai-lens/gencost02-bp02.md +++ b//wellarchitected/latest/generative-ai-lens/gencost02-bp02.md @@ -27,0 +28,10 @@ Self-hosted model infrastructure should be optimized based on the model used and +In SageMaker AI HyperPod with both Amazon EKS and Slurm orchestration, use the system's advanced task governance capabilities and flexible training plans to dynamically allocate compute resources based on priority and demand, reducing costs through improved utilization. + +For EKS-based HyperPod, implement the managed Kubernetes orchestration with Hyperpod Task Governance. Configure automated scaling policies, priority classes, and node selectors to verify that your production workloads use cost-effective committed capacity while development tasks use On-Demand or Spot Instances when appropriate. Use the usage reporting feature to provide granular visibility into GPU, CPU, and Neuron Core consumption at both team and task levels, enabling transparent cost attribution and reducing guesswork in resource allocation. + +For Slurm-based HyperPod, use Slurm's native job scheduling and resource management features combined with HyperPod's auto-resume functionality to minimize wasted compute cycles during hardware failures, potentially reducing total training time in large clusters. Both systems benefit from implementing right-sizing strategies through SageMaker AI HyperPod Recipes that provide pre-configured, benchmarked training stacks optimized for specific model architectures like Llama and Mistral, providing optimized performance while minimizing resource waste. + +Additionally, establish flexible training plans that can set timeline and budget constraints, and allow HyperPod to automatically find the best combination of capacity blocks and create cost-optimized execution plans that avoid overspending by overprovisioning servers for training jobs. + +Inference workloads can be optimized using advanced techniques such as quantization or LoRA adaptation. These advanced capabilities are available for certain models in Amazon Bedrock or on self-hosted models on Amazon SageMaker AI. These advanced inference techniques can further optimize resource consumption for inference, thus reducing hosting and inference serving costs. + @@ -43 +53 @@ Self-hosted model infrastructure should be optimized based on the model used and -**Related practices:** +**Related best practices:** @@ -54 +64 @@ Self-hosted model infrastructure should be optimized based on the model used and -**Related guides, videos, and documentation:** +**Related videos and documents:** @@ -59,0 +70,2 @@ Self-hosted model infrastructure should be optimized based on the model used and + * [Get Started with Amazon SageMaker AI HyperPod Flexible Training Plans](https://www.youtube.com/watch?v=Itcw8zhdArY) + @@ -64,0 +77,2 @@ Self-hosted model infrastructure should be optimized based on the model used and + * [Easily deploy and manage hundreds of LoRA adapters with SageMaker AI efficient multi-adapter inference](https://aws.amazon.com/blogs/machine-learning/easily-deploy-and-manage-hundreds-of-lora-adapters-with-sagemaker-efficient-multi-adapter-inference/) + @@ -72,0 +87,10 @@ Self-hosted model infrastructure should be optimized based on the model used and + * [Maximize Accelerator Utilization for Model Development with New Amazon SageMaker AI HyperPod Task Governance](https://aws.amazon.com/blogs/aws/maximize-accelerator-utilization-for-model-development-with-new-amazon-sagemaker-hyperpod-task-governance/) + + * [Introducing Amazon SageMaker AI HyperPod to train foundation models at scale](https://aws.amazon.com/blogs/machine-learning/introducing-amazon-sagemaker-hyperpod-to-train-foundation-models-at-scale/) + + * [Best practices for Amazon SageMaker AI HyperPod task governance](https://aws.amazon.com/blogs/machine-learning/best-practices-for-amazon-sagemaker-hyperpod-task-governance/) + + * [Get started with Amazon SageMaker AI HyperPod task governance](https://www.youtube.com/watch?v=_wDhBAPwhoM) + + * [Usage reporting for cost attribution in SageMaker AI HyperPod](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-hyperpod-usage-reporting.html) +