AWS eks documentation change
Summary
Expanded cost optimization guidance with new sections on node tagging, multi-tenant clusters, scheduled autoscaling, compute capacity types (Spot, Savings Plans, On-Demand, Fargate), and compute optimization techniques (Graviton, GPUs). Added YAML examples and implementation details.
Security assessment
Changes focus exclusively on cost optimization strategies without addressing security vulnerabilities, security controls, or security features. Content covers resource tagging, capacity planning, instance selection, and workload scheduling without any security-related context or security-specific recommendations.
Diff
diff --git a/eks/latest/best-practices/cost-opt-compute.md b/eks/latest/best-practices/cost-opt-compute.md index 21ca44300..fcf308b1c 100644 --- a//eks/latest/best-practices/cost-opt-compute.md +++ b//eks/latest/best-practices/cost-opt-compute.md @@ -5 +5 @@ -Right-size your workloadsReduce consumptionReduce unused capacityRemove under-utilized nodes by adjusting Cluster Autoscaler parameters +Right-size your workloadsReduce consumptionReduce unused capacityOptimize compute capacity typesOptimize Compute Usage @@ -122 +122,10 @@ For workloads that might not be interruptible e.g. long running batch jobs witho -8"" linenumbering="unnumbered">apiVersion: v1 kind: Pod metadata: name: label-demo labels: environment: production annotations: + "karpenter.sh/do-not-evict": "true" spec: containers: * name: nginx image: nginx ports: ** containerPort: 80 + apiVersion: v1 + kind: Pod + metadata: + name: label-demo + labels: + environment: production + annotations: + + "karpenter.sh/do-not-evict": "true" + spec: + containers: @@ -124 +133,6 @@ For workloads that might not be interruptible e.g. long running batch jobs witho -## Remove under-utilized nodes by adjusting Cluster Autoscaler parameters + * name: nginx + image: nginx + ports: + ** containerPort: 80 + +### Remove under-utilized nodes by adjusting Cluster Autoscaler parameters @@ -131 +145,164 @@ You can prevent scale down from happening by ensuring that pods that are expensi -8"" linenumbering="unnumbered">apiVersion: v1 kind: Pod metadata: name: label-demo labels: environment: production annotations: + "cluster-autoscaler.kubernetes.io/safe-to-evict": "false" spec: containers: * name: nginx image: nginx ports: ** containerPort: 80 + apiVersion: v1 + kind: Pod + metadata: + name: label-demo + labels: + environment: production + annotations: + + "cluster-autoscaler.kubernetes.io/safe-to-evict": "false" + spec: + containers: + + * name: nginx + image: nginx + ports: + ** containerPort: 80 + +### Tag nodes with Cluster Autoscaler and Karpenter + +AWS resource [tags](https://docs.aws.amazon.com/tag-editor/latest/userguide/tagging.html) are used to organize your resources, and to track your AWS costs on a detailed level. They do not directly correlate with Kubernetes labels for cost tracking. It’s recommended to start with Kubernetes resource labeling and utilize tools like [Kubecost](https://aws.amazon.com/blogs/containers/aws-and-kubecost-collaborate-to-deliver-cost-monitoring-for-eks-customers/) to get infrastructure cost reporting based on Kubernetes labels on pods, namespaces etc. + +Worker nodes need to have tags to show billing information in AWS Cost Explorer. With Cluster Autoscaler, tag your worker nodes inside a managed node group using [launch template](https://docs.aws.amazon.com/eks/latest/userguide/launch-templates.html). For self managed node groups, tag your instances using [EC2 auto scaling group](https://docs.aws.amazon.com/autoscaling/ec2/userguide/ec2-auto-scaling-tagging.html). For instances provisioned by Karpenter, tag them using [spec.tags in the node template](https://karpenter.sh/docs/concepts/nodeclasses/#spectags). + +### Multi-tenant clusters + +When working on clusters that are shared by different teams you may not have visibility to other workloads running on the same node. While resource requests can help isolate some "noisy neighbor" concerns, such as CPU sharing, they may not isolate all resource boundaries such as disk I/O exhaustion. Not every consumable resource by a workload can be isolated or limited. Workloads that consume shared resources at higher rates than other workloads should be isolated through node [taints and tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/). Another advanced technique for such workload is [CPU pinning](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#static-policy) which ensures exclusive CPU instead of shared CPU for the container. + +Isolating workloads at a node level can be more expensive, but it may be possible to schedule [BestEffort](https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#besteffort) jobs or take advantage of additional savings by using [Reserved Instances](https://aws.amazon.com/ec2/pricing/reserved-instances/), [Graviton processors](https://aws.amazon.com/ec2/graviton/), or [Spot](https://aws.amazon.com/ec2/spot/). + +Shared clusters may also have cluster level resource constraints such as IP exhaustion, Kubernetes service limits, or API scaling requests. You should review the [scalability best practices guide](https://docs.aws.amazon.com/eks/latest/best-practices/scale-control-plane.html) to make sure your clusters avoid these limits. + +You can isolate resources at a namespace or Karpenter provisioner level. [Resource Quotas](https://kubernetes.io/docs/concepts/policy/resource-quotas/) provide a way to set limits on how many resources workloads in a namespace can consume. This can be a good initial guard rail but it should be continually evaluated to make sure it doesn’t artificially restrict workloads from scaling. + +Karpenter provisioners can [set limits on some of the consumable resources](https://karpenter.sh/docs/concepts/nodepools/#speclimitsresources) in a cluster (e.g. CPU, GPU), but you will need to configure tenant applications to use the appropriate provisioner. This can prevent a single provisioner from creating too many nodes in a cluster, but it should be continually evaluated to make sure the limit isn’t set too low and in turn, prevent workloads from scaling. + +### Scheduled Autoscaling + +You may have the need to scale down your clusters on weekends and off hours. This is particularly relevant for test and non-production clusters where you want to scale down to zero when they are not in use. Solutions like [cluster-turndown](https://github.com/kubecost/cluster-turndown) can scale down the replicas to zero based on a cron schedule. You can also acheive this with Karpenter, outlined in the following [AWS blog](https://aws.amazon.com/blogs/containers/manage-scale-to-zero-scenarios-with-karpenter-and-serverless/). + +## Optimize compute capacity types + +After optimizing the total capacity of compute in your cluster and utilizing bin packing, you should look at what type of compute you have provisioned in your clusters and how you pay for those resources. AWS has [Compute Savings plans](https://aws.amazon.com/savingsplans/compute-pricing/) that can reduce the cost for your compute which we will categorize into the following capacity types: + + * Spot + + * Savings Plans + + * On-Demand + + * Fargate + + + + +Each capacity type has different trade-offs for management overhead, availability, and long term commitments and you will need to decide which is right for your environment. No environment should rely on a single capacity type and you can mix multiple run types in a single cluster to optimize specific workload requirements and cost. + +### Spot + +The [spot](https://aws.amazon.com/ec2/spot/) capacity type provisions EC2 instances from spare capacity in an Availability Zone. Spot offers the largest discounts—up to 90% — but those instances can be interrupted when they are needed elsewhere. Additionally, there may not always be capacity to provision new Spot instances and existing Spot instances can be reclaimed with a [2 minute interruption notice](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html). If your application has a long startup or shutdown process, Spot instances may not be the best option. + +Spot compute should use a wide variety of instance types to reduce the likelihood of not having spot capacity available. Instance interruptions need to be handled to safely shutdown nodes. Nodes provisioned with Karpenter or part of a Managed Node Group automatically support [instance interruption notifications](https://docs.aws.amazon.com/eks/latest/best-practices/karpenter.html). If you are using self-managed nodes you will need to run the [node termination handler](https://github.com/aws/aws-node-termination-handler) separately to gracefully shutdown spot instances. + +It is possible to balance spot and on-demand instances in a single cluster. With Karpenter you can create [weighted provisioners](https://karpenter.sh/docs/concepts/scheduling/#on-demandspot-ratio-split) to achieve a balance of different capacity types. With Cluster Autoscaler you can create [mixed node groups with spot and on-demand or reserved instances](https://aws.amazon.com/blogs/containers/amazon-eks-now-supports-provisioning-and-managing-ec2-spot-instances-in-managed-node-groups/). + +Here is an example of using Karpenter to prioritize Spot instances ahead of On-Demand instances. When creating a provisioner, you can specify either Spot, On-Demand, or both (as shown below). When you specify both, and if the pod does not explicitly specify whether it needs to use Spot or On-Demand, then Karpenter prioritizes Spot when provisioning a node with [price-capacity-optimization allocation strategy](https://aws.amazon.com/blogs/compute/introducing-price-capacity-optimized-allocation-strategy-for-ec2-spot-instances/) . + + + apiVersion: karpenter.sh/v1 + kind: Provisioner + metadata: + name: spot-prioritized + spec: + requirements: + - key: "karpenter.sh/capacity-type" + operator: In + values: ["spot", "on-demand"] + +### Savings Plans, Reserved Instances, and AWS EDP + +You can reduce your compute spend by using a [compute savings plan](https://aws.amazon.com/savingsplans/compute-pricing/). Savings plans offer reduced prices for a 1 or 3 year commitment of compute usage. The usage can apply to EC2 instances in an EKS cluster but also applies to any compute usage such as Lambda and Fargate. With savings plans you can reduce costs and still pick any EC2 instance type during your commitment period. + +Compute savings plan can reduce your EC2 cost by up to 66% without requiring commitments on what instance types, families, or regions you want to use. Savings are automatically applied to instances as you use them. + +EC2 Instance Savings Plans provides up to 72% savings on compute with a commitment of usage in a specific region and EC2 family, e.g. instances from the C family. You can shift usage to any AZ within the region, use any generation of the instance family, e.g. c5 or c6, and use any size of instance within the family. The discount will automatically be applied for any instance in your account that matches the savings plan criteria. + +[Reserved Instances](https://aws.amazon.com/ec2/pricing/reserved-instances/) are similar to EC2 Instance Savings Plan but they also guarantee capacity in an Availability Zone or Region and reduce cost—up to 72% — over on-demand instances. Once you calculate how much reserved capacity you will need you can select how long you would like to reserve them for (1 year or 3 years). The discounts will automatically be applied as you run those EC2 instances in your account. + +Customers also have the option to enroll in an Enterprise Agreement with AWS. Enterprise Agreements give customers the option to tailor agreements that best suit their needs. Customers can enjoy discounts on the pricing based on AWS EDP (Enterprise Discount Program). For additional information on Enterprise Agreements please contact your AWS sales representative. + +### On-Demand + +On-Demand EC2 instances have the benefit of availability without interruptions — compared to spot — and no long term commitments — compared to savings plans. If you are looking to reduce costs in a cluster you should reduce your usage of on-demand EC2 instances. + +After optimizing your workload requirements you should be able to calculate a minimum and maximum capacity for your clusters. This number may change over time but rarely goes down. Consider using a Savings Plan for everything under the minimum, and spot for capacity that will not affect your application’s availability. Anything else that may not be continuously used or is required for availability can use on-demand. + +As mentioned in this section, the best way to reduce your usage is to consume fewer resources and utilize the resources you provision to the fullest extent possible. With the Cluster Autoscaler you can remove underutilized nodes with the `scale-down-utilization-threshold` setting. With Karpenter it is recommended to enable consolidation. + +To manually identify EC2 instance types that can be used with your workloads you should use [ec2-instance-selector](https://github.com/aws/amazon-ec2-instance-selector) which can show instances that are available in each region as well as instances compatible with EKS. Example usage for instances with x86 process architecture, 4 Gb of memory, 2 vCPUs and available in the us-east-1 region. + + + ec2-instance-selector --memory 4 --vcpus 2 --cpu-architecture x86_64 \ + -r us-east-1 --service eks + c5.large + c5a.large + c5ad.large + c5d.large + c6a.large + c6i.large + t2.medium + t3.medium + t3a.medium + +For non-production environments you can automatically have clusters scaled down during unused hours such as night and weekends. The kubecost project [cluster-turndown](https://github.com/kubecost/cluster-turndown) is an example of a controller that can automatically scale your cluster down based on a set schedule. + +### Fargate compute + +Fargate compute is a fully managed compute option for EKS clusters. It provides pod isolation by scheduling one pod per node in a Kubernetes cluster. It allows you to size your compute nodes to the CPU and RAM requirements of your workload to tightly control workload usage in a cluster. + +Fargate can scale workloads as small as .25 vCPU with 0.5 GB memory and as large as 16 vCPU with 120 GB memory. There are limits on how many [pod size variations](https://docs.aws.amazon.com/eks/latest/userguide/fargate-pod-configuration.html) are available and you will need to understand how your workload best fits into a Fargate configuration. For example, if your workload requires 1 vCPU with 0.5 GB of memory the smallest Fargate pod will be 1 vCPU with 2 GB of memory. + +While Fargate has many benefits such as no EC2 instance or operating system management, it may require more compute capacity than traditional EC2 instances due to the fact that every deployed pod is isolated as a separate node in the cluster. This requires more duplication for things like the Kubelet, logging agents, and any DaemonSets you would typically deploy to a node. DaemonSets are not supported in Fargate and they will need to be converted into pod "`sidecars"` and run alongside the application. + +Fargate cannot benefit from bin packing or CPU over provisioning because the boundary for the workload is a node which is not burstable or shareable between workloads. Fargate will save you EC2 instance management time which itself has a cost, but CPU and memory costs may be more expensive than other EC2 capacity types. Fargate pods can take advantage of compute savings plan to reduce the on-demand cost. + +## Optimize Compute Usage + +Another way to save money on your compute infrastructure is to use more efficient compute for the workload. This can come from more performant general purpose compute like [Graviton processors](https://aws.amazon.com/ec2/graviton/) which are up to 20% cheaper and 60% more energy efficient than x86—or workload specific accelerators such as GPUs and [FPGAs](https://aws.amazon.com/ec2/instance-types/f1/). You will need to build containers that can [run on arm architecture](https://aws.amazon.com/blogs/containers/how-to-build-your-containers-for-arm-and-save-with-graviton-and-spot-instances-on-amazon-ecs/) and [set up nodes with the right accelerators](https://aws.amazon.com/blogs/compute/running-gpu-accelerated-kubernetes-workloads-on-p3-and-p2-ec2-instances-with-amazon-eks/) for your workloads. + +EKS has the ability to run clusters with mixed architecture (e.g. amd64 and arm64) and if your containers are compiled for multiple architectures you can take advantage of Graviton processors with Karpenter by allowing both architectures in your provisioner. To keep consistent performance, however, it is recommended you keep each workload on a single compute architecture and only use different architecture if there is no additional capacity available. + +Provisioners can be configured with multiple architectures and workloads can also request specific architectures in their workload specification. + + + apiVersion: karpenter.sh/v1 + kind: Provisioner + metadata: + name: default + spec: + requirements: + - key: "kubernetes.io/arch" + operator: In + values: ["arm64", "amd64"] + +With Cluster Autoscaler you will need to create a node group for Graviton instances and set [node tolerations on your workload](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to utilize the new capacity. + +GPUs and FPGAs can greatly increase the performance for your workload, but the workload will need to be optimized to use the accelerator. Many workload types for machine learning and artificial intelligence can use GPUs for compute and instances can be added to a cluster and mounted into a workload using resource requests. + + + spec: + template: + spec: + - containers: + ... + resources: + limits: + nvidia.com/gpu: "1" + +Some GPU hardware can be shared across multiple workloads so a single GPU can be provisioned and used. To see how to configure workload GPU sharing see the [virtual GPU device plugin](https://aws.amazon.com/blogs/opensource/virtual-gpu-device-plugin-for-inference-workload-in-kubernetes/) for more information. You can also refer to the following blogs: + + * [GPU sharing on Amazon EKS with NVIDIA time-slicing and accelerated EC2 instances](https://aws.amazon.com/blogs/containers/gpu-sharing-on-amazon-eks-with-nvidia-time-slicing-and-accelerated-ec2-instances/) + + * [Maximizing GPU utilization with NVIDIA’s Multi-Instance GPU (MIG) on Amazon EKS: Running more pods per GPU for enhanced performance](https://aws.amazon.com/blogs/containers/maximizing-gpu-utilization-with-nvidias-multi-instance-gpu-mig-on-amazon-eks-running-more-pods-per-gpu-for-enhanced-performance/) + + +