AWS batch documentation change
Summary
Restructured tutorial with numbered steps, added overview sections, time/cost estimates, detailed EKS cluster creation instructions, and enhanced IAM/RBAC configuration guidance
Security assessment
Added documentation about required IAM permissions (eks:DescribeCluster) and RBAC configuration for Kubernetes roles. While these are security-related configurations, there's no evidence of addressing a specific vulnerability. The changes improve security documentation by clarifying access control requirements for AWS Batch on EKS.
Diff
diff --git a/batch/latest/userguide/getting-started-eks.md b/batch/latest/userguide/getting-started-eks.md index 09e41a844..34dcdf2ae 100644 --- a//batch/latest/userguide/getting-started-eks.md +++ b//batch/latest/userguide/getting-started-eks.md @@ -5 +5 @@ -PrerequisitesPrepare your Amazon EKS cluster for AWS BatchCreate an Amazon EKS compute environmentCreate a job queue and attach the compute environmentCreate a job definitionSubmit a job(Optional) Submit a job with overrides +OverviewPrerequisitesStep 1: Create your Amazon EKS cluster for AWS BatchStep 2: Prepare your Amazon EKS cluster for AWS BatchStep 3: Create an Amazon EKS compute environmentStep 4: Create a job queue and attach the compute environmentStep 5: Create a job definitionStep 6: Submit a jobStep 7: View the Job's outputStep 8: (Optional) Submit a job with overridesStep 9: Clean up your tutorial resourcesAdditional resources @@ -7 +7 @@ PrerequisitesPrepare your Amazon EKS cluster for AWS BatchCreate an Amazon EKS c -# Tutorial: Getting started with AWS Batch on Amazon EKS +# Getting started with AWS Batch on Amazon EKS @@ -20,0 +21,2 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. + * [Overview](./getting-started-eks.html#getting-started-eks-context) + @@ -23 +25,31 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [Prepare your Amazon EKS cluster for AWS Batch](./getting-started-eks.html#getting-started-eks-step-1) + * [Step 1: Create your Amazon EKS cluster for AWS Batch](./getting-started-eks.html#getting-started-eks-step-0) + + * [Step 2: Prepare your Amazon EKS cluster for AWS Batch](./getting-started-eks.html#getting-started-eks-step-1) + + * [Step 3: Create an Amazon EKS compute environment](./getting-started-eks.html#getting-started-eks-step-2) + + * [Step 4: Create a job queue and attach the compute environment](./getting-started-eks.html#getting-started-eks-step-3) + + * [Step 5: Create a job definition](./getting-started-eks.html#getting-started-eks-step-4) + + * [Step 6: Submit a job](./getting-started-eks.html#getting-started-eks-step-5) + + * [Step 7: View the Job's output](./getting-started-eks.html#getting-started-eks-step-7) + + * [Step 8: (Optional) Submit a job with overrides](./getting-started-eks.html#getting-started-eks-step-6) + + * [Step 9: Clean up your tutorial resources](./getting-started-eks.html#getting-started-eks-step-8) + + * [Additional resources](./getting-started-eks.html#getting-started-eks-additional-resources) + + + + +## Overview + +This tutorial demonstrates how to setup AWS Batch with Amazon EKS using the AWS CLI, `kubectl` and `eksctl`. + +**Intended Audience** + + +This tutorial is designed for system administrators and developers responsible for setting up, testing, and deploying AWS Batch. @@ -25 +57 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [Create an Amazon EKS compute environment](./getting-started-eks.html#getting-started-eks-step-2) +**Features Used** @@ -27 +58,0 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [Create a job queue and attach the compute environment](./getting-started-eks.html#getting-started-eks-step-3) @@ -29 +60 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [Create a job definition](./getting-started-eks.html#getting-started-eks-step-4) +This tutorial shows you how to use the AWS CLI, to: @@ -31 +62 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [Submit a job](./getting-started-eks.html#getting-started-eks-step-5) + * Create and configure an Amazon EKS compute environment @@ -33 +64 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. - * [(Optional) Submit a job with overrides](./getting-started-eks.html#getting-started-eks-step-6) + * Create a job queue. @@ -34,0 +66 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. + * Create a job definition @@ -35,0 +68 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. + * Create and submit a job to run @@ -36,0 +70,23 @@ The following sections cover how to get set up for AWS Batch on Amazon EKS. + * Submit a job with overrides + + + + +**Time Required** + + +It should take about 30–40 minutes to complete this tutorial. + +**Regional Restrictions** + + +There are no country or regional restrictions associated with using this solution. + +**Resource Usage Costs** + + +There's no charge for creating an AWS account. However, by implementing this solution, you might incur some or all of the costs that are listed in the following table. + +Description | Cost (US dollars) +---|--- +You are charged by the cluster hour | Varies depending on Instance, see [Amazon EKS pricing](https://aws.amazon.com/eks/pricing/) @@ -50 +106 @@ Before starting this tutorial, you must install and configure the following tool - * Creating an Amazon EKS cluster – For more information, see [Getting started with Amazon EKS – `eksctl`](https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html) in the __Amazon EKS User Guide__. + * Permissions – Users calling the [CreateComputeEnvironment](https://docs.aws.amazon.com/batch/latest/APIReference/API_CreateComputeEnvironment.html) API operation to create a compute environment that uses Amazon EKS resources require permissions to the `eks:DescribeCluster` API operation. @@ -52 +108,3 @@ Before starting this tutorial, you must install and configure the following tool -###### Note + * AWS account number – You need to know your AWS account ID. Follow the directions in [Viewing your AWS account ID](https://docs.aws.amazon.com/IAM/latest/UserGuide/console-account-id.html). + + * (Optional) CloudWatch – To examine the details of (Optional) Submit a job with overrides, logging must be configured. For more information, see [Use CloudWatch Logs to monitor AWS Batch on Amazon EKS jobs](./batch-eks-cloudwatch-logs.html). @@ -54 +111,0 @@ Before starting this tutorial, you must install and configure the following tool -AWS Batch only supports Amazon EKS clusters with API server endpoints that have public access, accessible to the public internet. By default, Amazon EKS clusters API server endpoints have public access. For more information, see [Amazon EKS cluster endpoint access control](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html) in the __Amazon EKS User Guide__. @@ -56 +112,0 @@ AWS Batch only supports Amazon EKS clusters with API server endpoints that have -###### Note @@ -58 +113,0 @@ AWS Batch only supports Amazon EKS clusters with API server endpoints that have -AWS Batch doesn't provide managed node orchestration for CoreDNS or other deployment pods. If you need CoreDNS, see [Adding the CoreDNS Amazon EKS add-on](https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html#adding-coredns-eks-add-on) in the __Amazon EKS User Guide__. Or, use `eksctl create cluster create` to create the cluster, it includes CoreDNS by default. @@ -60 +115,3 @@ AWS Batch doesn't provide managed node orchestration for CoreDNS or other deploy - * Permissions – Users calling the [CreateComputeEnvironment](https://docs.aws.amazon.com/batch/latest/APIReference/API_CreateComputeEnvironment.html) API operation to create a compute environment that uses Amazon EKS resources require permissions to the `eks:DescribeCluster` API operation. Using the AWS Management Console to create a compute resource using Amazon EKS resources requires permissions to both `eks:DescribeCluster` and `eks:ListClusters`. +## Step 1: Create your Amazon EKS cluster for AWS Batch + +###### Important @@ -61,0 +119 @@ AWS Batch doesn't provide managed node orchestration for CoreDNS or other deploy +To get started as simply and quickly as possible, this tutorial includes steps with default settings. Before creating for production use, we recommend that you familiarize yourself with all settings and deploy with the settings that meet your requirements. @@ -62,0 +121 @@ AWS Batch doesn't provide managed node orchestration for CoreDNS or other deploy +Once you have installed the prerequisites you need to create your cluster using `eksctl`. Creating the cluster can take between 10-15 minutes. @@ -65 +124,14 @@ AWS Batch doesn't provide managed node orchestration for CoreDNS or other deploy -## Prepare your Amazon EKS cluster for AWS Batch + $ eksctl create cluster --name my-cluster-name --region region-code + +In the preceding command replace: + + * Replace `my-cluster-name` with the name you want to use for your cluster. + + * Replace `region-code` with the AWS Region to create the cluster in, for example `us-west-2`. + + + + +The cluster name and region are needed for later in this tutorial. + +## Step 2: Prepare your Amazon EKS cluster for AWS Batch @@ -96,4 +168,0 @@ Use `kubectl` to create a Kubernetes role for the cluster to allow AWS Batch to -###### Note - -For more information about using RBAC authorization, see [Using RBAC Authorization](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) in the _Kubernetes User Guide_. - @@ -148 +217 @@ Output: -Create namespace-scoped Kubernetes role for AWS Batch to manage and lifecycle pods and bind it. You must do this once for each unique namespace. + 3. Create namespace-scoped Kubernetes role for AWS Batch to manage and lifecycle pods and bind it. You must do this once for each unique namespace. @@ -189 +259,5 @@ Output: -Update Kubernetes `aws-auth` configuration map to map the preceding RBAC permissions to the AWS Batch service-linked role. + 4. Update Kubernetes `aws-auth` configuration map to map the preceding RBAC permissions to the AWS Batch service-linked role. + +In the following command replace: + + * Replace `<your-account-number>` with your AWS account number. @@ -193 +267 @@ Update Kubernetes `aws-auth` configuration map to map the preceding RBAC permis - --arn "arn:aws:iam::<your-account>:role/AWSServiceRoleForBatch" \ + --arn "arn:aws:iam::<your-account-number>:role/AWSServiceRoleForBatch" \ @@ -199 +273 @@ Output: - 2022-10-25 20:19:57 [ℹ] adding identity "arn:aws:iam::<your-account>:role/AWSServiceRoleForBatch" to auth ConfigMap + 2022-10-25 20:19:57 [ℹ] adding identity "arn:aws:iam::<your-account-number>:role/AWSServiceRoleForBatch" to auth ConfigMap @@ -208 +282 @@ The path `aws-service-role/batch.amazonaws.com/` has been removed from the ARN o -## Create an Amazon EKS compute environment +## Step 3: Create an Amazon EKS compute environment @@ -213,0 +288,27 @@ Now that the **AWSServiceRoleForBatch** service-linked role has access to your A + * For `subnets` run `eksctl get cluster `my-cluster-name`` to get the subnets used by the cluster. + + * For `securityGroupIds` parameter you can use the same security group as the Amazon EKS cluster. This command retrieves the security group ID for the cluster. + + $ aws eks describe-cluster \ + --name my-cluster-name \ + --query cluster.resourcesVpcConfig.clusterSecurityGroupId + + * The `instanceRole` is created when you create the cluster. To find the `instanceRole` list all entities that use the `AmazonEKSWorkerNodePolicy` policy: + + $ aws iam list-entities-for-policy --policy-arn arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy + +The name of the policy role contains the name of the cluster that you created `eksctl-`my-cluster-name`-nodegroup-example`. + +To find the `instanceRole` arn run the following command: + + $ aws iam list-instance-profiles-for-role --role-name eksctl-my-cluster-name-nodegroup-example + +Output: + + INSTANCEPROFILES arn:aws:iam::<your-account-number>:instance-profile/eks-04cb2200-94b9-c297-8dbe-87f12example + +For more information, see [Creating the Amazon EKS node IAM role](https://docs.aws.amazon.com/eks/latest/userguide/create-node-role.html#create-worker-node-role) and [Enabling IAM principal access to your cluster](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) in the _Amazon EKS User Guide_. If you're using pod networking, see [Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html) in the __Amazon EKS User Guide__. + + + + @@ -221 +322 @@ Now that the **AWSServiceRoleForBatch** service-linked role has access to your A - "eksClusterArn": "arn:aws:eks:<region>:123456789012:cluster/<cluster-name>", + "eksClusterArn": "arn:aws:eks:region-code:your-account-number:cluster/my-cluster-name", @@ -242,5 +342,0 @@ Now that the **AWSServiceRoleForBatch** service-linked role has access to your A - $ aws batch create-compute-environment --cli-input-json file://./batch-eks-compute-environment.json - -###### Notes - - * The `serviceRole` parameter should not be specified, then the AWS Batch service-linked role will be used. AWS Batch on Amazon EKS only supports the AWS Batch service-linked role. @@ -248 +343,0 @@ Now that the **AWSServiceRoleForBatch** service-linked role has access to your A - * Only `BEST_FIT_PROGRESSIVE`, `SPOT_CAPACITY_OPTIMIZED`, and `SPOT_PRICE_CAPACITY_OPTIMIZED` allocation strategies are supported for Amazon EKS compute environments. @@ -250,5 +344,0 @@ Now that the **AWSServiceRoleForBatch** service-linked role has access to your A -###### Note - -We recommend that you use `SPOT_PRICE_CAPACITY_OPTIMIZED` rather than `SPOT_CAPACITY_OPTIMIZED` in most instances. - - * For the `instanceRole`, see [Creating the Amazon EKS node IAM role](https://docs.aws.amazon.com/eks/latest/userguide/create-node-role.html#create-worker-node-role) and [Enabling IAM principal access to your cluster](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) in the _Amazon EKS User Guide_. If you're using pod networking, see [Configuring the Amazon VPC CNI plugin for Kubernetes to use IAM roles for service accounts](https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html) in the __Amazon EKS User Guide__. @@ -256,3 +346 @@ We recommend that you use `SPOT_PRICE_CAPACITY_OPTIMIZED` rather than `SPOT_CAPA - * A way to get working subnets for the `subnets` parameter is to use the Amazon EKS managed node groups public subnets that were created by `eksctl` when creating an Amazon EKS cluster. Otherwise, use subnets that have a network path that supports pulling images. - - * The `securityGroupIds` parameter can use the same security group as the Amazon EKS cluster. This command retrieves the security group ID for the cluster.