AWS eks documentation change
Summary
Added new troubleshooting section for 'Node IP not in remote node network CIDR' error and 'kubectl logs/exec commands not working' issues. Updated command formatting, grammar, and clarified network configuration requirements.
Security assessment
The changes primarily improve troubleshooting guidance and clarify command usage. The new Node IP CIDR section addresses network configuration errors but does not indicate a specific security vulnerability. Security-related IAM Roles Anywhere/SSM content remains unchanged except for formatting improvements.
Diff
diff --git a/eks/latest/userguide/hybrid-nodes-troubleshooting.md b/eks/latest/userguide/hybrid-nodes-troubleshooting.md index e063aea2e..f99a68eea 100644 --- a//eks/latest/userguide/hybrid-nodes-troubleshooting.md +++ b//eks/latest/userguide/hybrid-nodes-troubleshooting.md @@ -13 +13 @@ To contribute to this user guide, choose the **Edit this page on GitHub** link t -This topic covers some common errors that you may see while using Amazon EKS Hybrid Nodes and how to fix them. For other troubleshooting information, see [Troubleshoot problems with Amazon EKS clusters and nodes](./troubleshooting.html) and [Knowledge Center tag for Amazon EKS](https://repost.aws/tags/knowledge-center/TA4IvCeWI1TE66q4jEj4Z9zg/amazon-elastic-kubernetes-service) on _AWS re:Post_. If you cannot resolve the issue, contact AWS Support. +This topic covers some common errors that you might see while using Amazon EKS Hybrid Nodes and how to fix them. For other troubleshooting information, see [Troubleshoot problems with Amazon EKS clusters and nodes](./troubleshooting.html) and [Knowledge Center tag for Amazon EKS](https://repost.aws/tags/knowledge-center/TA4IvCeWI1TE66q4jEj4Z9zg/amazon-elastic-kubernetes-service) on _AWS re:Post_. If you cannot resolve the issue, contact AWS Support. @@ -19 +19 @@ You can run the `nodeadm debug` command from your hybrid nodes to validate netwo -The troubleshooting topics in this section are related to installing the hybrid nodes dependencies on hosts with the `nodeadm install` command. +The following troubleshooting topics are related to installing the hybrid nodes dependencies on hosts with the `nodeadm install` command. @@ -21 +21 @@ The troubleshooting topics in this section are related to installing the hybrid -**nodeadm command failed`must run as root` ** +**`nodeadm` command failed `must run as root` ** @@ -23 +23 @@ The troubleshooting topics in this section are related to installing the hybrid -The `nodeadm install` command must be run with a user that has root/sudo privileges on your host. If you run `nodeadm install` with a user that does not have root/sudo privileges, you will see the following error in the nodeadm output. +The `nodeadm install` command must be run with a user that has root or `sudo` privileges on your host. If you run `nodeadm install` with a user that does not have root or `sudo` privileges, you will see the following error in the `nodeadm` output. @@ -30 +30 @@ The `nodeadm install` command must be run with a user that has root/sudo privile -The `nodeadm install` command installs the dependencies required for hybrid nodes. The hybrid nodes dependencies include `containerd`, `kubelet`, `kubectl`, and AWS SSM or AWS IAM Roles Anywhere components. You must have access from where you are running nodeadm install to download these dependencies. For more information on the list of locations that you must be able to access, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). If you do not have access, you will see errors similar to the following in the `nodeadm install` output. +The `nodeadm install` command installs the dependencies required for hybrid nodes. The hybrid nodes dependencies include `containerd`, `kubelet`, `kubectl`, and AWS SSM or AWS IAM Roles Anywhere components. You must have access from where you are running `nodeadm install` to download these dependencies. For more information on the list of locations that you must be able to access, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). If you do not have access, you will see errors similar to the following in the `nodeadm install` output. @@ -37 +37 @@ The `nodeadm install` command installs the dependencies required for hybrid node -The `nodeadm install` command runs apt update or yum/dnf update before installing the hybrid nodes dependencies. If this step does not succeed you may see errors similar to the following. To remediate, you can run apt update or yum/dnf update before running `nodeadm install` or you can attempt to re-run `nodeadm install`. +The `nodeadm install` command runs `apt update` or `yum update` or `dnf update` before installing the hybrid nodes dependencies. If this step does not succeed you might see errors similar to the following. To remediate, you can run `apt update` or `yum update` or `dnf update` before running `nodeadm install` or you can attempt to re-run `nodeadm install`. @@ -44 +44 @@ The `nodeadm install` command runs apt update or yum/dnf update before installin -When running `nodeadm install`, if you see issues at various stages of the install process with errors that indicate there was a timeout or context deadline exceeded, you may have a slow connection that is preventing the installation of the hybrid nodes dependencies before timeouts are met. To work around these issues, you can attempt to use the `--timeout` flag in `nodeadm` to extend the duration of the timeouts for downloading the dependencies. +When running `nodeadm install`, if you see issues at various stages of the install process with errors that indicate there was a timeout or context deadline exceeded, you might have a slow connection that is preventing the installation of the hybrid nodes dependencies before timeouts are met. To work around these issues, you can attempt to use the `--timeout` flag in `nodeadm` to extend the duration of the timeouts for downloading the dependencies. @@ -67,0 +68,40 @@ When running `nodeadm init`, `nodeadm` attempts to gather information about your +**Node IP not in remote node network CIDR** + +When running `nodeadm init`, you might encounter an error if the node’s IP address is not within the specified remote node network CIDRs. The error will look similar to the following example: + + + node IP 10.18.0.1 is not in any of the remote network CIDR blocks [10.0.0.0/16 192.168.0.0/16] + +This example shows a node with IP 10.18.0.1 attempting to join a cluster with remote network CIDRs 10.0.0.0/16 and 192.168.0.0/16. The error occurs because 10.18.0.1 isn’t within either of the ranges. + +Confirm that you’ve properly configured your `RemoteNodeNetworks` to include all node IP addresses. For more information on networking configuration, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). + + * Run the following command in the region your cluster is located to check your `RemoteNodeNetwork` configurations. Verify that the CIDR blocks listed in the output include the IP range of your node and is the same as the CIDR blocks listed in the error message. If they do not match, confirm the cluster name and region in your `nodeConfig.yaml` match your intended cluster. + + + + + + aws eks describe-cluster --name CLUSTER_NAME --region REGION_NAME --query cluster.remoteNetworkConfig.remoteNodeNetworks + + * Verify you’re working with the intended node: + + * Confirm you’re on the correct node by checking its hostname and IP address match the one you intend to register with the cluster. + + * Confirm this node is in the correct on-premises network (the one whose CIDR range was registered as `RemoteNodeNetwork` during cluster setup). + + + + +If your node IP is still not what you expected, check the following: + + * If you are using IAM Roles Anywhere, `kubelet` performs a DNS lookup on the IAM Roles Anywhere `nodeName` and uses an IP associated with the node name if available. If you maintain DNS entries for your nodes, confirm that these entries point to IPs within your remote node network CIDRs. + + * If your node has multiple network interfaces, `kubelet` might select an interface with an IP address outside your remote node network CIDRs as default. To use a different interface, specify its IP address using the `--node-ip` `kubelet` flag in your `nodeConfig.yaml`. For more information, see [Hybrid nodes nodeadm reference](./hybrid-nodes-nodeadm.html). You can view your node’s network interfaces and its IP addresses by running the following command on your node: + + + + + + ip addr show + @@ -70 +110 @@ When running `nodeadm init`, `nodeadm` attempts to gather information about your -If you ran `nodeadm init` and it completed but your hybrid nodes do not appear in your cluster, there may be issues with the network connection between your hybrid nodes and the EKS control plane, you may not have the required security group permissions configured, or you may not have the required mapping of your Hybrid Nodes IAM role to Kubernetes Role-Based Access Control (RBAC). You can start the debugging process by checking the status of kubelet and the kubelet logs with the following commands. Run the following commands from the hybrid nodes that failed to join your cluster. +If you ran `nodeadm init` and it completed but your hybrid nodes do not appear in your cluster, there might be issues with the network connection between your hybrid nodes and the EKS control plane, you might not have the required security group permissions configured, or you might not have the required mapping of your Hybrid Nodes IAM role to Kubernetes Role-Based Access Control (RBAC). You can start the debugging process by checking the status of `kubelet` and the `kubelet` logs with the following commands. Run the following commands from the hybrid nodes that failed to join your cluster. @@ -80 +120 @@ If you ran `nodeadm init` and it completed but your hybrid nodes do not appear i -If your hybrid node was unable to communicate with the cluster control plane, you may see logs similar to the following. +If your hybrid node was unable to communicate with the cluster control plane, you might see logs similar to the following. @@ -104 +144 @@ If you see these messages, check the following to ensure it meets the hybrid nod -If your hybrid node was able to communicate with the EKS control plane but was not able to register, you may see logs similar to the following. Note the key difference in the log messages below is the `Unauthorized` error. This signals that the node was not able to perform its tasks because it does not have the required permissions. +If your hybrid node was able to communicate with the EKS control plane but was not able to register, you might see logs similar to the following. Note the key difference in the log messages below is the `Unauthorized` error. This signals that the node was not able to perform its tasks because it does not have the required permissions. @@ -128 +168 @@ If you see these messages, check the following to ensure it meets the hybrid nod -If your hybrid nodes successfully registered with your EKS cluster, but the hybrid nodes show status `Not Ready`, the first thing to check is your Container Networking Interface (CNI) status. If you have not installed a CNI, then it is expected that your hybrid nodes have status `Not Ready`. Once a CNI is installed and running successfully, nodes transition to have the status Ready. If you attempted to install a CNI but it is not running successfully, see Hybrid nodes CNI troubleshooting on this page. +If your hybrid nodes successfully registered with your EKS cluster, but the hybrid nodes show status `Not Ready`, the first thing to check is your Container Networking Interface (CNI) status. If you have not installed a CNI, then it is expected that your hybrid nodes have status `Not Ready`. Once a CNI is installed and running successfully, nodes are updated to the status `Ready`. If you attempted to install a CNI but it is not running successfully, see Hybrid nodes CNI troubleshooting on this page. @@ -136 +176 @@ After connecting hybrid nodes to your EKS cluster, if you see that there are pen -If you changed your `nodeadm` configuration and attempt to re-register the node with the new configuration, you may see an error that states that the hybrid profile already exists but its contents have changed. Instead of running `nodeadm init` in between configuration changes, run `nodeadm uninstall` followed by a `nodeadm install` and `nodeadm init`. This ensures a proper clean up with the changes in configuration. +If you changed your `nodeadm` configuration and attempt to reregister the node with the new configuration, you might see an error that states that the hybrid profile already exists but its contents have changed. Instead of running `nodeadm init` in between configuration changes, run `nodeadm uninstall` followed by a `nodeadm install` and `nodeadm init`. This ensures a proper clean up with the changes in configuration. @@ -143 +183 @@ If you changed your `nodeadm` configuration and attempt to re-register the node -After running `nodeadm init`, if you see an error in the `kubelet` logs that shows failures to contact the EKS Kubernetes API server because there is `no such host`, you may have to change your DNS entry for the EKS Kubernetes API endpoint in your on-premises network or at the host level. See [Forwarding inbound DNS queries to your VPC](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-forwarding-inbound-queries.html) in the _AWS Route53 documentation_. +After running `nodeadm init`, if you see an error in the `kubelet` logs that shows failures to contact the EKS Kubernetes API server because there is `no such host`, you might have to change your DNS entry for the EKS Kubernetes API endpoint in your on-premises network or at the host level. See [Forwarding inbound DNS queries to your VPC](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-forwarding-inbound-queries.html) in the _AWS Route53 documentation_. @@ -154 +194 @@ If you have registered your hybrid nodes but are unable to view them in the EKS -If your hybrid nodes registered with your EKS cluster, had status `Ready`, and then transitioned to status `Not Ready`, there are a wide range of issues that may have contributed to the unhealthy status such as the node lacking sufficient resources for CPU, memory, or available disk space, or the node is disconnected from the EKS control plane. You can use the steps below to troubleshoot your nodes, and if you cannot resolve the issue, contact AWS Support. +If your hybrid nodes registered with your EKS cluster, had status `Ready`, and then transitioned to status `Not Ready`, there are a wide range of issues that might have contributed to the unhealthy status such as the node lacking sufficient resources for CPU, memory, or available disk space, or the node is disconnected from the EKS control plane. You can use the steps below to troubleshoot your nodes, and if you cannot resolve the issue, contact AWS Support. @@ -186 +226 @@ Run `nodeadm debug` from your hybrid nodes to validate networking and credential -**Check kubectl logs** +**Check`kubectl` logs** @@ -196 +236 @@ Run `nodeadm debug` from your hybrid nodes to validate networking and credential -If applications, add-ons, or webhooks running on your hybrid nodes are not starting properly, you may have networking issues that block the communication to the pods. For the EKS control plane to contact webhooks running on hybrid nodes, you must configure your EKS cluster with a remote pod network and have routes for your on-premises pod CIDR in your VPC routing table with the target as your Transit Gateway (TGW), virtual private gateway (VPW), or other gateway you are using to connect your VPC with your on-premises network. For more information on the networking requirements for hybrid nodes, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). You additionally must allow this traffic in your on-premises firewall and ensure your router can properly route to your pods. See [Configure webhooks for hybrid nodes](./hybrid-nodes-webhooks.html) for more information on the requirements for running webhooks on hybrid nodes. +If applications, add-ons, or webhooks running on your hybrid nodes are not starting properly, you might have networking issues that block the communication to the pods. For the EKS control plane to contact webhooks running on hybrid nodes, you must configure your EKS cluster with a remote pod network and have routes for your on-premises pod CIDR in your VPC routing table with the target as your Transit Gateway (TGW), virtual private gateway (VPW), or other gateway you are using to connect your VPC with your on-premises network. For more information on the networking requirements for hybrid nodes, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). You additionally must allow this traffic in your on-premises firewall and ensure your router can properly route to your pods. See [Configure webhooks for hybrid nodes](./hybrid-nodes-webhooks.html) for more information on the requirements for running webhooks on hybrid nodes. @@ -202,0 +243,14 @@ A common pod log message for this scenario is shown below the following where ip +**`kubectl logs` or `kubectl exec` commands not working** + +If `kubectl logs` or `kubectl exec` commands time out while other `kubectl` commands succeed, the issue is likely related to remote network configuration. These commands connect through the cluster to the `kubelet` endpoint on the node. For more information see [kubelet endpoint](./hybrid-nodes-concepts-kubernetes.html#hybrid-nodes-concepts-k8s-kubelet-api). + +Verify that your node IPs and pod IPs fall within the remote node network and remote pod network CIDRs configured for your cluster. Use the commands below to examine IP assignments. + + + kubectl get nodes -o wide + + + kubectl get pods -A -o wide + +Compare these IPs with your configured remote network CIDRs to ensure proper routing. For network configuration requirements, see [Prepare networking for hybrid nodes](./hybrid-nodes-networking.html). + @@ -225 +279 @@ On-premises network -Are there routes and access to/from the EKS control plane and to/from the hybrid nodes and the pods running on hybrid nodes? +Are there routes and access to and from the EKS control plane and to and from the hybrid nodes and the pods running on hybrid nodes? @@ -234 +288 @@ If using an overlay network, does the IP pool configuration for the CNI match th -If your hybrid nodes are showing status `Ready`, but you have not installed a CNI on your cluster, it is possible that there are old CNI artifacts on your hybrid nodes. By default, when you uninstall Cilium and Calico with tools such as Helm, the on-disk resources are not removed from your physical or virtual machines. Additionally, the Custom Resource Definitions (CRDs) for these CNIs may still be present on your cluster from an old installation. For more information, see the Delete Cilium and Delete Calico sections of [Configure a CNI for hybrid nodes](./hybrid-nodes-cni.html). +If your hybrid nodes are showing status `Ready`, but you have not installed a CNI on your cluster, it is possible that there are old CNI artifacts on your hybrid nodes. By default, when you uninstall Cilium and Calico with tools such as Helm, the on-disk resources are not removed from your physical or virtual machines. Additionally, the Custom Resource Definitions (CRDs) for these CNIs might still be present on your cluster from an old installation. For more information, see the Delete Cilium and Delete Calico sections of [Configure a CNI for hybrid nodes](./hybrid-nodes-cni.html). @@ -238 +292 @@ If your hybrid nodes are showing status `Ready`, but you have not installed a CN -If you are having issues running Cilium on hybrid nodes, see [the troubleshooting steps](https://docs.cilium.io/en/stable/operations/troubleshooting/) in the Cilium documentation. The sections below cover issues that may be unique to deploying Cilium on hybrid nodes. +If you are having issues running Cilium on hybrid nodes, see [the troubleshooting steps](https://docs.cilium.io/en/stable/operations/troubleshooting/) in the Cilium documentation. The sections below cover issues that might be unique to deploying Cilium on hybrid nodes. @@ -256 +310 @@ If you are using Cilium BGP Control Plane to advertise your pod or service addre -If BGP is working correctly, you should your hybrid nodes with Session State `established` in the output. You may need to work with your networking team to identify the correct values for your environment’s Local AS, Peer AS, and Peer Address. +If BGP is working correctly, you should your hybrid nodes with Session State `established` in the output. You might need to work with your networking team to identify the correct values for your environment’s Local AS, Peer AS, and Peer Address. @@ -264 +318 @@ If BGP is working correctly, you should your hybrid nodes with Session State `es -If you are using Cilium BGP to advertise the IPs of Services with type `LoadBalancer`, you must have the same label on both your `CiliumLoadBalancerIPPool` and Service, which should be used in the selector of your `CiliumBGPAdvertisement`. An example is shown below. Note, if you are using Cilium BGP to advertise the IPs of Services with type LoadBalancer, the BGP routes may be disrupted during Cilium agent restart. For more information, see [Failure Scenarios](https://docs.cilium.io/en/latest/network/bgp-control-plane/bgp-control-plane-operation/#failure-scenarios) in the Cilium documentation. +If you are using Cilium BGP to advertise the IPs of Services with type `LoadBalancer`, you must have the same label on both your `CiliumLoadBalancerIPPool` and Service, which should be used in the selector of your `CiliumBGPAdvertisement`. An example is shown below. Note, if you are using Cilium BGP to advertise the IPs of Services with type LoadBalancer, the BGP routes might be disrupted during Cilium agent restart. For more information, see [Failure Scenarios](https://docs.cilium.io/en/latest/network/bgp-control-plane/bgp-control-plane-operation/#failure-scenarios) in the Cilium documentation. @@ -321 +375 @@ If you are using Cilium BGP to advertise the IPs of Services with type `LoadBala -If you are having issues running Calico on hybrid nodes, see [the troubleshooting steps](https://docs.tigera.io/calico/latest/operations/troubleshoot/) in the Calico documentation. The sections below cover issues that may be unique to deploying Calico on hybrid nodes. +If you are having issues running Calico on hybrid nodes, see [the troubleshooting steps](https://docs.tigera.io/calico/latest/operations/troubleshoot/) in the Calico documentation. The sections below cover issues that might be unique to deploying Calico on hybrid nodes. @@ -328 +382 @@ Calico API server | Node -Calico kube controllers | Pod +Calico Controllers for Kubernetes | Pod @@ -330 +384 @@ Calico node agent | Node -Calico typha | Node +Calico `typha` | Node @@ -393 +447 @@ For both AWS SSM hybrid activations and AWS IAM Roles Anywhere, you can validate -If you are using AWS SSM hybrid activations for your hybrid nodes credentials, be aware of the following SSM directories and artifacts that are installed on your hybrid nodes by nodeadm. For more information on the SSM agent, see [Working with the SSM agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html) in the _AWS Systems Manager User Guide_. +If you are using AWS SSM hybrid activations for your hybrid nodes credentials, be aware of the following SSM directories and artifacts that are installed on your hybrid nodes by `nodeadm`. For more information on the SSM agent, see [Working with the SSM agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html) in the _AWS Systems Manager User Guide_. @@ -440 +494 @@ When running `nodeadm install` if you see an issue with the `ssm-setup-cli` chec -If you see an error registering your instance with AWS SSM, confirm the `region`, `activationCode`, and `activationId` in your `nodeConfig.yaml` are correct. The AWS Region for your EKS cluster must match the region of your SSM hybrid activation. If these values are misconfigured, you may see an error similar to the following. +If you see an error registering your instance with AWS SSM, confirm the `region`, `activationCode`, and `activationId` in your `nodeConfig.yaml` are correct. The AWS Region for your EKS cluster must match the region of your SSM hybrid activation. If these values are misconfigured, you might see an error similar to the following. @@ -447 +501 @@ If you see an error registering your instance with AWS SSM, confirm the `region` -If the SSM agent is not able to refresh credentials, you may see an `ExpiredTokenException`. In this scenario, if you are able to connect to the SSM endpoints from your hybrid nodes, you may need to restart the SSM agent to force a credential refresh. +If the SSM agent is not able to refresh credentials, you might see an `ExpiredTokenException`. In this scenario, if you are able to connect to the SSM endpoints from your hybrid nodes, you might need to restart the SSM agent to force a credential refresh. @@ -454 +508 @@ If the SSM agent is not able to refresh credentials, you may see an `ExpiredToke -If you see an error registering the machine with SSM, you may need to re-run `nodeadm install` to make sure all of the SSM dependencies are properly installed. +If you see an error registering the machine with SSM, you might need to re-run `nodeadm install` to make sure all of the SSM dependencies are properly installed. @@ -471 +525 @@ When running `nodeadm init`, if you see an error registering the instance with S -If you see a failure to refresh cached credentials, the `/root/.aws/credentials` file may have been deleted on your host. First check your SSM hybrid activation and ensure it is active and your hybrid nodes are configured correctly to use the activation. Check the SSM agent logs at `/var/log/amazon/ssm` and re-run the nodeadm init command once you have resolved the issue on the SSM side. +If you see a failure to refresh cached credentials, the `/root/.aws/credentials` file might have been deleted on your host. First check your SSM hybrid activation and ensure it is active and your hybrid nodes are configured correctly to use the activation. Check the SSM agent logs at `/var/log/amazon/ssm` and re-run the `nodeadm init` command once you have resolved the issue on the SSM side. @@ -488 +542 @@ To remove the SSM agent from your host, you can run the following commands. -If you are using AWS IAM Roles Anywhere for your hybrid nodes credentials, be aware of the following directories and artifacts that are installed on your hybrid nodes by nodeadm. For more information on the troubleshooting IAM Roles Anywhere, see [Troubleshooting AWS IAM Roles Anywhere identity and access](https://docs.aws.amazon.com/rolesanywhere/latest/userguide/security_iam_troubleshoot.html) in the _AWS IAM Roles Anywhere User Guide_. +If you are using AWS IAM Roles Anywhere for your hybrid nodes credentials, be aware of the following directories and artifacts that are installed on your hybrid nodes by `nodeadm`. For more information on the troubleshooting IAM Roles Anywhere, see [Troubleshooting AWS IAM Roles Anywhere identity and access](https://docs.aws.amazon.com/rolesanywhere/latest/userguide/security_iam_troubleshoot.html) in the _AWS IAM Roles Anywhere User Guide_. @@ -498 +552 @@ Default key location and name | `/etc/iam/pki/server.key` -If you see a failure to refresh cached credentials, review the contents of `/etc/aws/hybrid/config` and confirm that IAM Roles Anywhere was configured correctly in your `nodeadm` configuration. Confirm that `/etc/iam/pki` exists. Each node must have a unique certificate and key. By default, when using IAM Roles Anywhere as the credential provider, `nodeadm` uses `/etc/iam/pki/server.pem` for the certificate location and name, and `/etc/iam/pki/server.key` for the private key. You may need to create the directories before placing the certificates and keys in the directories with `sudo mkdir -p /etc/iam/pki`. You can verify the content of your certificate with the command below. +If you see a failure to refresh cached credentials, review the contents of `/etc/aws/hybrid/config` and confirm that IAM Roles Anywhere was configured correctly in your `nodeadm` configuration. Confirm that `/etc/iam/pki` exists. Each node must have a unique certificate and key. By default, when using IAM Roles Anywhere as the credential provider, `nodeadm` uses `/etc/iam/pki/server.pem` for the certificate location and name, and `/etc/iam/pki/server.key` for the private key. You might need to create the directories before placing the certificates and keys in the directories with `sudo mkdir -p /etc/iam/pki`. You can verify the content of your certificate with the command below. @@ -515 +569 @@ In the `kubelet` logs, if you see an access denied issue for the `sts:AssumeRole -**IAM Roles Anywhere not authorized to set roleSessionName** +**IAM Roles Anywhere not authorized to set`roleSessionName` **