AWS res documentation change
Summary
Added detailed documentation for setting up custom domains with certificates and troubleshooting Keycloak failures
Security assessment
The custom domain setup documentation emphasizes proper certificate management via ACM/Secrets Manager and secure configuration of ALB/NLB listeners. While this improves security posture through best practices, there is no evidence it addresses a specific vulnerability.
Diff
diff --git a/res/archive/release-minus-4/ug/res-troubleshooting-issue-runbooks.md b/res/archive/release-minus-4/ug/res-troubleshooting-issue-runbooks.md index 5e8b699ec..ac7665bbd 100644 --- a//res/archive/release-minus-4/ug/res-troubleshooting-issue-runbooks.md +++ b//res/archive/release-minus-4/ug/res-troubleshooting-issue-runbooks.md @@ -12,0 +13,2 @@ The following section contains issues that may occur, how to detect them, and su + * I want to set up custom domains after I install RES + @@ -110,0 +113,2 @@ The following section contains issues that may occur, how to detect them, and su + * Demo stack keycloak not working + @@ -117,0 +122,2 @@ The following section contains issues that may occur, how to detect them, and su + * I want to set up custom domains after I install RES + @@ -134,0 +141,88 @@ The following section contains issues that may occur, how to detect them, and su +........................ + +### I want to set up custom domains after I install RES + +###### Note + + _Prerequisites_ : You must store Certificate and PrivateKey contents in a Secrets Manager secret before performing these steps. + +###### Add certs to the web client + + 1. Update the cert attached to the listener of the external-alb load balancer: + + 1. Navigate to the RES external load balancer in the AWS console under **EC2** > **Load Balancing** > **Load Balancers**. + + 2. Search for the load balancer that follows the naming convention ``<env-name>`-external-alb`. + + 3. Check the listeners attached to the load balancer. + + 4. Update the listener that has a Default SSL/TLS certificate attached with the new certificate details. + + 5. Save your changes. + + 2. In the cluster-settings table: + + 1. Find the cluster-settings table in DynamoDB -> Tables -> ``<env-name>`.cluster-settings`. + + 2. Go to **Explore Items** and **Filter by Attribute** – name "key", Type "string", condition "contains", and value "external_alb". + + 3. Set `cluster.load_balancers.external_alb.certificates.provided` to True. + + 4. Update the value of `cluster.load_balancers.external_alb.certificates.custom_dns_name`. This is the custom domain name for web user interface. + + 5. Update the value of `cluster.load_balancers.external_alb.certificates.acm_certificate_arn`. This is the Amazon Resource Name (ARN) for the corresponding certificate stored in Amazon Certificate Manager (ACM). + + 3. Update the corresponding Route53 subdomain record you created for your web client to point to the DNS name of the external alb load balancer `<env-name>-external-alb`. + + 4. If SSO is already configured in the environment, re-configure SSO with the same inputs as you used initially from the **General Settings** > **Identity Provider** > **Single Sign On** > **Status** > **Edit** button in the RES web portal. + + + + +###### Add certs to the VDIs + + 1. Grant the RES application permission to perform a GetSecret operation on the secret by adding the following tags to the secrets: + + * `res:EnvironmentName` : ``<env-name>`` + + * `res:ModuleName` : `virtual-desktop-controller` + + 2. In the cluster-settings table: + + 1. Find the cluster-settings table in DynamoDB -> Tables -> ``<env-name>`.cluster-settings`. + + 2. Go to **Explore Items** and **Filter by Attribute** – name "key", Type "string", condition "contains", and value "dcv_connection_gateway". + + 3. Set `vdc.dcv_connection_gateway.certificate.provided` to True. + + 4. Update the value of `vdc.dcv_connection_gateway.certificate.custom_dns_name`. This is the custom domain name for VDI access. + + 5. Update the value of `vdc.dcv_connection_gateway.certificate.certificate_secret_arn`. This is the ARN for the secret that holds the Certificate contents. + + 6. Update the value of `vdc.dcv_connection_gateway.certificate.private_key_secret_arn`. This is the ARN for the secret that holds the Private Key contents. + + 3. Update the launch template used for the gateway instance: + + 1. Open the Auto Scaling group in the AWS Console under **EC2** > **Auto Scaling** > **Auto Scaling Groups**. + + 2. Select the gateway auto scaling group that corresponds to the RES environment. The name follows the naming convention ``<env-name>`-vdc-gateway-asg`. + + 3. Find and open the Launch Template in the details section. + + 4. Under **Details** > **Actions** > choose **Modify template** (Create new version). + + 5. Scroll down to **Advanced details**. + + 6. Scroll to the very bottom, to **User data**. + + 7. Look for the words `CERTIFICATE_SECRET_ARN` and `PRIVATE_KEY_SECRET_ARN`. Update these values with the ARNs given to the secrets that hold the Certificate (see step 2.c) and Private Key (see step 2.d) contents. + + 8. Ensure the Auto Scaling group is configured to use the recently created version of the launch template (from the Auto Scaling group page). + + 4. Update the corresponding Route53 subdomain record you created for your virtual desktops to point to the DNS name of the external nlb load balancer: ``<env-name>`-external-nlb`. + + 5. Terminate the existing dcv-gateway instance: ``<env-name>`-vdc-gateway` and wait for a new one to spin up. + + + + @@ -268 +362 @@ If the logs display `Invalid credentials`, the ServiceAccount password entered d - 3. Choose **Reset user password**. + 3. Select **Reset user password**. @@ -280 +374 @@ If the logs display `Invalid credentials`, the ServiceAccount password entered d - 3. Open the secret to view the details page. Under **Secret Value** , select **Retrieve secret value** and choose **Plaintext**. + 3. Open the secret to view the details page. Under **Secret Value** , choose **Retrieve secret value** then choose **Plaintext**. @@ -282 +376 @@ If the logs display `Invalid credentials`, the ServiceAccount password entered d - 4. Select **Edit**. + 4. Choose **Edit**. @@ -284 +378 @@ If the logs display `Invalid credentials`, the ServiceAccount password entered d - 5. Set a new password for the ServiceAccount user and select **Save**. + 5. Set a new password for the ServiceAccount user and choose **Save**. @@ -435 +529 @@ This issue occurs when your SSO integration is misconfigured. To determine the i - 3. Open the table and select **Explore table items**. + 3. Open the table and choose **Explore table items**. @@ -445 +539 @@ This issue occurs when your SSO integration is misconfigured. To determine the i - 5. Select **Run**. + 5. Choose **Run**. @@ -487 +581 @@ If you have added a user to the Active Directory but they are missing in RES, th - 3. Select **Send** and receive messages. + 3. Choose **Send** and receive messages. @@ -548,0 +643,2 @@ Amazon EFS +Amazon FSx ONTAP + @@ -603 +699 @@ If the volume is going to be used by both Linux and Windows clients we need to s -Currently, when you are create Amazon FSx for NetApp ONTAP from the RES console, the file system gets provisioned but it does not join the domain. To join the created ONTAP file system SVM to your domain, see [Joining SVMs to a Microsoft Active Directory](https://docs.aws.amazon.com/fsx/latest/ONTAPGuide/self-managed-AD-join.html) and follow the steps on the [ Amazon FSx console](https://console.aws.amazon.com/fsx/home#file-systems). Make sure required [ permissions are delegated to the Amazon FSx Service Account](https://docs.aws.amazon.com/fsx/latest/ONTAPGuide/self-managed-AD-best-practices.html#connect_delegate_privileges) in AD. Once the SVM joins the domain successfully, go to SVM **Summary > Endpoints > SMB DNS name**, and copy the DNS name because you will need it later. +Currently, when you create Amazon FSx for NetApp ONTAP from the RES console, the file system gets provisioned but it does not join the domain. To join the created ONTAP file system SVM to your domain, see [Joining SVMs to a Microsoft Active Directory](https://docs.aws.amazon.com/fsx/latest/ONTAPGuide/self-managed-AD-join.html) and follow the steps on the [ Amazon FSx console](https://console.aws.amazon.com/fsx/home#file-systems). Make sure required [ permissions are delegated to the Amazon FSx Service Account](https://docs.aws.amazon.com/fsx/latest/ONTAPGuide/self-managed-AD-best-practices.html#connect_delegate_privileges) in AD. Once the SVM joins the domain successfully, go to SVM **Summary > Endpoints > SMB DNS name**, and copy the DNS name because you will need it later. @@ -609 +705 @@ After it is joined to the domain, edit the SMB DNS config key in the cluster set - 2. Select **Tables** , then choose `<stack-name>-cluster-settings`. + 2. Choose **Tables** , then choose `<stack-name>-cluster-settings`. @@ -623 +719 @@ After it is joined to the domain, edit the SMB DNS config key in the cluster set - 6. Select **Save and close**. + 6. Choose **Save and close**. @@ -630 +726 @@ In addition, ensure the security group associated with the file system allows tr -Alternatively, you may onboard an existing file system which is already joined to your domain using RES Onboard File System capability- from **Environment Management** select **File Systems** , **Onboard File System**. +Alternatively, you may onboard an existing file system which is already joined to your domain using RES Onboard File System capability- from **Environment Management** choose **File Systems** , **Onboard File System**. @@ -746 +842 @@ The default limit for the number of virtual desktops that a user can launch is 5 - * Select **Submit**. + * Choose **Submit**. @@ -809 +905 @@ The following are potential causes of this issue. -The bootstrap provisioning script failed to complete because the AMI does not have the expected configuration or tooling required. The log files on the instance, such as `/root/bootstrap/logs/` on a Linux instance, may contain useful information regarding this. AMIs ids taken from the AWS Marketplace may not work for RES desktop instances. They require testing to confirm if they are supported. +The bootstrap provisioning script failed to complete because the Amazon Machine Image (AMI) does not have the expected configuration or tooling required. The log files on the instance, such as `/root/bootstrap/logs/` on a Linux instance, may contain useful information regarding this. AMIs ids taken from the AWS Marketplace may not work for RES desktop instances. They require testing to confirm if they are supported. @@ -927 +1023 @@ Example log line: - * You can access the ServiceAccount Username provided during RES deployment from the [ SecretsManager console](https://console.aws.amazon.com/secretsmanager/listsecrets). Find the corresponding secret in Secrets manager and select **Retrieve Plain text**. If the Username is incorrect, select **Edit** to update the secret value. Terminate the current cluster-manager and vdc-controller instances. The new instances will come up in a stable state. + * You can access the ServiceAccount Username provided during RES deployment from the [ SecretsManager console](https://console.aws.amazon.com/secretsmanager/listsecrets). Find the corresponding secret in Secrets manager and choose **Retrieve Plain text**. If the Username is incorrect, choose **Edit** to update the secret value. Terminate the current cluster-manager and vdc-controller instances. The new instances will come up in a stable state. @@ -937 +1033 @@ Example log line: - * You can read the password you entered during env creation by accessing the secret that stores the password in the [Secrets Manager console](https://console.aws.amazon.com/secretsmanager/listsecrets). Select the secret (for example, `<env_name>directoryserviceServiceAccountPassword`) and select **Retrieve plain text**. + * You can read the password you entered during env creation by accessing the secret that stores the password in the [Secrets Manager console](https://console.aws.amazon.com/secretsmanager/listsecrets). Select the secret (for example, `<env_name>directoryserviceServiceAccountPassword`) and choose **Retrieve plain text**. @@ -939 +1035 @@ Example log line: - * If the password in the secret is incorrect, select **Edit** to update its value in the secret. Terminate the current cluster-manager and vdc-controller instances. The new instances will use the updated password and come up in a stable state. + * If the password in the secret is incorrect, choose **Edit** to update its value in the secret. Terminate the current cluster-manager and vdc-controller instances. The new instances will use the updated password and come up in a stable state. @@ -945 +1041 @@ Example log line: - 2. Select **Actions** , **Reset user password** then fill out the form with the username (for example, "ServiceAccount") and the new password. + 2. Choose **Actions** , **Reset user password** then fill out the form with the username (for example, "ServiceAccount") and the new password. @@ -966 +1062 @@ The SQS subscriber is busy and stuck in an infinite loop because it cannot get t -The reason it is not able to get to the user account may be that RES was not configured correctly for the AD in use. An example might be that the `ServiceAccountUsername` parameter used at BI/RES environment creation was not the correct value, such as using "ServiceAccount" instead of "Admin". +The reason it is not able to get to the user account may be that RES was not configured correctly for the AD in use. An example might be that the `ServiceAccountCredentialsSecretArn` parameter used at BI/RES environment creation was not the correct value. @@ -1042 +1138 @@ If you notice that the "res-xxx-cluster" stack is in "DELETE_FAILED" state and c -If you see the stack in a "DELETE_FAILED" state, first try to manually delete it. It may pop up a dialog confirming Delete Stack. Select **Delete**. +If you see the stack in a "DELETE_FAILED" state, first try to manually delete it. It may pop up a dialog confirming Delete Stack. Choose **Delete**. @@ -1046 +1142 @@ If you see the stack in a "DELETE_FAILED" state, first try to manually delete it -Sometimes, even if you delete all the required stack resources, you may still see the message to select resources to retain. In that case, select all the resources as the "resources to retain" and select **Delete**. +Sometimes, even if you delete all the required stack resources, you may still see the message to select resources to retain. In that case, select all the resources as the "resources to retain" and choose **Delete**. @@ -1054 +1150 @@ This means that the role required to delete the stack got deleted first before t - * For **Trusted entity type** select **AWS service**. + * For **Trusted entity type** choose **AWS service**. @@ -1063 +1159 @@ This means that the role required to delete the stack got deleted first before t -Select **Next**. Make sure you give the role '`AWSCloudFormationFullAccess`' and '`AdministratorAccess`' permissions. Your review page should look like this: +Choose **Next**. Make sure you give the role '`AWSCloudFormationFullAccess`' and '`AdministratorAccess`' permissions. Your review page should look like this: @@ -1144 +1240 @@ One of the ways to get the logs would be to upload them to S3 and then download - 4. In the **Permission Policies** section from the **Add permissions** dropdown menu, select **Attach Policies** then choose the **AmazonS3FullAccess** policy. + 4. In the **Permission Policies** section from the **Add permissions** dropdown menu, choose **Attach Policies** then select the **AmazonS3FullAccess** policy. @@ -1146 +1242 @@ One of the ways to get the logs would be to upload them to S3 and then download - 5. Select **Add permissions** to attach that policy. + 5. Choose **Add permissions** to attach that policy. @@ -1208 +1304 @@ After this, go to the S3 console, select the bucket with name `<environment_name - 1. Go to the deployed stack and choose the **Resources** tab. + 1. Go to the deployed stack and select the **Resources** tab. @@ -1224,0 +1321,2 @@ After this, go to the S3 console, select the bucket with name `<environment_name + * Demo stack keycloak not working + @@ -1250,0 +1349,31 @@ If you attempt to log in and get an 'Unexpected error when handling authenticati +........................ + +### Demo stack keycloak not working + +**Issue** + +If your keycloak server crashed and, when you restarted the server, the IP of the instance changed, this might have resulted in keycloak breaking– the login page of your RES portal either fails to load or becomes stuck in a loading state which never resolves. + +**Mitigation** + +You will need to delete the existing infrastructure and redeploy the Keycloak stack to restore Keycloak to a healthy state. Follow these steps: + + 1. Go to Cloudformation. You should see two keycloak related stacks there: + + * ``<env-name>`-RESSsoKeycloak-`<random characters>`` (Stack1) +