AWS opensearch-service documentation change
Summary
Complete restructuring of documentation with added sections about use cases, architecture, requirements, provisioning steps, and expanded best practices. Added detailed node recommendations table and CLI examples.
Security assessment
Changes focus on performance optimization, resource utilization, and operational guidance without mentioning security vulnerabilities, patches, or security-specific features. The ENI reduction mentioned relates to network efficiency rather than security.
Diff
diff --git a/opensearch-service/latest/developerguide/Dedicated-coordinator-nodes.md b/opensearch-service/latest/developerguide/Dedicated-coordinator-nodes.md index 514c8bcce..d87de3439 100644 --- a//opensearch-service/latest/developerguide/Dedicated-coordinator-nodes.md +++ b//opensearch-service/latest/developerguide/Dedicated-coordinator-nodes.md @@ -5 +5 @@ -Best practices +When to use dedicated coordinator nodesArchitecture and behaviorRequirements and limitationsProvisioning dedicated coordinator nodesBest practices @@ -7 +7 @@ Best practices -# Dedicated coordinator nodes +# Dedicated coordinator nodes in Amazon OpenSearch Service @@ -9 +9 @@ Best practices -Amazon OpenSearch Service offers the option to provision a dedicated coordinator nodes. Dedicated coordinator nodes relieve data nodes from the responsibilities of coordinating and hosting OpenSearch Dashboards and provide better resource utilization of data nodes which can improve OpenSearch domain resiliency. Dedicated coordinator nodes help in reducing the reservation of private IP addresses required for virtual private cloud (VPC) domains. Dedicated coordinator nodes enhance resource efficency, delivering up to a 15% increase in indexing througput and a 20% improvement in query performance depending on workload demands. For more information on enhanced resource efficiency, see [Improve OpenSearch Service cluster resiliency and performance with dedicated coordinator nodes](https://aws.amazon.com/blogs/big-data/improve-opensearch-service-cluster-resiliency-and-performance-with-dedicated-coordinator-nodes/). +Dedicated coordinator nodes in Amazon OpenSearch Service are specialized nodes that offload coordination tasks from data nodes. These tasks include managing search requests and hosting OpenSearch Dashboards. By separating these functions, dedicated coordinator nodes reduce the load on data nodes, which allows them to focus on data storage, indexing, and search operations. This improves overall cluster performance and resource utilization. @@ -11 +11 @@ Amazon OpenSearch Service offers the option to provision a dedicated coordinator -The primary function of a coordinator node role is to forward requests to the data nodes that hold the data and reduce each node's results into a single global result set. In the absence of a dedicated coordinator node, data nodes perform coordination roles alongside their core responsibilities of search, data storage, and indexing. Data nodes require hosting Kibana and OpenSearch Dashboards and can potentially face resource (memory and CPU) strain. Provisioning a dedicated coordinator node allows a cluster to offload coordination and dashboard hosting responsibilities from data nodes and can improve a domain's resilience. +Additionally, dedicated coordinator nodes help reduce the number of private IP addresses required for VPC configurations, which leads to more efficient network management. This setup can result in up to 15% improvement in indexing throughput and 20% better query performance, depending on workload characteristics. @@ -13 +13 @@ The primary function of a coordinator node role is to forward requests to the da -All OpenSearch versions and Elasticsearch (open source) versions 6.8 to 7.10 support provisioning of a dedicated coordinator node. Dedicated coordinator node provisioning is available for domains with dedicated cluster manager enabled. +## When to use dedicated coordinator nodes @@ -15 +15,66 @@ All OpenSearch versions and Elasticsearch (open source) versions 6.8 to 7.10 sup -OpenSearch VPC domains attach the elastic network interface (ENI) to dedicated coordinator nodes instead of data nodes. Dedicated coordinator nodes typically represent around 10% of total data nodes. As a result, significantly small number of private IP addresses will be reserved for VPC domains. +Dedicated coordinator nodes are most beneficial in the following scenarios. + + * **Large clusters** – In environments with a high volume of data or complex queries, offloading coordination tasks to dedicated nodes can improve cluster performance. + + * **Frequent queries** – Workloads involving frequent search queries or aggregations, especially those with complex date histograms or multiple aggregations, benefit from faster query processing. + + * **Heavy Dashboards use** – OpenSearch Dashboards can be resource-intensive. Offloading this responsibility to dedicated coordinator nodes reduces the strain on data nodes. + + + + +## Architecture and behavior + +In an OpenSearch cluster, dedicated coordinator nodes handle two key responsibilities. + + * **Request handling** – These nodes receive incoming search requests and forward them to the appropriate data nodes, which store the relevant data. They then consolidate the results from multiple data nodes into a single global result set, which is returned to the client. + + * **Dashboards hosting** – Coordinator nodes manage OpenSearch Dashboards, which relieves data nodes from the additional burden of hosting OpenSearch Dashboards and handling related traffic. + + + + +In VPC domains, dedicated coordinator nodes are assigned Elastic Network Interfaces (ENIs) rather than data nodes. This arrangement helps reduce the number of private IP addresses required for VPCs, which improves network efficiency. Typically, dedicated coordinator nodes make up around 10% of the total data nodes. + +## Requirements and limitations + +Dedicated coordinator nodes have the following requirements and limitations. + + * Dedicated coordinator nodes are supported in all OpenSearch versions and Elasticsearch versions 6.8 to 7.10. + + * To enable dedicated coordinator nodes, your domain must have dedicated master nodes enabled. For more information, see [Dedicated master nodes in Amazon OpenSearch Service](./managedomains-dedicatedmasternodes.html). + + * Provisioning dedicated coordinator nodes can incur additional costs. However, the improved resource efficiency and enhanced performance justify the investment, particularly for large or complex clusters. + + + + +## Provisioning dedicated coordinator nodes + +Perform the following steps to provision dedicated coordinator nodes in an existing domain. Make sure your domain has dedicated _master_ nodes enabled before you provision coordinator nodes. + +###### To provision dedicated coordinator nodes in the AWS Management Console + + 1. Sign in to the Amazon OpenSearch Service console at [https://console.aws.amazon.com/aos/home](https://console.aws.amazon.com/aos/home). + + 2. Choose **Domains** , then select the domain you want to modify. + + 3. In the **Cluster configuration** section, choose **Edit**. + + 4. Choose **Enable dedicated coordinator nodes**. + + 5. Select the instance type and number of coordinator nodes to provision. + + 6. Choose **Save changes**. It might take several minutes for the domain to update. + + + + +To provision dedicated coordinator nodes using the AWS CLI, use the [update-domain-config](https://docs.aws.amazon.com/cli/latest/reference/opensearch/update-domain-config.html) command. The following example provisions three `r6g.large.search` coordinator nodes in a domain. + + + aws opensearch update-domain-config \ + --domain-name my-opensearch-domain \ + --cluster-config InstanceCount=3,InstanceType=r6g.large.search,DedicatedCoordinatorCount=3,ZoneAwarenessEnabled=true,DedicatedCoordinatorEnabled=true + +This command enables dedicated coordinator nodes, sets the instance type and count for the coordinator nodes, and enables zone awareness for higher availability. @@ -19 +84,9 @@ OpenSearch VPC domains attach the elastic network interface (ENI) to dedicated c -This chapter provides best practices for provisioning a dedicated coordinator node and includes general guidelines that apply to many use cases. Each workload is unique, with unique characteristics, so no generic recommendation is exactly right for every use case. +Consider the following best practices when you use dedicated coordinator nodes. + + * Use general purpose instances for most use cases. They provide a balanced approach between cost and performance. Memory-optimized instances are ideal for workloads that require substantial memory resources, such as those that involve complex aggregations or large-scale searches. + + * A good starting point is to provision between 5% and 10% of your data nodes as dedicated coordinator nodes. For example, if your domain has 90 `r6g.large` data nodes, consider provisioning 5 to 9 `r6g.large` coordinator nodes. + + * To minimize the risk of a single point of failure, provision at least two dedicated coordinator nodes. This ensures that your cluster remains operational even if one node fails. + + * If you're using cross-Region search, provision dedicated coordinator nodes in the destination domains. Source domains typically don't handle coordination tasks. @@ -21 +94 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * General-purpose instances are sufficient for most of the use cases. + * For indexing-heavy environments, consider CPU-optimized instances that match the instance size of your data nodes for optimal performance. @@ -23 +96 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * Keep the instance family the same for a dedicated cooordinating node and data node. + * For memory-intensive workloads, use a slightly larger instance type for your dedicated coordinator nodes to help manage the increased memory demands. @@ -25 +98 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * Provision 5% to 10% of your domain's total data nodes as dedicated coordinator nodes. For example, if your domain has 90 R6g.large data nodes, you can plan for coordinator nodes to make up anywhere between 5 and 9 of the R6g.large instance type. + * Track the `CoordinatorCPUUtilization` Amazon CloudWatch metric. If it consistently exceeds 80%, it might indicate that you need larger or additional coordinator nodes to handle the load. @@ -27 +99,0 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * As a starting point, the instance size of your dedicated coordinator nodes should be the same as your domain's data nodes. However, in certain cases, it may be advisable to use a smaller instance type with a higher count to ensure availability. For example, if you have R6g.8xlarge as the instance size for a data node, you can try 3 instances of M6g.2xlarge size instead of one M6g.8xlarge size to achieve better availability. @@ -29 +100,0 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * Source domains typically do not perform coordination tasks. If you are using cross region search, provision a dedicated coordinator node on the destination domains. @@ -31 +101,0 @@ This chapter provides best practices for provisioning a dedicated coordinator no - * To avoid a single point of failure, provision a minimum of two dedicated coordinator nodes. @@ -32,0 +103 @@ This chapter provides best practices for provisioning a dedicated coordinator no +### Node recommendations by cluster size @@ -33,0 +105 @@ This chapter provides best practices for provisioning a dedicated coordinator no +Use the following guidelines as a starting point for provisioning dedicated coordinator nodes based on your cluster size. Adjust the number and type of nodes based on workload characteristics and performance metrics. @@ -34,0 +107,5 @@ This chapter provides best practices for provisioning a dedicated coordinator no +Cluster size | Recommended coordinator nodes | Instance type +---|---|--- +Small (up to 50 nodes) | 3-5 nodes | General purpose +Medium (50-100 nodes) | 5-9 nodes | Memory optimized +Large (100+ nodes) | 10-15 nodes | Memory optimized @@ -44 +121 @@ Petabyte scale -Dedicated manager nodes +Dedicated master nodes