AWS opensearch-service documentation change
Summary
Updated documentation for OpenSearch Serverless semantic enrichment feature with improved structure, terminology consistency, and UI alignment. Changes include section renames, clarification of semantic search concepts, updated console instructions, and removal of throttling information.
Security assessment
The changes are editorial improvements and terminology updates without any security vulnerability fixes. The removal of throttling details doesn't indicate a security issue resolution. Network policy instructions were updated for UI accuracy but don't introduce new security concepts.
Diff
diff --git a/opensearch-service/latest/developerguide/serverless-semantic-enrichment.md b/opensearch-service/latest/developerguide/serverless-semantic-enrichment.md index 61286a9cf..cb3b81544 100644 --- a//opensearch-service/latest/developerguide/serverless-semantic-enrichment.md +++ b//opensearch-service/latest/developerguide/serverless-semantic-enrichment.md @@ -7 +7 @@ -IntroductionWhat is semantic searchModel details and performance benchmarkLanguages SupportedSet up an automatic semantic enrichment index for serverless collectionsUpdate an existing indexData ingestion and searchConfiguring permissions for automatic semantic enrichmentQuery RewritesLimitations of automatic semantic enrichmentPricing +OverviewSemantic search conceptsModel details and performance benchmarkLanguages supportedSet up an automatic semantic enrichment index for serverless collectionsUpdate an existing indexData ingestion and searchConfiguring permissions for automatic semantic enrichmentQuery rewritesLimitations of automatic semantic enrichmentPricing @@ -11 +11 @@ IntroductionWhat is semantic searchModel details and performance benchmarkLangua -## Introduction +## Overview @@ -15 +15 @@ The automatic semantic enrichment feature can help improve search relevance by u -## What is semantic search +## Semantic search concepts @@ -17 +17 @@ The automatic semantic enrichment feature can help improve search relevance by u -Traditional search engines rely on word-to-word matching (referred to as lexical search) to find results for queries. Although this works well for specific queries such as television model numbers, it struggles with more abstract searches. For example, when searching for "shoes for the beach," a lexical search merely matches individual words "shoes," "beach," "for," and "the" in catalog items, potentially missing relevant products like "water-resistant sandals" or "surf footwear" that don't contain the exact search terms. +Traditional search engines rely on word-to-word matching (referred to as lexical search) to find results for queries. Although this works well for specific queries such as television model numbers, it might not return relevant results for more abstract searches. For example, when searching for "shoes for the beach," a lexical search merely matches individual words "shoes," "beach," "for," and "the" in catalog items, potentially missing relevant products like "water-resistant sandals" or "surf footwear" that don't contain the exact search terms. @@ -32 +32 @@ Semantic search returns query results that incorporate not just keyword matching -While this feature handles the technical complexities behind the scenes without exposing the underlying model, we provide transparency through a brief model description and benchmark results to help you make informed decisions about feature adoption in your critical workloads. +Although this feature handles the technical complexities behind the scenes without exposing the underlying model, the following model description and benchmark results help you make informed decisions about feature adoption in your critical workloads. @@ -34 +34 @@ While this feature handles the technical complexities behind the scenes without -Automatic semantic enrichment uses a service-managed, pre-trained sparse model that works effectively without requiring custom fine-tuning. The model analyzes the fields you specify, expanding them into sparse vectors based on learned associations from diverse training data. The expanded terms and their significance weights are stored in native Lucene index format for efficient retrieval. We’ve optimized this process using [document-only mode,](https://docs.opensearch.org/docs/latest/vector-search/ai-search/neural-sparse-with-pipelines/#step-1a-choose-the-search-mode) where encoding happens only during data ingestion. Search queries are merely tokenized rather than processed through the sparse model, making the solution both cost-effective and performant. +Automatic semantic enrichment uses a service-managed, pre-trained sparse model that works effectively without requiring custom fine-tuning. The model analyzes the fields you specify, expanding them into sparse vectors based on learned associations from diverse training data. The expanded terms and their significance weights are stored in native Lucene index format for efficient retrieval. We’ve optimized this process using [document-only mode](https://docs.opensearch.org/docs/latest/vector-search/ai-search/neural-sparse-with-pipelines/#step-1a-choose-the-search-mode), where encoding happens only during data ingestion. Search queries are tokenized rather than processed through the sparse model, making the solution both cost-effective and performant. @@ -36 +36 @@ Automatic semantic enrichment uses a service-managed, pre-trained sparse model t -Our performance validation during feature development used the [MS MARCO](https://huggingface.co/datasets/BeIR/msmarco) passage retrieval dataset, featuring passages averaging 334 characters. For relevance scoring, we measured average Normalized Discounted Cumulative Gain (NDCG) for the first 10 search results (ndcg@10) on the [BEIR](https://github.com/beir-cellar/beir) benchmark for English content and average ndcg@10 on MIRACL for multilingual content. We assessed latency through client-side, 90th-percentile (p90) measurements and search response p90 [took values.](https://github.com/beir-cellar/beir) These benchmarks provide baseline performance indicators for both search relevance and response times. Here are the key benchmark numbers - +Performance validation during feature development used the [MS MARCO](https://huggingface.co/datasets/BeIR/msmarco) passage retrieval dataset, featuring passages averaging 334 characters. For relevance scoring, the metric used was average Normalized Discounted Cumulative Gain (NDCG) for the first 10 search results (ndcg@10) on the [BEIR](https://github.com/beir-cellar/beir) benchmark for English content and average ndcg@10 on MIRACL for multilingual content. Latency assessment used client-side, 90th-percentile (p90) measurements and search response p90 [took values.](https://github.com/beir-cellar/beir) These benchmarks provide baseline performance indicators for both search relevance and response times. The following are the key benchmark numbers: @@ -45 +45 @@ Our performance validation during feature development used the [MS MARCO](https: -Given the unique nature of each workload, we encourage you to evaluate this feature in your development environment using your own benchmarking criteria before making implementation decisions. +Given the unique nature of each workload, you can evaluate this feature in your development environment using your own benchmarking criteria before making implementation decisions. @@ -47 +47 @@ Given the unique nature of each workload, we encourage you to evaluate this feat -## Languages Supported +## Languages supported @@ -53 +53 @@ The feature supports English. In addition, the model also supports Arabic, Benga -Setting up an index with automatic semantic enrichment enabled for your text fields is easy, and you can manage it through the console, APIs, and CloudFormation templates during new index creation. To enable it for an existing index, you need to recreate the index with automatic semantic enrichment enabled for text fields. +You can set up an index with automatic semantic enrichment enabled for your text fields through the console, APIs, and CloudFormation templates during new index creation. To enable it for an existing index, you must recreate the index with automatic semantic enrichment enabled for text fields. @@ -55 +55 @@ Setting up an index with automatic semantic enrichment enabled for your text fie -Console experience - The AWS console allows you to easily create an index with automatic semantic enrichment fields. Once you select a collection, you will find the create index button at the top of the console. Once you click the create index button, you will find options to define automatic semantic enrichment fields. In one index, you can have combinations of automatic semantic enrichment for English and multilingual, as well as lexical fields. +With the AWS console, you can create an index with automatic semantic enrichment fields. After you select a collection, you can find the **Create index** button at the top of the console. After you choose **Create index** , the console provides options to define automatic semantic enrichment fields. In one index, you can have combinations of automatic semantic enrichment for English and multilingual, as well as lexical fields. @@ -59 +59 @@ Console experience - The AWS console allows you to easily create an index with a -API experience - To create an automatic semantic enrichment index using the AWS Command Line Interface (AWS CLI), use the create-index command: +To create an automatic semantic enrichment index using the AWS Command Line Interface (AWS CLI), use the create-index command: @@ -68 +68 @@ API experience - To create an automatic semantic enrichment index using the AWS -In the following example index-schema, the _title_semantic_ field has a field type set to _text_ and has parameter _semantic_enrichment_ set to status _ENABLED_. Setting the _semantic_enrichment_ parameter enables automatic semantic enrichment on the _title_semantic_ field. You can use the _language_options_ field to specify either _english_ or _MULTI-LINGUAL_. +In the following example index-schema, the `title_semantic` field has a field type set to `text` and has parameter `semantic_enrichment` set to status `ENABLED`. Setting the `semantic_enrichment` parameter enables automatic semantic enrichment on the `title_semantic` field. You can use the `language_options` field to specify either `english` or `multi-lingual`. @@ -187 +187 @@ If the index has a custom ingest or search pipeline that was not created by auto -Once you've created an index with automatic semantic enrichment enabled, the feature works automatically during data ingestion process, no additional configuration required. +After you create an index with automatic semantic enrichment enabled, the feature works automatically during data ingestion process, no additional configuration required. @@ -206 +206 @@ Search: The semantic enrichment data is already indexed, so queries run efficien -Before creating an automated semantic enrichment index, you need to configure the required permissions. This section explains the permissions needed and how to set them up. +Before creating an automated semantic enrichment index, you must configure the required permissions. This section explains the permissions needed and how to set them up. @@ -334 +334 @@ To allow service APIs to access private collections, you must configure network - 2. In the left navigation, choose _Network policies_. Then do one of the following: + 2. In the left navigation pane, choose **Network policies**. Then do one of the following: @@ -336 +336 @@ To allow service APIs to access private collections, you must configure network - * Choose an existing policy name and choose _Edit_ + * Choose an existing policy name and choose **Edit** @@ -338 +338 @@ To allow service APIs to access private collections, you must configure network - * Choose _Create network policy_ and configure the policy details + * Choose **Create network policy** and configure the policy details @@ -340 +340 @@ To allow service APIs to access private collections, you must configure network - 3. In the _Access type_ area, choose _Private (recommended)_ , and then select _AWS service private access_. + 3. In the **Access type** area, choose **Private (recommended)** , and then select **AWS service private access**. @@ -342 +342 @@ To allow service APIs to access private collections, you must configure network - 4. In the search field, choose _Service_ , and then choose _aoss.amazonaws.com_. + 4. In the search field, choose **Service** , and then choose **aoss.amazonaws.com**. @@ -344 +344 @@ To allow service APIs to access private collections, you must configure network - 5. In the _Resource type_ area, select the _Enable access to OpenSearch endpoint_ box. + 5. In the **Resource type** area, select the **Enable access to OpenSearch endpoint** checkbox. @@ -346 +346 @@ To allow service APIs to access private collections, you must configure network - 6. For _Search collection(s), or input specific prefix term(s)_ , in the search field, select _Collection Name_. Then enter or select the name of the collections to associate with the network policy. + 6. For **Search collection(s), or input specific prefix term(s)** , in the search field, select **Collection Name**. Then enter or select the name of the collections to associate with the network policy. @@ -348 +348 @@ To allow service APIs to access private collections, you must configure network - 7. Choose _Create_ for a new network policy or _Update_ for an existing network policy. + 7. Choose **Create** for a new network policy or **Update** for an existing network policy. @@ -353 +353 @@ To allow service APIs to access private collections, you must configure network -## Query Rewrites +## Query rewrites @@ -359 +359 @@ Automatic semantic enrichment automatically converts your existing “match” q -Automatic semantic search is most effective when applied to small-to-medium sized fields containing natural language content, such as movie titles, product descriptions, reviews, and summaries. Although semantic search enhances relevance for most use cases, it might not be optimal for certain scenarios. Consider following limitations when deciding whether to implement automatic semantic enrichment for your specific use case. +Automatic semantic search is most effective when applied to small-to-medium sized fields containing natural language content, such as movie titles, product descriptions, reviews, and summaries. Although semantic search enhances relevance for most use cases, it might not be optimal for certain scenarios. Consider the following limitations when deciding whether to implement automatic semantic enrichment for your specific use case. @@ -367,2 +366,0 @@ Automatic semantic search is most effective when applied to small-to-medium size - * Throttling – Indexing inference requests are currently capped at 100 TPS for OpenSearch Serverless. This is a soft limit; reach out to AWS Support for higher limits. -