AWS opensearch-service documentation change
Summary
Major rewrite of the direct query documentation - removed detailed quota tables and region lists for S3, CloudWatch Logs, and Security Lake; added Amazon Managed Service for Prometheus as a new data source; updated query languages to include PromQL; streamlined getting started instructions with console link.
Security assessment
This is a documentation restructuring and feature update, not a security fix. The changes involve adding new data source support (Prometheus), updating query languages, and removing detailed quota/region tables. There is no evidence of addressing vulnerabilities, security incidents, or adding security features. The change appears to be routine documentation maintenance and feature expansion.
Diff
diff --git a/opensearch-service/latest/developerguide/direct-query.md b/opensearch-service/latest/developerguide/direct-query.md index 6fdfd291f..a15769499 100644 --- a//opensearch-service/latest/developerguide/direct-query.md +++ b//opensearch-service/latest/developerguide/direct-query.md @@ -5,2 +4,0 @@ -QuotasSupported AWS Regions - @@ -9,162 +7 @@ QuotasSupported AWS Regions -Use Amazon OpenSearch Service direct query to analyze data in Amazon CloudWatch Logs, Amazon S3, and Amazon Security Lake without building ingestion pipelines. This zero-ETL integration lets you query data in place using OpenSearch SQL or PPL, and explore it in **Discover**. - -To get started, configure your data source in the OpenSearch Service console. For Amazon S3, use domain connections and create tables with SQL in Query Workbench. CloudWatch Logs and Security Lake use preconfigured sources and AWS Glue Data Catalog tables. - -## Direct query quotas - -Your account has the following quotas related to OpenSearch Service direct queries. - -### Quotas for Amazon S3 - -Each time you initiate a query to an Amazon S3 data source, OpenSearch Service opens a _session_ and keeps it alive for at least three minutes. This reduces query latency by removing session start-up time in subsequent queries. - -Description | Maximum | Can override ----|---|--- -Connections per domain | 10 | Yes -Data sources per domain | 20 | Yes -Indexes per domain | 5 | Yes -Concurrent sessions per data source | 10 | Yes -Maximum OCU per query | 60 | Yes -Maximum query execution time (minutes) | 30 | Yes -Maximum OCUs per acceleration | 20 | Yes -Maximum ephemeral storage | 20 | Yes - -### Quotas for CloudWatch Logs - -###### Note - -If you're looking to perform direct queries using CloudWatch Logs Insights, make sure that you refer to [Additional information for CloudWatch Logs Insights users using OpenSearch SQL](./supported-directquery-sql.html#supported-sql-for-multi-log-queries). - -Description | Value | Soft limit? | Notes ----|---|---|--- -Account-level TPS limit across direct query APIs | 3 TPS | Yes | -Maximum number of data sources | 20 | Yes | Limit is per AWS account. -Maximum auto-refreshing indexes or materialized views | 30 | Yes | Limit is per data source. -Maximum concurrent queries | 30 | Yes | Limit is per data source and applies to queries in pending or running state. Includes interactive queries (for example, data retrieval commands like `SELECT`) and index queries (for example, operations like `CREATE`/`ALTER`). -Maximum concurrent OCU per query | 512 | Yes | OpenSearch Compute Units (OCU). Limit based on 15 executors and 1 driver, each with 16 vCPU and 32 GB memory. Represents concurrent processing power. -Maximum query execution time in minutes | 60 | No | Limit applies to OpenSearch PPL/SQL queries in CloudWatch Logs Insights. -Period for purging stale query IDs | 90 days | Yes | This is the time period after which OpenSearch Service purges query metadata for older entries. For example, calling GetDirectQuery or GetDirectQueryResult fails for queries older than 90 days. - -### Quotas for Security Lake - -Description | Value | Soft limit? | Notes ----|---|---|--- -Account-level TPS limit across direct query APIs | 3 TPS | Yes | -Maximum number of data sources | 20 | Yes | Limit is per AWS account. -Maximum auto-refreshing indexes or materialized views | 30 | Yes | Limit applies per data source. Only includes indices and materialized views (MVs) with auto-refresh set to true. -Maximum concurrent queries | 30 | Yes | Limit applies to queries in pending or running state. Includes interactive queries (for example, data retrieval commands like `SELECT`) and index queries (for example, operations like `CREATE`/`ALTER`/`DROP`). -Maximum concurrent OCU per query | 512 | Yes | OpenSearch Compute Units (OCU). Limit based on 15 executors and 1 driver, each with 16 vCPU and 32 GB memory. Represents concurrent processing power. -Maximum query execution time in minutes | 30 | No | Applies only to interactive queries (for example, data retrieval commands like `SELECT`). For `REFRESH` queries, the limit is 6 hours. -Period for purging stale query IDs | 90 days | Yes | This is the time period after which OpenSearch Service purges query metadata for older entries. For example, calling GetDirectQuery or GetDirectQueryResult fails for queries older than 90 days. - -## Supported AWS Regions - -The following AWS Regions are supported for OpenSearch Service direct queries in Amazon S3, CloudWatch Logs, and Security Lake: - -### Available AWS Regions for Amazon S3 - - * Asia Pacific (Hong Kong) - - * Asia Pacific (Mumbai) - - * Asia Pacific (Seoul) - - * Asia Pacific (Singapore) - - * Asia Pacific (Sydney) - - * Asia Pacific (Tokyo) - - * Canada (Central) - - * Europe (Frankfurt) - - * Europe (Ireland) - - * Europe (Stockholm) - - * US East (N. Virginia) - - * US East (Ohio) - - * US West (Oregon) - - - - -### Available AWS Regions for CloudWatch Logs - - * Asia Pacific (Mumbai) - - * Asia Pacific (Hong Kong) - - * Asia Pacific (Osaka) - - * Asia Pacific (Seoul) - - * Asia Pacific (Singapore) - - * Asia Pacific (Sydney) - - * Asia Pacific (Tokyo) - - * Canada (Central) - - * Europe (Frankfurt) - - * Europe (Ireland) - - * Europe (Stockholm) - - * Europe (Milan) - - * Europe (Spain) - - * US East (N. Virginia) - - * US East (Ohio) - - * US West (Oregon) - - * US West (N. California) - - * Europe (Paris) - - * Europe (London) - - * South America (Sao Paulo) - - - - -### Available AWS Regions for Security Lake - - * Asia Pacific (Mumbai) - - * Asia Pacific (Singapore) - - * Asia Pacific (Sydney) - - * Asia Pacific (Tokyo) - - * Canada (Central) - - * Europe (Frankfurt) - - * Europe (Ireland) - - * Europe (Stockholm) - - * US East (N. Virginia) - - * US East (Ohio) - - * US West (Oregon) - - * Europe (Paris) - - * Europe (London) - - * South America (Sao Paulo) - - +Use Amazon OpenSearch Service direct query to analyze data in Amazon CloudWatch Logs, Amazon S3, Amazon Security Lake, and Amazon Managed Service for Prometheus without building ingestion pipelines. This zero-ETL integration lets you query data in place using PromQL, PPL, or SQL, and explore it in **Discover**. @@ -171,0 +9 @@ The following AWS Regions are supported for OpenSearch Service direct queries in +To get started with Amazon Managed Service for Prometheus, CloudWatch Logs, or Security Lake, configure your data source in the [AWS Management Console](https://console.aws.amazon.com/aos/home#opensearch/data-sources). For Amazon S3, use domain connections and create tables with SQL in Query Workbench. CloudWatch Logs and Security Lake use preconfigured data sources and schema. For Amazon S3 and Security Lake, data is cataloged using AWS Glue Data Catalog tables—Amazon S3 requires you to create these tables manually, while Security Lake configures them automatically as part of the ingestion process. @@ -181 +19 @@ Restarting the Dashboards process -Pricing +Direct queries in S3