AWS opensearch-service documentation change
Summary
Major restructuring of observability documentation, changing from a workflow-based explanation to a feature-focused approach highlighting OpenTelemetry-native integration, unified interface, and purpose-driven query languages. Removes detailed workflow steps and OpenSearch Dashboards comparison, focusing on OpenSearch UI.
Security assessment
The changes are purely documentation restructuring and feature updates. There is no mention of security vulnerabilities, patches, or security incidents. The changes focus on observability capabilities, user interfaces, and data processing workflows without any security context.
Diff
diff --git a/opensearch-service/latest/developerguide/observability.md b/opensearch-service/latest/developerguide/observability.md index a7f791840..0bd9203da 100644 --- a//opensearch-service/latest/developerguide/observability.md +++ b//opensearch-service/latest/developerguide/observability.md @@ -5 +5 @@ -The observability workflow in OpenSearch ServiceOpenSearch UI and OpenSearch Dashboards +What Amazon OpenSearch Service providesThe Amazon OpenSearch Service approach to observability @@ -9 +9 @@ The observability workflow in OpenSearch ServiceOpenSearch UI and OpenSearch Das -**Observability** is the practice of gaining insight into the internal state and performance of your complex systems by examining their outputs. Traditional monitoring can tell you that your system is down, observability helps you understand why it's down by allowing you to ask new questions about your data. +Observability is the practice of understanding the internal state and performance of complex systems by examining their outputs. Traditional monitoring tells you a system is down; observability helps you understand why by letting you ask new questions about your telemetry data. @@ -11 +11 @@ The observability workflow in OpenSearch ServiceOpenSearch UI and OpenSearch Das -Amazon OpenSearch Service provides a unified solution by collecting and correlating key types of telemetry data. +## What Amazon OpenSearch Service provides @@ -13 +13 @@ Amazon OpenSearch Service provides a unified solution by collecting and correlat - * **Logs** provide timestamped records of events, such as application errors, user requests, or system status messages. +Amazon OpenSearch Service provides a unified observability solution by collecting, correlating, and visualizing three types of telemetry data: @@ -15 +15 @@ Amazon OpenSearch Service provides a unified solution by collecting and correlat - * **Traces** represent the end-to-end journey of a request as it travels through the different services in a distributed system. + * **Logs** – Timestamped records of events. @@ -16,0 +17 @@ Amazon OpenSearch Service provides a unified solution by collecting and correlat + * **Traces** – End-to-end journey of requests through distributed services. @@ -17,0 +19 @@ Amazon OpenSearch Service provides a unified solution by collecting and correlat + * **Metrics** – Time-series data representing system health, through the Amazon Managed Service for Prometheus direct query integration. @@ -20 +21,0 @@ Amazon OpenSearch Service provides a unified solution by collecting and correlat -By bringing these data types together, Amazon OpenSearch Service helps operations teams, site reliability engineers, and developers detect, diagnose, and resolve operational issues faster. @@ -22 +22,0 @@ By bringing these data types together, Amazon OpenSearch Service helps operation -## The observability workflow in OpenSearch Service @@ -24 +24 @@ By bringing these data types together, Amazon OpenSearch Service helps operation -Getting data from your applications into OpenSearch Service for analysis uses a multi-stage pipeline: +By bringing these together in a single interface, Amazon OpenSearch Service helps operations teams, SREs, and developers detect, diagnose, and resolve issues faster. @@ -26 +26 @@ Getting data from your applications into OpenSearch Service for analysis uses a - +## The Amazon OpenSearch Service approach to observability @@ -28 +28 @@ Getting data from your applications into OpenSearch Service for analysis uses a - 1. **Collection** +Amazon OpenSearch Service differentiates itself in three key ways: @@ -30 +30 @@ Getting data from your applications into OpenSearch Service for analysis uses a -The process begins at the source with lightweight agents or collectors that gather telemetry data from your signal producers such as applications and infrastructure. Two common open-source agents are: + * **OpenTelemetry-native with OpenSearch Ingestion as the last mile** – Standardize on OTel for instrumentation and collection. Amazon OpenSearch Ingestion serves as the fully managed pipeline that filters, enriches, transforms, and routes your telemetry data before indexing. @@ -32 +32 @@ The process begins at the source with lightweight agents or collectors that gath - * [OpenTelemetry](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-otel.html) (OTel) collector – The industry standard and preferred method for collecting logs and traces. + * **Unified logs, traces, and metrics in OpenSearch UI** – Analyze all three signal types from a single observability workspace. Correlate a slow trace to its application logs, or overlay Prometheus metrics on your service dashboards. @@ -34 +34 @@ The process begins at the source with lightweight agents or collectors that gath - * [Fluent Bit](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-fluentbit.html) – A lightweight, high-performance log processor and forwarder that supports the OTel schema for logs and traces. + * **Purpose-driven query languages** – Use [Piped Processing Language (PPL)](https://observability.opensearch.org/docs/ppl/) for logs and traces, and PromQL for metrics. Each language is optimized for its signal type, giving you expressive querying without compromise. @@ -36 +35,0 @@ The process begins at the source with lightweight agents or collectors that gath - 2. **Ingestion (Amazon OpenSearch Ingestion)** @@ -38 +36,0 @@ The process begins at the source with lightweight agents or collectors that gath -After telemetry is collected, data is sent to OpenSearch Ingestion, a fully managed, serverless data pipeline. You can create custom pipelines to: @@ -40 +37,0 @@ After telemetry is collected, data is sent to OpenSearch Ingestion, a fully mana - * **Filter** – Remove unnecessary data to reduce storage costs. @@ -42 +39 @@ After telemetry is collected, data is sent to OpenSearch Ingestion, a fully mana - * **Enrich** – Add valuable metadata, such as geographic information from an IP address. + @@ -44 +41 @@ After telemetry is collected, data is sent to OpenSearch Ingestion, a fully mana - * **Transform and normalize** – Structure unstructured logs into a consistent format. +###### Note @@ -46,25 +43 @@ After telemetry is collected, data is sent to OpenSearch Ingestion, a fully mana - * **Route** – Send different types of data to different OpenSearch Service indexes or Amazon S3. - - 3. **Analytics and visualization** - -After processing, data is loaded into an OpenSearch Service domain or collection. You can store, index, and analyze vast amounts of data in near real time. You interact with this data through a visualization interface such as [OpenSearch UI's observability workspace](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/application.html) to run queries, build dashboards, and set up alerts. - - - - - - -## OpenSearch UI and OpenSearch Dashboards - -OpenSearch Service provides two distinct user interfaces for observability. We recommend that you use OpenSearch UI and set up an [observability workspace](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/application.html) for new installations and migrate from existing OpenSearch Dashboards. Below is a table outlining the benefits of OpenSearch UI v. traditional OpenSearch Dashboards. - -Feature | OpenSearch UI | OpenSearch Dashboards ----|---|--- -Data sources | **Multi-source** – can connect to multiple OpenSearch Service domains, OpenSearch Serverless collections, and other data sources. | **Single-source** – co-located with a single OpenSearch Service domain. -Updates | New features arrive here first because it is not tied to a specific OpenSearch version. | New features are tied to the OpenSearch version. Updates may be deprecated in the future. -Availability | Hosted in the AWS Cloud ensuring zero downtime during cluster upgrades. | Can become temporarily unavailable during domain maintenance and upgrades -Collaboration | Features workspaces for curated team collaboration on specific workflows. | Collaboration is based on sharing saved objects in a single domain. - -**Note** – To make getting started easier, we’ve created a new [Get Started](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/configure-client-otel.html) workflow for logs in the Amazon OpenSearch Service console which will set up a new OTel tailored ingestion pipeline, allow you to select an existing OpenSearch cluster, and create a new OpenSearch UI application with an observability workspace created. All you have to do is point your OTel agents to the new ingestion endpoint and you are ready to unlock insights on your OTel formatted data. - -Since ingestion and analytics are handled differently in logs and traces, we’ve created separate sections to dive deep into. +The observability features described in this section are available only in OpenSearch UI. They are not available in OpenSearch Dashboards. For new observability workloads, we recommend setting up an [Using OpenSearch UI in Amazon OpenSearch Service](./application.html) with an observability workspace. @@ -78 +51 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Security Analytics +Configure permissions @@ -80 +53 @@ Security Analytics -Logs +Get started