AWS AmazonElastiCache documentation change
Summary
Added documentation on using ElastiCache for semantic caching in generative AI applications, including RAG and memory mechanisms
Security assessment
The changes introduce new AI/ML use cases and caching optimizations but contain no references to security vulnerabilities, access controls, or encryption. Mentions of TTLs and eviction strategies are operational features rather than security controls.
Diff
diff --git a/AmazonElastiCache/latest/dg/elasticache-use-cases.md b/AmazonElastiCache/latest/dg/elasticache-use-cases.md index 24fbccbd2..623030fa9 100644 --- a//AmazonElastiCache/latest/dg/elasticache-use-cases.md +++ b//AmazonElastiCache/latest/dg/elasticache-use-cases.md @@ -5 +5 @@ -In-Memory Data StoreGaming LeaderboardsMessaging (Pub/Sub)Recommendation Data (Hashes)ElastiCache Customer Testimonials +In-Memory Data StoreGaming LeaderboardsMessaging (Pub/Sub)Recommendation Data (Hashes)Semantic caching for generative AI applicationsElastiCache Customer Testimonials @@ -28,0 +29,2 @@ The following examples illustrate some of the ways using ElastiCache can improve + * Semantic caching for generative AI applications + @@ -203,0 +206,24 @@ Using INCR or DECR in Valkey or Redis OSS makes compiling recommendations simple +## Semantic caching for generative AI applications + +Operating generative AI applications at scale can be challenging due to the cost and latency associated with inference calls to large language models (LLMs). You can use ElastiCache for semantic caching in generative AI applications, allowing you to reduce the cost and latency of LLM inference calls. With semantic caching, you can return a cached response by using vector-based matching to find similarities between current and prior prompts. If a user’s prompt is semantically similar to a prior prompt, a cached response will be returned instead of making a new LLM inference call, reducing the cost of generative AI applications and providing faster responses that improve the user experience. You can control which queries are routed to the cache by configuring similarity thresholds for prompts and applying tag or numeric metadata filters. + +The inline real-time index updates provided by vector search for ElastiCache help ensure that the cache updates continuously as user prompts and LLM responses flow in. This real-time indexing is crucial to maintain freshness of cached results and cache hit rates, particularly for spiky traffic. In addition, ElastiCache simplifies operations for semantic caching through mature cache primitives such as per-key TTLs, configurable eviction strategies, atomic operations, and rich data structure and scripting support. + +**Memory for generative AI assistants and agents** + +You can use ElastiCache to deliver more personalized, context-aware responses by implementing memory mechanisms that surface cross-session conversation history to LLMs. Conversational memory allows generative AI assistants and agents to retain and use past interactions to personalize responses and improve relevancy. However, simply aggregating all prior interactions into the prompt is ineffective since irrelevant extra tokens increase cost, degrade response quality and risk exceeding the LLM’s context window. Instead, you can use vector search to retrieve and provide only the most relevant data in the context for each LLM invocation. + +ElastiCache for Valkey provides integrations with open-source memory layers, providing built-in connectors to store and retrieve memories for LLM applications and agents. Vector search for ElastiCache provides fast index updates, keeping memory up to date and making new memories immediately searchable. Low latency vector search makes memory lookups fast, enabling them to be implemented in the online path of every request, not just background tasks. Beyond vector search, ElastiCache for Valkey also provides caching primitives for session state, user preferences, and feature flags, providing a single service to store short-lived session state and long-term “memories” in one datastore. + +**Retrieval augmented generation (RAG)** + +RAG is the process of providing LLMs with up-to-date information in the prompt to improve the relevance of responses. RAG reduces hallucinations and improves factual accuracy by grounding outputs with real-world data sources. RAG applications use vector search to retrieve semantically relevant content from a knowledge base. Low latency vector search provided by ElastiCache makes it suitable for implementing RAG in workloads that have large datasets with millions of vectors and above. Further, support for online vector index updates makes ElastiCache suitable for assistants with upload workflows that need to ensure any uploaded data is immediately searchable. RAG in agentic AI systems ensures that agents have up-to-date information for accurate actions. Low latency vector search is also crucial for RAG in agentic AI systems where a single query can trigger multiple LLM calls and stack up latency from underlying vector search. + +The following diagram illustrates an example architecture using ElastiCache to implement a semantic cache, memory mechanisms, and RAG to enhance a generative AI application in production. + + + +**Semantic Search** + +Vector search retrieves the most relevant text, speech, image, or video data based on closeness in meaning or features. This capability enables machine learning applications that rely on similarity search across diverse data modalities, including recommendation engines, anomaly detection, personalization, and knowledge management systems. Recommendation systems use vector representations to capture complex patterns in user behavior and item characteristics, enabling them to suggest the most relevant content. Vector search for ElastiCache is well suited for these applications because of its near real-time updates and low latency, enabling similarity comparisons that deliver instant, highly relevant recommendations based on live user interactions. +