AWS bedrock documentation change
Summary
Added documentation for extended 1-hour TTL option in prompt caching, updated model support table with TTL details, added guidance on effective cache usage scenarios and constraints, and included API examples for TTL specification.
Security assessment
The changes focus on feature enhancements (extended TTL options) and usage guidance without addressing vulnerabilities, security incidents, or security weaknesses. No security-related terminology or vulnerability fixes are present in the diff.
Diff
diff --git a/bedrock/latest/userguide/prompt-caching.md b/bedrock/latest/userguide/prompt-caching.md index eb02bf3e5..b390c13d0 100644 --- a//bedrock/latest/userguide/prompt-caching.md +++ b//bedrock/latest/userguide/prompt-caching.md @@ -5 +5 @@ -How it worksSupported models, Regions, and limitsSimplified Cache Management for Claude ModelsGetting started +How it worksSupported models, Regions, and limitsSimplified Cache Management for Claude ModelsHow to effectively use prompt cachingGetting started @@ -9,4 +8,0 @@ How it worksSupported models, Regions, and limitsSimplified Cache Management for -###### Note - -Amazon Bedrock prompt caching is generally available with Claude 3.7 Sonnet, Claude 3.5 Haiku, Amazon Nova Micro, Amazon Nova Lite, Amazon Nova Pro, and Amazon Nova Premier. Customers who were given access to Claude 3.5 Sonnet v2 during the prompt caching preview will retain their access, however no additional customers will be granted access to prompt caching on the Claude 3.5 Sonnet v2 model. - @@ -23 +19 @@ If you opt to use prompt caching, Amazon Bedrock creates a cache composed of _ca -Cache checkpoints have a minimum and maximum number of tokens, dependent on the specific model you're using. You can only create a cache checkpoint if your total prompt prefix meets the minimum number of tokens. For example, the Anthropic Claude 3.7 Sonnet model requires at least 1,024 tokens per cache checkpoint. That means that your first cache checkpoint can be defined after 1,024 tokens and your second cache checkpoint can be defined after 2,048 tokens. If you try to add a cache checkpoint before meeting the minimum number of tokens, your inference will still succeed, but your prefix will not be cached. The cache has a five minute Time To Live (TTL), which resets with each successful cache hit. During this period, the context in the cache is preserved. If no cache hits occur within the TTL window, your cache expires. +Cache checkpoints have a minimum and maximum number of tokens, dependent on the specific model you're using. You can only create a cache checkpoint if your total prompt prefix meets the minimum number of tokens. For example, the Anthropic Claude 3.7 Sonnet model requires at least 1,024 tokens per cache checkpoint. That means that your first cache checkpoint can be defined after 1,024 tokens and your second cache checkpoint can be defined after 2,048 tokens. If you try to add a cache checkpoint before meeting the minimum number of tokens, your inference will still succeed, but your prefix will not be cached. The cache has a Time To Live (TTL), which resets with each successful cache hit. During this period, the context in the cache is preserved. If no cache hits occur within the TTL window, your cache expires. Most models support a 5-minute TTL, while Claude Opus 4.5, Claude Haiku 4.5, and Claude Sonnet 4.5 also support an extended 1-hour TTL option. @@ -53,15 +49,15 @@ The following table lists the supported models along with their token minimums, -Model name | Model ID | Release Type | Minimum number of tokens per cache checkpoint | Maximum number of cache checkpoints per request | Fields that accept prompt cache checkpoints ----|---|---|---|---|--- -Claude Opus 4.5 | anthropic.claude-opus-4-5-20251101-v1:0 | Generally Available | 4,096 | 4 | `system`, `messages`, and `tools` -Claude Opus 4.1 | anthropic.claude-opus-4-1-20250805-v1:0 | Generally Available | 1,024 | 4 | `system`, `messages`, and `tools` -Claude Opus 4 | anthropic.claude-opus-4-20250514-v1:0 | Generally Available | 1,024 | 4 | `system`, `messages`, and `tools` -Claude Sonnet 4.5 | anthropic.claude-sonnet-4-5-20250929-v1:0 | Generally Available | 1,024 | 4 | `system`, `messages`, and `tools` -Claude Haiku 4.5 | anthropic.claude-haiku-4-5-20251001-v1:0 | Generally Available | 4,096 | 4 | `system`, `messages`, and `tools` -Claude Sonnet 4 | anthropic.claude-sonnet-4-20250514-v1:0 | Generally Available | 1,024 | 4 | `system`, `messages`, and `tools` -Claude 3.7 Sonnet | anthropic.claude-3-7-sonnet-20250219-v1:0 | Generally Available | 1,024 | 4 | `system`, `messages`, and `tools` -Claude 3.5 Haiku | anthropic.claude-3-5-haiku-20241022-v1:0 | Generally Available | 2,048 | 4 | `system`, `messages`, and `tools` -Claude 3.5 Sonnet v2 | anthropic.claude-3-5-sonnet-20241022-v2:0 | Preview | 1,024 | 4 | `system`, `messages`, and `tools` -Amazon Nova Micro | amazon.nova-micro-v1:0 | Generally available | 1K1 | 4 | `system` and `messages` -Amazon Nova Lite | amazon.nova-lite-v1:0 | Generally available | 1K1 | 4 | `system` and `messages`2 -Amazon Nova Pro | amazon.nova-pro-v1:0 | Generally available | 1K1 | 4 | `system` and `messages`2 -Amazon Nova Premier | amazon.nova-premier-v1:0 | Generally available | 1K1 | 4 | `system` and `messages`2 +Model name | Model ID | Release Type | Minimum number of tokens per cache checkpoint | Maximum number of cache checkpoints per request | Supported TTL | Fields that accept prompt cache checkpoints +---|---|---|---|---|---|--- +Claude Opus 4.5 | anthropic.claude-opus-4-5-20251101-v1:0 | Generally Available | 4,096 | 4 | 5 minutes, 1 hour | `system`, `messages`, and `tools` +Claude Opus 4.1 | anthropic.claude-opus-4-1-20250805-v1:0 | Generally Available | 1,024 | 4 | 5 minutes | `system`, `messages`, and `tools` +Claude Opus 4 | anthropic.claude-opus-4-20250514-v1:0 | Generally Available | 1,024 | 4 | 5 minutes | `system`, `messages`, and `tools` +Claude Sonnet 4.5 | anthropic.claude-sonnet-4-5-20250929-v1:0 | Generally Available | 1,024 | 4 | 5 minutes, 1 hour | `system`, `messages`, and `tools` +Claude Haiku 4.5 | anthropic.claude-haiku-4-5-20251001-v1:0 | Generally Available | 4,096 | 4 | 5 minutes, 1 hour | `system`, `messages`, and `tools` +Claude Sonnet 4 | anthropic.claude-sonnet-4-20250514-v1:0 | Generally Available | 1,024 | 4 | 5 minutes | `system`, `messages`, and `tools` +Claude 3.7 Sonnet | anthropic.claude-3-7-sonnet-20250219-v1:0 | Generally Available | 1,024 | 4 | 5 minutes | `system`, `messages`, and `tools` +Claude 3.5 Haiku | anthropic.claude-3-5-haiku-20241022-v1:0 | Generally Available | 2,048 | 4 | 5 minutes | `system`, `messages`, and `tools` +Claude 3.5 Sonnet v2 | anthropic.claude-3-5-sonnet-20241022-v2:0 | Preview | 1,024 | 4 | 5 minutes | `system`, `messages`, and `tools` +Amazon Nova Micro | amazon.nova-micro-v1:0 | Generally available | 1K1 | 4 | 5 minutes | `system` and `messages` +Amazon Nova Lite | amazon.nova-lite-v1:0 | Generally available | 1K1 | 4 | 5 minutes | `system` and `messages`2 +Amazon Nova Pro | amazon.nova-pro-v1:0 | Generally available | 1K1 | 4 | 5 minutes | `system` and `messages`2 +Amazon Nova Premier | amazon.nova-premier-v1:0 | Generally available | 1K1 | 4 | 5 minutes | `system` and `messages`2 @@ -72,0 +69,2 @@ Amazon Nova Premier | amazon.nova-premier-v1:0 | Generally available | 1K1 | 4 | +To use the 1-hour TTL option with supported models (Claude Opus 4.5, Claude Haiku 4.5, and Claude Sonnet 4.5), specify the `ttl` field in your cache checkpoint. In the Converse API, add `"ttl": "1h"` to your `cachePoint` object. In the InvokeModel API for Claude models, add `"ttl": "1h"` to your `cache_control` object. If no `ttl` value is provided, the default 5-minute caching behavior applies. The 1-hour TTL is useful for longer-running sessions or batch processing scenarios where you want to maintain the cache across extended periods. + @@ -86,0 +85,17 @@ The automatic prefix checking only looks back approximately 20 content blocks fr +## How to effectively use prompt caching + +If you have prompts that are used at a regular cadence (i.e., system prompts that are used more frequently than every 5 minutes), continue to use the 5-minute cache, since this will continue to be refreshed at no additional charge. + +The 1-hour cache is best used in the following scenarios: + + * When you have prompts that are likely used less frequently than 5 minutes, but more frequently than every hour. For example, when an agentic side-agent will take longer than 5 minutes, or when storing a long chat conversation with a user and you generally expect that user may not respond in the next 5 minutes. + + * When latency is important and your follow-up prompts may be sent beyond 5 minutes. + + * When you want to improve your rate limit utilization, since cache hits are not deducted against your rate limit. + + + + +You can use both 1-hour and 5-minute cache controls in the same request, but with an important constraint: Cache entries with longer TTL must appear before shorter TTLs (i.e., a 1-hour cache entry must appear before any 5-minute cache entries). + @@ -96,0 +112,9 @@ The following examples show a cache checkpoint set in the `messages`, `system`, +Specify the desired ttl value as below, when ttl value not specified the default behavior of 5 minutes caching applies. + + + "cachePoint" : { + "type": "default", + "ttl" : "5m | 1h" + } + + @@ -183 +207 @@ In this example, you provide your tool definition in the `toolSpec` field. (Alte -The model response from the Converse API includes two new fields that are specific to prompt caching. The `CacheReadInputTokens` and `CacheWriteInputTokens` values tell you how many tokens were read from the cache and how many tokens were written to the cache because of your previous request. These are values that you're charged for by Amazon Bedrock, at a rate that's lower than the cost of full model inference. +The model response from the Converse API includes three new fields that are specific to prompt caching. The `CacheReadInputTokens` and `CacheWriteInputTokens` values tell you how many tokens were read from the cache and how many tokens were written to the cache because of your previous request. The `CacheDetails` values tell you the ttl used for the number of token written to cache. These are values that you're charged for by Amazon Bedrock, at a rate that's lower than the cost of full model inference. @@ -191,0 +216,9 @@ The following example shows how to structure the body of your InvokeModel reques +Specify the desired ttl value as below, when ttl value not specified the default behavior of 5 minutes caching applies. + + + "cache_control" : { + "type": "ephemeral", + "ttl" : "5m | 1h" + } + +