AWS bedrock documentation change
Summary
Restructured documentation to focus on choosing between Geographic vs Global cross-Region inference types, added comparison table of features, removed implementation details, and added security considerations about data encryption and logging
Security assessment
Added documentation about data being encrypted in transit between regions and CloudTrail logging of inference regions. While these are security-related features, there is no evidence of addressing a specific vulnerability or security incident.
Diff
diff --git a/bedrock/latest/userguide/cross-region-inference.md b/bedrock/latest/userguide/cross-region-inference.md index 161cde67f..78893d763 100644 --- a//bedrock/latest/userguide/cross-region-inference.md +++ b//bedrock/latest/userguide/cross-region-inference.md @@ -5 +5 @@ -Use a cross-Region (system-defined) inference profile +Choosing between Geographic and Global cross-Region inferenceGeneral considerations @@ -11 +11 @@ With cross-Region inference, you can choose either a cross-Region inference prof -When running model inference in on-demand mode, your requests might be restricted by service quotas or during peak usage times. Cross-Region inference enables you to seamlessly manage unplanned traffic bursts by utilizing compute across different AWS Regions. With cross-Region inference, you can distribute traffic across multiple AWS Regions, enabling higher throughput. +Both types of cross-Region inference work through [inference profiles](./inference-profiles.html), which define a foundation model (FM) and the AWS Regions to which requests can be routed. When running model inference in on-demand mode, your requests might be restricted by service quotas or during peak usage times. Cross-Region inference enables you to seamlessly manage unplanned traffic bursts by utilizing compute across different AWS Regions. @@ -17 +17 @@ To see the Regions and models with which you can use inference profiles to run c -Cross-region (system-defined) inference profiles are named after the model that they support and defined by the Regions that they support. To understand how a cross-region inference profile handles your requests, review the following definitions: +###### Topics @@ -19 +19 @@ Cross-region (system-defined) inference profiles are named after the model that - * **Source Region** – The Region from which you make the API request that specifies the inference profile. + * Choosing between Geographic and Global cross-Region inference @@ -21 +21 @@ Cross-region (system-defined) inference profiles are named after the model that - * **Destination Region** – A Region to which the Amazon Bedrock service can route the request from your source Region. + * General considerations @@ -22,0 +23 @@ Cross-region (system-defined) inference profiles are named after the model that + * [Geographic cross-Region inference](./geographic-cross-region-inference.html) @@ -23,0 +25 @@ Cross-region (system-defined) inference profiles are named after the model that + * [Global cross-Region inference](./global-cross-region-inference.html) @@ -26 +27,0 @@ Cross-region (system-defined) inference profiles are named after the model that -When you invoke a cross-Region inference profile in Amazon Bedrock, your request originates from a source Region and is automatically routed to one of the destination Regions defined in that profile, optimizing for performance. The destination Regions for Global cross-Region inference profiles include all commercial Regions. @@ -28 +28,0 @@ When you invoke a cross-Region inference profile in Amazon Bedrock, your request -###### Note @@ -30 +30 @@ When you invoke a cross-Region inference profile in Amazon Bedrock, your request -The destination Regions in a cross-Region inference profile can include _opt-in Regions_ , which are Regions that you must explicitly enable at AWS account or Organization level. To learn more, see [Enable or disable AWS Regions in your account](https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-regions.html). When using a cross-Region inference profile, your inference request can be routed to any of the destination Regions in the profile, even if you did not opt-in to such Regions in your account. +## Choosing between Geographic and Global cross-Region inference @@ -32 +32 @@ The destination Regions in a cross-Region inference profile can include _opt-in -Service Control Policies (SCPs) and AWS Identity and Access Management (IAM) policies work together to control where cross-Region inference is allowed. Using SCPs, you can control which Regions Amazon Bedrock can use for inference, and using IAM policies, you can define which users or roles have permission to run inference. If any destination Region in a cross-Region inference profile is blocked in your SCPs, the request will fail even if other Regions remain allowed. To ensure efficient operation with cross-region inference, you can update your SCPs and IAM policies to allow all required Amazon Bedrock inference actions (for example, `bedrock:InvokeModel*` or `bedrock:CreateModelInvocationJob`) in all destination Regions included in your chosen inference profile. To learn more, see [Enabling Amazon Bedrock cross-Region inference in multi-account environments.](https://aws.amazon.com/blogs/machine-learning/enable-amazon-bedrock-cross-region-inference-in-multi-account-environments/) +Amazon Bedrock provides two types of cross-Region inference profiles, each designed for different use cases and compliance requirements: @@ -34 +34,7 @@ Service Control Policies (SCPs) and AWS Identity and Access Management (IAM) pol -###### Note +Feature | Geographic Cross-Region Inference | Global Cross-Region Inference | Recommendation +---|---|---|--- +Data residency | Within geographic boundaries (US, EU, APAC, etc.) | Any supported AWS commercial Region worldwide | Choose Geographic for compliance requirements +Throughput | Higher than single-region | Highest available | Choose Global for maximum performance +Cost | Standard pricing | Approximately 10% savings | Choose Global for cost optimization +SCP requirements | Allow all destination Regions in profile | Allow `"aws:RequestedRegion": "unspecified"` | Configure based on your organizational policies +Best suited for | Organizations with data residency regulations | Organizations prioritizing cost and performance | Assess your compliance and performance needs @@ -36 +42 @@ Service Control Policies (SCPs) and AWS Identity and Access Management (IAM) pol -Some inference profiles route to different destination Regions depending on the source Region from which you call it. For example, if you call `us.anthropic.claude-3-haiku-20240307-v1:0` from US East (Ohio), it can route requests to `us-east-1`, `us-east-2`, or `us-west-2`, but if you call it from US West (Oregon), it can route requests to only `us-east-1` and `us-west-2`. +Choose Geographic cross-Region inference when you have data residency requirements and need to ensure data processing remains within specific geographic boundaries. Choose Global cross-Region inference when you want maximum throughput and cost savings without geographic restrictions. @@ -38,14 +44 @@ Some inference profiles route to different destination Regions depending on the -To check the source and destination Regions for an inference profile, you can do one of the following: - - * Expand the corresponding section in the [list of supported cross-region inference profiles](./inference-profiles-support.html). - - * Send a [GetInferenceProfile](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetInferenceProfile.html) request with an [Amazon Bedrock control plane endpoint](https://docs.aws.amazon.com/general/latest/gr/bedrock.html#br-cp) from a source Region and specify the Amazon Resource Name (ARN) or ID of the inference profile in the `inferenceProfileIdentifier` field. The `models` field in the response maps to a list of model ARNs, in which you can identify each destination Region. - - - - -###### Note - -Global cross-Region inference profile for a specific model can change over time as AWS adds more commercial Regions where your requests can be processed. However, if an inference profile is tied to a geography (such as US, EU, or APAC), its destination Region list will never change. AWS might create new inference profiles that incorporate new Regions. You can update your systems to use these inference profiles by changing the IDs in your setup to the new ones. - -The Global cross-region inference profile is currently only supported on Anthropic Claude Sonnet 4 model for the following source Regions: US West (Oregon), US East (N. Virginia), US East (Ohio), Europe (Ireland), and Asia Pacific (Tokyo). The destination Regions for Global inference profile include all commercial AWS Regions. +## General considerations @@ -57,5 +50 @@ Note the following information about cross-Region inference: - * Global Cross-Region inference profiles provide higher throughput than an inference profile tied to a particular geography. An inference profile tied to a particular geography offers higher throughput than single-region inference. - - * To see the default quotas for cross-Region throughput when using inference profiles tied to a geography (such as US, EU and APAC), refer to the **Cross-region model inference requests per minute for ${Model}** and **Cross-region model inference tokens per minute for ${Model}** values in [Amazon Bedrock service quotas](https://docs.aws.amazon.com/general/latest/gr/bedrock.html#limits_bedrock) in the _AWS General Reference_. - - * To see the default quotas for cross-Region throughput when using Global inference profiles, refer to the **Global Cross-region model inference requests per minute for ${Model}** and **Global Cross-region model inference tokens per minute for ${Model}** values in [Amazon Bedrock service quotas](https://docs.aws.amazon.com/general/latest/gr/bedrock.html#limits_bedrock) in the _AWS General Reference_. + * Cross-Region inference can route requests to AWS Regions that are not manually enabled in your AWS account. Manual Region enablement is not required for cross-Region inference to function. @@ -63 +52 @@ Note the following information about cross-Region inference: -You can request, view, and manage quotas for the Global Cross-Region Inference Profile from the [Service Quotas console](https://console.aws.amazon.com/servicequotas/home/services/bedrock/quotas?region=us-east-1) or by using AWS CLI commands in the **US East (N. Virginia) Region**. Note that Global Cross-Region inference quotas will not appear in the Service Quotas console or AWS CLI for other source Regions listed in the Global Inference Profile. + * All data transmitted during cross-Region operations remains on the AWS network and does not traverse the public internet. Data is encrypted in transit between AWS Regions. @@ -65 +54 @@ You can request, view, and manage quotas for the Global Cross-Region Inference P - * Cross-Region inference requests to an inference profile tied to a geography (e.g. US, EU and APAC) are kept within the AWS Regions that are part of the geography where the data originally resides. For example, a request made within the US is kept within the AWS Regions in the US. Although the data remains stored only in the source Region, your input prompts and output results might move outside of your source Region during cross-Region inference. All data will be transmitted encrypted across Amazon’s secure network. + * All cross-Region inference requests are logged in CloudTrail in your source Region. Look for the `additionalEventData.inferenceRegion` field to identify where requests were processed. @@ -72,29 +60,0 @@ You can request, view, and manage quotas for the Global Cross-Region Inference P -## Use a cross-Region (system-defined) inference profile - -To use cross-Region inference, you include an [inference profile](./inference-profiles.html) when running model inference in the following ways: - - * **On-demand model inference** – Specify the ID of the inference profile as the `modelId` when sending an [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html), [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html), [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html), or [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html) request. An inference profile defines one or more Regions to which it can route inference requests originating from your source Region. Use of cross-Region inference increases throughput and performance by dynamically routing model invocation requests across the Regions defined in inference profile. Routing factors in user traffic, demand and utilization of resources. For more information, see [Submit prompts and generate responses with model inference](./inference.html) - - * **Batch inference** – Submit requests asynchronously with batch inference by specifying the ID of the inference profile as the `modelId` when sending a [CreateModelInvocationJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelInvocationJob.html) request. Using an inference profile lets you utilize compute across multiple AWS Regions and achieve faster processing times for your batch jobs. After the job is complete, you can retrieve the output files from the Amazon S3 bucket in the source Region. - - * **Agents** – Specify the ID of the inference profile in the `foundationModel` field in a [CreateAgent](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgent.html) request. For more information, see [Create and configure agent manually](./agents-create.html). - - * **Knowledge base response generation** – You can use cross-Region inference when generating a response after querying a knowledge base. For more information, see [Test your knowledge base with queries and responses](./knowledge-base-test.html). - - * **Model evaluation** – You can submit an inference profile as a model to evaluate when submitting a model evaluation job. For more information, see [Evaluate the performance of Amazon Bedrock resources](./evaluation.html). - - * **Prompt management** – You can use cross-Region inference when generating a response for a prompt you created in Prompt management. For more information, see [Construct and store reusable prompts with Prompt management in Amazon Bedrock](./prompt-management.html) - - * **Prompt flows** – You can use cross-Region inference when generating a response for a prompt you define inline in a prompt node in a prompt flow. For more information, see [Build an end-to-end generative AI workflow with Amazon Bedrock Flows](./flows.html). - - - - -###### Note - -Global inference profile is supported for On-demand model inference, Batch inference, Agents, Model evaluation, Prompt management, and Prompt flows. - -To learn how to use an inference profile to send model invocation requests across Regions, see [Use an inference profile in model invocation](./inference-profiles-use.html). - -To learn more about cross-Region inference, see [Getting started with cross-Region inference in Amazon Bedrock](https://aws.amazon.com/blogs/machine-learning/getting-started-with-cross-region-inference-in-amazon-bedrock/). - @@ -109 +69 @@ Processing use cases -Provisioned Throughput: Increase model throughput +Geographic cross-Region inference