AWS bedrock documentation change
Summary
Restructured documentation to add detailed model support matrix, cross-region inference requirements, and provisioned throughput notes. Added specific model IDs, regions, and provider pairings for distillation.
Security assessment
Added documentation about required service role permissions for cross-region inference profiles and model access restrictions. While these relate to security configuration, there's no evidence of addressing a specific vulnerability. The changes provide operational security guidance rather than patching a security issue.
Diff
diff --git a/bedrock/latest/userguide/prequisites-model-distillation.md b/bedrock/latest/userguide/prequisites-model-distillation.md index b9aef1358..0f2814215 100644 --- a//bedrock/latest/userguide/prequisites-model-distillation.md +++ b//bedrock/latest/userguide/prequisites-model-distillation.md @@ -5 +5,15 @@ -# Decide on your teacher and student models for distillation +Supported models and Regions for distillation + +# Choose teacher and student models for distillation + +For Model Distillation, you choose a teacher and student model. + + * **Choose a teacher model** + +Choose a teacher model that's significantly larger and more capable than the student model, and whose accuracy you want to achieve for your use case. To make distillation more effective, choose a model that's already trained on tasks similar to your use case. + +For some teacher models, you can choose a Cross-Region inference profile ([Increase throughput with cross-Region inference](./cross-region-inference.html)). Cross-Region inference automatically selects the optimal AWS Region within your geography to process your inference request. This improves customer experience by maximizing available resources and model availability. To use a Cross-Region inference profile, your service role must have permissions to invoke the inference profile in an AWS Region, in addition to the model in each Region in the inference profile. For a policy example, see [(Optional) Permissions to create a Distillation job with a cross-region inference profile](./custom-model-job-access-security.html#custom-models-cross-region-inference-profile-permissions). + + * **Choose a student model** + +Choose a student model that's significantly smaller in size than the teacher model. The student model must be one of the student models paired with your teacher model in the following table. @@ -7 +20,0 @@ -For model distillation, you must decide on a teacher and related student model. @@ -9 +21,0 @@ For model distillation, you must decide on a teacher and related student model. - 1. **Decide on a teacher model** @@ -11 +22,0 @@ For model distillation, you must decide on a teacher and related student model. -Choose a teacher model that is significantly larger and more capable than the student model, and whose accuracy you want to achieve for your use case. To make the distillation job more effective, select a model that is already trained on task similar to your use case. For information on the teacher models supported by Amazon Bedrock see [Supported models and Regions for Amazon Bedrock Model Distillation](./model-distillation-supported.html). @@ -13 +24 @@ Choose a teacher model that is significantly larger and more capable than the st - 2. **Decide on a student model** +The following section lists the supported models and regions for Amazon Bedrock Model Distillation. After you choose your teacher and student models, you prepare and optimize your training datasets for distillation. For more information, see [Prepare your training datasets for distillation](./distillation-prepare-datasets.html). @@ -15 +26,21 @@ Choose a teacher model that is significantly larger and more capable than the st -Choose a student model that is significantly smaller in size than the teacher model. For information on the student models that Amazon Bedrock supports, see [Supported models and Regions for Amazon Bedrock Model Distillation](./model-distillation-supported.html). +## Supported models and Regions for Amazon Bedrock Model Distillation + +The following table shows which models and AWS Regions Amazon Bedrock Model Distillation supports for teacher and student models. If you use a Cross Region Inference Profile, only System Inference Profiles are supported for model distillation. For more information, see [Increase throughput with cross-Region inference](./cross-region-inference.html). + +Provider | Teacher | Teacher ID | Inference profile support | Student | Student ID | Region +---|---|---|---|---|---|--- +Amazon | Nova Pro | amazon.nova-pro-v1:0 | Both | Nova LiteNova Micro | amazon.nova-lite-v1:0:300kamazon.nova-micro-v1:0:128k | US East (N. Virginia) +Nova Premier | amazon.nova-premier-v1:0 | Inference profile only | Nova LiteNova MicroNova Pro | amazon.nova-lite-v1:0:300kamazon.nova-micro-v1:0:128kamazon.nova-pro-v1:0:300k | US East (N. Virginia) +Anthropic | Claude 3.5 v1 | anthropic.claude-3-5-sonnet-20240620-v1:0 | Both | Claude 3 Haiku | anthropic.claude-3-haiku-20240307-v1:0:200k | US West (Oregon) +Claude 3.5 v2 | anthropic.claude-3-5-sonnet-20241022-v2:0 | Both | Claude 3 Haiku | anthropic.claude-3-haiku-20240307-v1:0:200k | US West (Oregon) +Meta | Llama 3.1 405B | meta.llama3-1-405b-instruct-v1:0 | On demand | Llama 3.1 8BLlama 3.1 70BLlama 3.2 1B | meta.llama3-1-8b-instruct-v1:0:128kmeta.llama3-1-70b-instruct-v1:0:128kmeta.llama3-2-1b-instruct-v1:0:128k | US West (Oregon) +Llama 3.1 70B | meta.llama3-1-70b-instruct-v1:0 | Both | Llama 3.1 8BLlama 3.2 1BLlama 3.2 3B | meta.llama3-1-8b-instruct-v1:0:128kmeta.llama3-2-1b-instruct-v1:0:128kmeta.llama3-2-3b-instruct-v1:0:128k | US West (Oregon) +Llama 3.3 70B | meta.llama3-3-70b-instruct-v1:0 | Inference profile only | Llama 3.1 8BLlama 3.2 1BLlama 3.2 3B | meta.llama3-1-8b-instruct-v1:0:128kmeta.llama3-2-1b-instruct-v1:0:128kmeta.llama3-2-3b-instruct-v1:0:128k | US West (Oregon) + +###### Note + + * You must buy [provisioned throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html) to be able to perofrm inference with the distilled model. + + * For Claude and Llama models, the distillation job is run in US West (Oregon). You can either buy [provisioned throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html) in US West (Oregon) or [copy distilled model](https://docs.aws.amazon.com/bedrock/latest/userguide/copy-model.html) to another Region and then buy [provisioned throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html). + + * For Nova models, you run distillation job in US East (N. Virginia). For inference, you need to buy [provisioned throughput](https://docs.aws.amazon.com/bedrock/latest/userguide/prov-throughput.html) in US East (N. Virginia). You can't copy Nova models to other Regions. @@ -26 +57 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Supported models and Regions for distillation +Access and security for Model Distillation