AWS bedrock documentation change
Summary
Restructured documentation with detailed steps for model distillation, added security/privacy clarifications, and updated invocation log filtering instructions
Security assessment
Added explicit statements about data isolation ('Only you can access the final distilled model') and clarified security controls for invocation logs (model matching requirements, metadata filtering). These document existing security features rather than addressing vulnerabilities.
Diff
diff --git a/bedrock/latest/userguide/model-distillation.md b/bedrock/latest/userguide/model-distillation.md index 0784381c7..3260fc233 100644 --- a//bedrock/latest/userguide/model-distillation.md +++ b//bedrock/latest/userguide/model-distillation.md @@ -9 +9,33 @@ How Amazon Bedrock Model Distillation works - _Model distillation_ is the process of transferring knowledge from a larger more intelligent model (known as teacher) to a smaller, faster, cost-efficient model (known as student). In this process, the student model becomes as performant as the teacher for a specific use case. Amazon Bedrock Model Distillation uses the latest data synthesis techniques to generate diverse, high-quality responses (known as synthetic data) from the teacher model, and fine-tunes the student model. + _Model distillation_ is the process of transferring knowledge from a larger more intelligent model (known as teacher) to a smaller, faster, cost-efficient model (known as student). In this process, the student model's performance improves for a specific use case. Amazon Bedrock Model Distillation uses the latest data synthesis techniques to generate diverse, high-quality responses (known as synthetic data) from the teacher model, and fine-tunes the student model. + +To use Amazon Bedrock Model Distillation, you do the following: + + 1. Choose a teacher model and a student model. For more information, see [Choose teacher and student models for distillation](./prequisites-model-distillation.html). + + 2. Prepare your training data for distillation. Your training data is a collection of prompts stored in `.jsonl` files. Amazon Bedrock uses the input data to generate responses from the teacher model and uses the responses to fine-tune the student model. + + * You can optimize the synthetic data generation process by formatting your input prompts for the use case that you want. For more information, see [Optimize your input prompts for synthetic data generation](./distillation-prepare-datasets.html#distillation-data-prep-prompt-optimization). + + * You can prepare labeled input data as prompt-response pairs. Amazon Bedrock can use these pairs as golden examples while generating responses from the teacher model. For more information, see [Option 1: Provide your own prompts for data preparation](./distillation-data-prep-option-1.html). + + * If you enable CloudWatch Logs invocation logging, you can use existing teacher responses from invocation logs stored in Amazon S3 as training data. An invocation log in Amazon Bedrock is a detailed record of model invocations. For more information, see [Option 2: Use invocation logs for data preparation](./distillation-data-prep-option-2.html). + + 3. Create a Distillation job. This job creates a smaller, faster, and more cost-effective model for your use case. Only you can access the final distilled model. Amazon Bedrock doesn't use your data to train any other teacher or student model for public use. For more information, see [Submit a model distillation job in Amazon Bedrock](./submit-model-distillation-job.html). When your Distillation job completes, you can analyze the results of the customization process. For more information see [Analyze the results of a model customization job](./model-customization-analyze.html). + + + + +###### Topics + + * How Amazon Bedrock Model Distillation works + + * [Access and security for Model Distillation](./model-distillation-access-security.html) + + * [Choose teacher and student models for distillation](./prequisites-model-distillation.html) + + * [Prepare your training datasets for distillation](./distillation-prepare-datasets.html) + + * [Submit a model distillation job in Amazon Bedrock](./submit-model-distillation-job.html) + + * [Clone a distillation job](./clone-model-distillation-job.html) + @@ -11 +42,0 @@ How Amazon Bedrock Model Distillation works -To use Amazon Bedrock Model Distillation, you select a teacher model whose accuracy you want to achieve for your use case, and a student model to fine-tune. Then, you provide use case-specific prompts as input data. Amazon Bedrock generates responses from the teacher model for the given prompts, and then uses the responses to fine-tune the student model. You can optionally provide labeled input data as prompt-response pairs. Amazon Bedrock may use these pairs as golden examples while generating responses from the teacher model. Or, if you already have responses that the teacher model generated and you've stored them in the invocation logs, then you can use those existing teacher responses to fine-tune the student model. For this, you must provide Amazon Bedrock access to your invocation logs. An invocation log in Amazon Bedrock is a detailed record of model invocations. For more information, see [Monitor model invocation using CloudWatch Logs](https://docs.aws.amazon.com/bedrock/latest/userguide/model-invocation-logging.html). @@ -13 +43,0 @@ To use Amazon Bedrock Model Distillation, you select a teacher model whose accur -Only you can access the final distilled model. Amazon Bedrock doesn't use your data to train any other teacher or student model for public use. @@ -21 +51 @@ After you've identified your teacher and student models, you can choose how you -### Creating a distilled model using prompts that you provide +###### Note @@ -23 +53 @@ After you've identified your teacher and student models, you can choose how you -Amazon Bedrock uses the input prompts that you provide to generate responses from the teacher model. Amazon Bedrock then uses the responses to fine-tune the student model that you've identified. Depending on your use case, Amazon Bedrock might add proprietary data synthesis techniques to generate diverse and higher-quality responses. For example, Amazon Bedrock might generate similar prompts to generate more diverse responses from the teacher model. Or, if you optionally provide a handful of labeled input data as prompt-response pairs, then Amazon Bedrock might use these pairs as golden examples to instruct the teacher to generate similar high-quality responses. +If Amazon Bedrock Model Distillation uses its proprietary data synthesis techniques to generate higher-quality teacher responses, then your AWS account will incur additional charges for inference calls to the teacher model. These charges will be billed at the on-demand inference rates of the teacher model. Data synthesis techniques may increase the size of the fine-tuning dataset to a maximum of 15k prompt-response pairs. For more information about Amazon Bedrock charges, see [Amazon Bedrock Pricing](https://aws.amazon.com/bedrock/pricing/). @@ -25 +55 @@ Amazon Bedrock uses the input prompts that you provide to generate responses fro -###### Note +### Creating a distilled model using prompts that you provide @@ -27 +57 @@ Amazon Bedrock uses the input prompts that you provide to generate responses fro -If Amazon Bedrock Model Distillation uses its proprietary data synthesis techniques to generate higher-quality teacher responses, then your AWS account will incur additional charges for inference calls to the teacher model. These charges will be billed at the on-demand inference rates of the teacher model. Data synthesis techniques might increase the size of the fine-tuning dataset to a maximum of 15k prompt-response pairs. For more information about Amazon Bedrock charges, see [Amazon Bedrock Pricing](https://aws.amazon.com/bedrock/pricing/). +Amazon Bedrock uses the input prompts that you provide to generate responses from the teacher model. Amazon Bedrock then uses the responses to fine-tune the student model that you've identified. Depending on your use case, Amazon Bedrock might add proprietary data synthesis techniques to generate diverse and higher-quality responses. For example, Amazon Bedrock might generate similar prompts to generate more diverse responses from the teacher model. Or, if you optionally provide a handful of labeled input data as prompt-response pairs, then Amazon Bedrock might use these pairs as golden examples to instruct the teacher to generate similar high-quality responses. @@ -33 +63,3 @@ If you already have responses generated by the teacher model and stored them in -If you choose this option, then you can continue to use Amazon Bedrocks inference API operations, such as [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) API, and collect the invocation logs, model input data (prompts), and model output data (responses) for all invocations used in Amazon Bedrock. When you generate responses from the model using the `InvokeModel` or `Converse` API operations, you can optionally add `requestMetadata` to the responses. This can help you filter your invocation logs for specific use cases, and then use the filtered responses to fine-tune your student model. When you choose to use invocation logs to fine-tune your student model, you can have Amazon Bedrock use the prompts only, or use prompt-response pairs. +If you choose this option, then you can continue to use Amazon Bedrocks inference API operations, such as [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) API, and collect the invocation logs, model input data (prompts), and model output data (responses) for all invocations used in Amazon Bedrock. + +When you generate responses from the model using the `InvokeModel` or `Converse` API operations, you can optionally add `requestMetadata` to the responses. When you create a Distillation job, you can filter by this metadata as part of the invocation logs configuration. You can filter by your specific use cases, and then Amazon Bedrock only uses the filtered responses to fine-tune your student model. When you choose to use invocation logs to fine-tune your student model, you can have Amazon Bedrock use the prompts only, or use prompt-response pairs. @@ -39,4 +70,0 @@ If you choose to have Amazon Bedrock use only the prompts from the invocation lo -###### Note - -If Amazon Bedrock Model Distillation uses its proprietary data synthesis techniques to generate higher-quality teacher responses, then your AWS account will incur additional charges for inference calls to the teacher model. These charges will be billed at the on-demand inference rates of the teacher model. Data synthesis techniques may increase the size of the fine-tuning dataset to a maximum of 15k prompt-response pairs. For more information about Amazon Bedrock charges, see [Amazon Bedrock Pricing](https://aws.amazon.com/bedrock/pricing/). - @@ -45 +73 @@ If Amazon Bedrock Model Distillation uses its proprietary data synthesis techniq -If you choose to have Amazon Bedrock use prompt-response pairs from the invocation logs, then Amazon Bedrock won't re-generate responses from the teacher model and use the responses from the invocation log to fine-tune the student model. For Amazon Bedrock to read the responses from the invocation logs, the teacher model specified in your model distillation job must match the model used in the invocation log. If you've added request metadata to the responses in the invocation log, then to fine-tune the student model, you can specify the request metadata filters so that Amazon Bedrock reads only specific logs that are valid for your use case. +If you choose to have Amazon Bedrock use prompt-response pairs from the invocation logs, then Amazon Bedrock won't re-generate responses from the teacher model and use the responses from the invocation log to fine-tune the student model. For Amazon Bedrock to read the responses from the invocation logs, the teacher model specified in your model distillation job must match the model used in the invocation log. If they don't match, the invocation logs aren't used. If you've added request metadata to the responses in the invocation log, then to fine-tune the student model, you can specify the request metadata filters so that Amazon Bedrock reads only specific logs that are valid for your use case. @@ -55 +83 @@ Model customization access and security -Supported models and Regions for distillation +Access and security for Model Distillation