AWS bedrock documentation change
Summary
Reorganized and updated content about model customization, replacing 'Continued Pre-training' with 'Distillation' as the first method and adding more details about the distillation process. Moved 'Continued Pre-training' section lower in the document.
Security assessment
The changes are focused on reorganizing and clarifying model customization techniques, with no mention of security vulnerabilities or security-specific features.
Diff
diff --git a/bedrock/latest/userguide/custom-models.md index 8baf8c81c..bb761f662 100644 --- a/bedrock/latest/userguide/custom-models.md +++ b/bedrock/latest/userguide/custom-models.md @@ -9 +9 @@ Model customization is the process of providing training data to a model in orde - * **Continued Pre-training** + * **Distillation** @@ -11 +11 @@ Model customization is the process of providing training data to a model in orde -Provide _unlabeled_ data to pre-train a foundation model by familiarizing it with certain types of inputs. You can provide data from specific topics in order to expose a model to those areas. The Continued Pre-training process will tweak the model parameters to accommodate the input data and improve its domain knowledge. +Use distillation to transfer knowledge from a larger more intelligent model (known as teacher) to a smaller, faster, and cost-efficient model (known as student). Amazon Bedrock automates the distillation process by using the latest data synthesis techniques to generate diverse, high-quality responses from the teacher model, and fine-tunes the student model. @@ -13 +13,3 @@ Provide _unlabeled_ data to pre-train a foundation model by familiarizing it wit -For example, you can train a model with private data, such as business documents, that are not publicly available for training large language models. Additionally, you can continue to improve the model by retraining the model with more unlabeled data as it becomes available. +To use distillation, you select a teacher model whose accuracy you want to achieve for your use case, and a student model to fine-tune. Then, you provide use case-specific prompts as input data. Amazon Bedrock generates responses from the teacher model for the given prompts, and then uses the responses to fine-tune the student model. You can optionally provide labeled input data as prompt-response pairs. + +For more information about using distillation see [Model distillation in Amazon Bedrock](./model-distillation.html). @@ -18,0 +21,6 @@ Provide _labeled_ data in order to train a model to improve performance on speci + * **Continued Pre-training** + +Provide _unlabeled_ data to pre-train a foundation model by familiarizing it with certain types of inputs. You can provide data from specific topics in order to expose a model to those areas. The Continued Pre-training process will tweak the model parameters to accommodate the input data and improve its domain knowledge. + +For example, you can train a model with private data, such as business documents, that are not publicly available for training large language models. Additionally, you can continue to improve the model by retraining the model with more unlabeled data as it becomes available. +