AWS sagemaker medium security documentation change
Summary
Restructured and expanded documentation for supervised fine-tuning (SFT) and parameter-efficient fine-tuning (PEFT), adding detailed configuration parameters, data requirements, example recipes, and limitations
Security assessment
Added explicit requirement that training/validation datasets must reside in customer-owned S3 buckets rather than escrow/service-managed buckets. This security control ensures data remains under customer control and prevents accidental storage in less secure service-managed locations.
Diff
diff --git a/sagemaker/latest/dg/nova-fine-tune.md b/sagemaker/latest/dg/nova-fine-tune.md index cf9929b90..b012c2db8 100644 --- a//sagemaker/latest/dg/nova-fine-tune.md +++ b//sagemaker/latest/dg/nova-fine-tune.md @@ -5 +5 @@ -Supervised Fine-Tuning (SFT)Parameter-Efficient Fine-Tuning (PEFT) +Full-rank supervised fine-tuning (SFT)Parameter-efficient fine-tuning (PEFT) @@ -7 +7 @@ Supervised Fine-Tuning (SFT)Parameter-Efficient Fine-Tuning (PEFT) -# Supervised Fine-Tuning (Full FT, PEFT) +# Supervised fine-tuning (SFT) @@ -9 +9 @@ Supervised Fine-Tuning (SFT)Parameter-Efficient Fine-Tuning (PEFT) -Fine-tuning is the process of adapting a pre-trained language model to specific tasks or domains by training it on targeted datasets. Unlike pre-training, which builds general language understanding, fine-tuning optimizes the model for particular applications. +The following sections outline the process of supervised fine-tuning (SFT), both full-rank SFT and parameter-efficient SFT. @@ -11 +11 @@ Fine-tuning is the process of adapting a pre-trained language model to specific -###### Here's an overview of key fine-tuning techniques: +## Full-rank supervised fine-tuning (SFT) @@ -13 +13 @@ Fine-tuning is the process of adapting a pre-trained language model to specific - * Supervised Fine-Tuning (SFT) +Supervised fine-tuning (SFT) is the process of providing a collection of prompt-response pairs to a foundation model to improve the performance of a pre-trained foundation model on a specific task. The labeled examples are formatted as prompt-response pairs and phrased as instructions. This fine-tuning process modifies the weights of the model. @@ -15 +15 @@ Fine-tuning is the process of adapting a pre-trained language model to specific - * Parameter-Efficient Fine-Tuning (PEFT) +You should use SFT when you have domain-specific data that requires providing specific prompt-response pairs for optimal results. @@ -16,0 +17 @@ Fine-tuning is the process of adapting a pre-trained language model to specific +Note that your training and validation input datasets must reside in customer-owned buckets, not in escrow, or service-managed S3 buckets. @@ -17,0 +19 @@ Fine-tuning is the process of adapting a pre-trained language model to specific +###### Data requirements @@ -18,0 +21 @@ Fine-tuning is the process of adapting a pre-trained language model to specific +For full-rank SFT and low-rank adapter (LoRA) SFT, the data should follow the [Amazon Bedrock Converse operation format](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html). For examples and constraints of this format, see [Preparing data for fine-tuning Understanding models](https://docs.aws.amazon.com/nova/latest/userguide/fine-tune-prepare-data-understanding.html). @@ -20 +23 @@ Fine-tuning is the process of adapting a pre-trained language model to specific -## Supervised Fine-Tuning (SFT) +To validate your dataset format before submission, we recommend using [the validation script from the Amazon Bedrock samples repository](https://github.com/aws-samples/amazon-nova-samples/blob/main/customization/bedrock-finetuning/understanding/dataset_validation/nova_ft_dataset_validator.py). This validation tool helps ensure that your JSONL files adhere to the required format specifications and identify any potential issues before you submit your fine-tuning job. @@ -22 +25 @@ Fine-tuning is the process of adapting a pre-trained language model to specific -**Supervised Fine-Tuning (SFT)** +The Amazon Nova parameters that are available for tuning with SFT are as follows: @@ -24 +27 @@ Fine-tuning is the process of adapting a pre-trained language model to specific -SFT adapts a pre-trained model using labeled examples of desired inputs and outputs. The model learns to generate responses that match the provided examples,effectively teaching it to follow specific instructions or produce outputs in a particular style. SFT typically involves updating all model parameters based on task-specific data. + * **Run configuration** @@ -26 +29 @@ SFT adapts a pre-trained model using labeled examples of desired inputs and outp -For detailed instructions about using SFT with Amazon Nova model customization, see the [Supervised fine-tuning (SFT)](https://docs.aws.amazon.com/nova/latest/userguide/customize-fine-tune-hyperpod-sft.html) section from _Amazon Nova user guide_. + * `name`: A descriptive name for your training job. This helps identify your job in the AWS Management Console. @@ -28 +31,345 @@ For detailed instructions about using SFT with Amazon Nova model customization, -## Parameter-Efficient Fine-Tuning (PEFT) + * `model_type`: The Amazon Nova model variant to use. The available options are `amazon.nova-micro-v1:0:128k`, `amazon.nova-lite-v1:0:300k`, or `amazon.nova-pro-v1:0:300k`. + + * `model_name_or_path`: The path to the base model to use for your training. Select the model to use from `nova-micro/prod`, `nova-lite/prod`, `nova-pro/prod`, or the S3 path for the post-training checkpoint (`s3://<escrow bucket>/<job id>/outputs/checkpoints`). + + * `replicas`: The number of compute instances to use for distributed training. Available values vary based on the model chosen. Amazon Nova Micro supports 2, 4, or 8 replicas. Amazon Nova Lite supports 4, 8, 16, or 32 replicas. Amazon Nova Pro supports 6, 12, or 24 replicas. + + * `data_s3_path`: The S3 location of the training dataset, which is a JSONL file. This file must reside in the same AWS account and Region as the cluster. All of the S3 locations within the provided S3 path must be in the same account and Region. + + * `validation_data_s3_path`: (Optional) The S3 location of the validation dataset, which is a JSONL file. This file must reside in the same account and Region as the cluster. All of the S3 locations within the provided S3 path must be in the same account and Region. + + * `output_s3_path`: The S3 location where the manifest and TensorBoard logs are stored. All of the S3 locations within the provided S3 path must be in the same account and region. + + * **Training configuration** + + * `max_length`: The maximum sequence length in tokens. This determines the context window size for training. The maximum supported value are 65,536 tokens for SFT. + +Longer sequences will improve training efficiencies at the cost of increased memory requirements. We recommend that you match the `max_length` parameter to your data distribution. + + * **Trainer settings** + + * `max_epochs`: The number of complete passes through your training dataset. The maximum value is 5. + +In general, larger datasets require fewer epochs to converge, while smaller datasets require more epochs to converge. We recommend that you adjust the number of epochs based on the size of your data. + + * **Model settings** + + * `hidden_dropout`: The probability of dropping hidden state outputs. Increase this value by approximately 0.0-0.2 to reduce over-fitting on smaller datasets. Valid values are between 0-1, inclusive. + + * `attention_dropout`: The probability of dropping attention weights. This parameter can help with generalization. Valid values are between 0-1, inclusive. + + * `ffn_dropout`: The probability of dropping feed-forward network outputs. Valid values are between 0-1, inclusive. + + * **Optimizer configuration** + + * `lr`: The learning rate, which controls the step size during optimization. Valid values are between 1e-6-1e-3, inclusive. We recommend values between 1e-6-1e-4 for good performance. + + * `name`: The optimizer algorithm. Currently, only `distributed_fused_adam` is supported. + + * `weight_decay`: The L2 regularization strength. Higher values (between 0.01-0.1) increase regularization. + + * `warmup_steps`: The number of steps to gradually increase learning rate. This improves training stability. Valid values are between 1-20, inclusive. + + * `min_lr`: The minimum learning rate at the end of decay. Valid values are between 0-1, inclusive, but must be less than learning rate. + + + + +###### Quick start with a full-rank SFT recipe + +The following is a recipe for full-rank SFT that's intended for you to quickly start an SFT job on a HyperPod cluster. This recipe also assumes that you have connected to your HyperPod cluster using the correct AWS credentials. + + + run: + name: "my-sft-micro-job" # gets appended with a unique ID for HP jobs + model_type: "amazon.nova-micro-v1:0:128k" + model_name_or_path: "nova-micro/prod" + replicas: 2 + data_s3_path: s3:Replace with your S3 bucket name/input.jsonl + validation_data_s3_path: [OPTIONAL] s3:your S3 bucket name/input.jsonl + output_s3_path: [S3_PATH_TO_STORE_MANIFEST] + + ## training specific configs + training_config: + max_length: 32768 + save_steps: 100000 + replicas: ${recipes.run.replicas} + micro_batch_size: 1 + task_type: sft + global_batch_size: 64 + weights_only: True + allow_percentage_invalid_samples: 10 + + exp_manager: + exp_dir: null + create_wandb_logger: False + create_tensorboard_logger: True + project: null + name: null + checkpoint_callback_params: + monitor: step + save_top_k: 10 + mode: max + every_n_train_steps: ${recipes.training_config.save_steps} + save_last: True + create_early_stopping_callback: True + early_stopping_callback_params: + min_delta: 0.001 + mode: min + monitor: "val_loss" + patience: 2 + + trainer: + log_every_n_steps: 1 + max_epochs: -1 + max_steps: 16 + val_check_interval: 100 + limit_test_batches: 0 + gradient_clip_val: 1.0 + num_nodes: ${recipes.training_config.replicas} + + model: + hidden_dropout: 0.0 # Dropout probability for hidden state transformer. + attention_dropout: 0.0 # Dropout probability in the attention layer. + ffn_dropout: 0.0 # Dropout probability in the feed-forward layer. + sequence_parallel: True + optim: + lr: 1e-5 + name: distributed_fused_adam + bucket_cap_mb: 10 + contiguous_grad_buffer: False + overlap_param_sync: False + contiguous_param_buffer: False + overlap_grad_sync: False + adam_w_mode: true + eps: 1e-06 + weight_decay: 0.0 + betas: + - 0.9 + - 0.999 + sched: + name: CosineAnnealing + warmup_steps: 10 + constant_steps: 0 + min_lr: 1e-6 + + mm_cfg: + llm: + freeze: false + image_projector: + freeze: true + require_newline: true + video_projector: + freeze: true + require_newline: false + + peft: + peft_scheme: null + + training_validation: + loader: + args: + data_loader_workers: 1 + prefetch_factor: 2 + collator: + args: + force_image_at_turn_beginning: false + +###### Sample full-rank recipe + +The following is a sample full-rank recipe for SFT with all components properly configured. + + + ## Run config + run: + name: "my-sft-run" # A descriptive name for your training job + model_type: "amazon.nova-lite-v1:0:300k" # Model variant specification + model_name_or_path: "nova-lite/prod" # Base model path + replicas: 4 # Number of compute instances for training