AWS bedrock documentation change
Summary
Added documentation for enhanced OpenAI-compatible API formats, new examples, and critical configuration requirements for GPT-OSS models
Security assessment
The changes primarily document new API format support and configuration requirements. While the update emphasizes proper generation_config.json usage to prevent runtime errors, this is a reliability concern rather than a security vulnerability. No security vulnerabilities, access controls, or encryption mechanisms are mentioned.
Diff
diff --git a/bedrock/latest/userguide/invoke-imported-model.md b/bedrock/latest/userguide/invoke-imported-model.md index 306e4b677..bbd8da11a 100644 --- a//bedrock/latest/userguide/invoke-imported-model.md +++ b//bedrock/latest/userguide/invoke-imported-model.md @@ -5 +5 @@ -Handling ModelNotReadyException +Enhanced API Support: Multiple API FormatsFrequently Asked QuestionsHandling ModelNotReadyException @@ -9 +9 @@ Handling ModelNotReadyException -The model import job can take several minutes to import your model after you send [CreateModelImportJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelImportJob.html) request. You can check the status of your import job in the console or by calling the [GetModelImportJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetModelImportJob.html) operation and checking the `Status` field in the response. The import job is complete if the Status for the model is **Complete**. +The model import job can take several minutes to import your model after you send [CreateModelImportJob](https://docs.aws.amazon.com//bedrock/latest/APIReference/API_CreateModelImportJob.html) request. You can check the status of your import job in the console or by calling the [GetModelImportJob](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetModelImportJob.html) operation and checking the `Status` field in the response. The import job is complete if the Status for the model is **Complete**. @@ -17 +17 @@ To interface with your imported model using the messages format, you can call th -Converse API is not supported for Qwen2.5, Qwen2-VL, and Qwen2.5-VL models. +Converse API is not supported for Qwen2.5, Qwen2-VL, Qwen2.5-VL, and GPT-OSS models. @@ -19 +19 @@ Converse API is not supported for Qwen2.5, Qwen2-VL, and Qwen2.5-VL models. -You'll need the model ARN to make inference calls to your newly imported model. After the successful completion of the import job and after your imported model is active, you can get the model ARN of your imported model in the console or by sending a [ListImportedModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListImportedModels.html) request. +## Enhanced API Support: Multiple API Formats @@ -21 +21 @@ You'll need the model ARN to make inference calls to your newly imported model. -To invoke your imported model, make sure to use the same inference parameters that is mentioned for the customized foundation model you are importing. For information on the inference parameters to use for the model you are importing, see [Inference request parameters and response fields for foundation models](./model-parameters.html). If you are using inference parameters that do not match with the inference parameters mentioned for that model, those parameters will be ignored. +Starting November 17, 2025, Amazon Bedrock Custom Model Import supports comprehensive OpenAI-compatible API formats, providing flexibility in how you integrate and deploy your custom models. All models imported after November 11, 2025, will automatically benefit from these enhanced capabilities with no additional configuration required. @@ -23 +23,36 @@ To invoke your imported model, make sure to use the same inference parameters th -###### Note +Custom Model Import now supports three API formats: + + * **BedrockCompletion (Text)** \- Compatible with current Bedrock workflows + + * **OpenAICompletion (Text)** \- OpenAI Completions Schema compatibility + + * **OpenAIChatCompletion (Text and Images)** \- Full conversational Schema compatibility + + + + +These enhanced capabilities include structured outputs for enforcing JSON schemas and patterns, enhanced vision support with multi-image processing, log probabilities for model confidence insights, and tool calling capabilities for GPT-OSS models. + +For detailed API reference documentation, see the official OpenAI documentation: + + * Completion: [OpenAI Completions API](https://platform.openai.com/docs/api-reference/completions) + + * ChatCompletion: [OpenAI Chat API](https://platform.openai.com/docs/api-reference/chat) + + + + +### API Format Examples + +The following examples demonstrate how to use each of the four supported API formats with your imported models. + +BedrockCompletion + + +**BedrockCompletion** format is compatible with current Bedrock workflows and supports text-based inference requests. + +Example request: + + + import json + import boto3 @@ -25 +60 @@ To invoke your imported model, make sure to use the same inference parameters th -When providing multi modal inputs, you will need to include the appropriate placeholders for multi modal tokens in your text prompt. For example, when sending an image input to a Qwen-VL model, the prompt should include `<|vision_start|><|image_pad|><|vision_end|>`. These notations are specific to the model’s tokenizer and can be applied using the following chat template. + client = boto3.client('bedrock-runtime', region_name='us-east-1') @@ -26,0 +62,5 @@ When providing multi modal inputs, you will need to include the appropriate plac + payload = { + "prompt": "How is the rainbow formed?", + "max_gen_len": 100, + "temperature": 0.5 + } @@ -28 +68,8 @@ When providing multi modal inputs, you will need to include the appropriate plac - from transformers import AutoProcessor, AutoTokenizer + response = client.invoke_model( + modelId='your-model-arn', + body=json.dumps(payload), + accept='application/json', + contentType='application/json' + ) + + response_body = json.loads(response['body'].read()) @@ -30,4 +77 @@ When providing multi modal inputs, you will need to include the appropriate plac - if vision_model: - processor = AutoProcessor.from_pretrained(model) - else: - processor = AutoTokenizer.from_pretrained(model) +Example response: @@ -36,2 +79,0 @@ When providing multi modal inputs, you will need to include the appropriate plac - # Create messages - messages = [ @@ -39,2 +81,40 @@ When providing multi modal inputs, you will need to include the appropriate plac - "role": "user", - "content": [ + "generation": " – A scientific explanation\nA rainbow is a beautiful natural phenomenon that occurs when sunlight passes through water droplets in the air. It is formed through a process called refraction, which is the bending of light as it passes from one medium to another.\nHere's a step-by-step explanation of how a rainbow is formed:\n1. Sunlight enters the Earth's atmosphere: The first step in forming a rainbow is for sunlight to enter the Earth's atmosphere. This sunlight is made up of a spectrum of", + "prompt_token_count": 7, + "generation_token_count": 100, + "stop_reason": "length", + "logprobs": null + } + +BedrockCompletion supports structured outputs using `response_format` parameter with `json_object` and `json_schema` types. + +OpenAICompletion + + +**OpenAICompletion** format provides OpenAI Completions Schema compatibility. To use this format, include the `max_tokens` parameter instead of `max_gen_len`. + +Example request: + + + import json + import boto3 + + client = boto3.client('bedrock-runtime', region_name='us-east-1') + + payload = { + "prompt": "How is the rainbow formed?", + "max_tokens": 100, + "temperature": 0.5 + } + + response = client.invoke_model( + modelId='your-model-arn', + body=json.dumps(payload), + accept='application/json', + contentType='application/json' + ) + + response_body = json.loads(response['body'].read()) + +Example response: + + @@ -42,3 +122,4 @@ When providing multi modal inputs, you will need to include the appropriate plac - "type": "image", - "image": "base64 encoded image", - }, + "id": "cmpl-b09d5810bd64428f8a853be71c31f912", + "object": "text_completion", + "created": 1763166682, + "choices": [ @@ -46,3 +127,4 @@ When providing multi modal inputs, you will need to include the appropriate plac - "type": "text", - "text": "Describe this image.", - }, + "index": 0, + "text": " The formation of a rainbow is a complex process that involves the interaction of sunlight with water droplets in the air. Here's a simplified explanation: 1. Sunlight enters the Earth's atmosphere and is refracted, or bent, as it passes through the air. 2. When sunlight encounters a water droplet, such as a cloud, mist, or fog, it is refracted again and split into its individual colors, a process known as dispersion. 3. The refracted and", + "finish_reason": "length" + } @@ -49,0 +132,4 @@ When providing multi modal inputs, you will need to include the appropriate plac + "usage": { + "prompt_tokens": 7, + "total_tokens": 107, + "completion_tokens": 100 @@ -51 +137,10 @@ When providing multi modal inputs, you will need to include the appropriate plac - ] + } + +OpenAICompletion supports full structured outputs capabilities including `json`, `regex`, `choice`, and `grammar` constraints using the `structured_outputs` parameter. + +OpenAIChatCompletion + + +**OpenAIChatCompletion** format provides full conversational Schema compatibility and supports both text and image inputs. + +Example request: @@ -53,8 +148,16 @@ When providing multi modal inputs, you will need to include the appropriate plac - # Apply chat template - prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) - """ - prompt = ''' - <|im_start|>system\nYou are a helpful assistant.<|im_end|>\n - <|im_start|>user\n<|vision_start|><|image_pad|><|vision_end|> - Describe this image.<|im_end|>\n<|im_start|>assistant\n''' - """ + + import json + import boto3 + + client = boto3.client('bedrock-runtime', region_name='us-east-1') + + payload = { + "messages": [ + { + "role": "user", + "content": "How is the rainbow formed?" + } + ], + "max_tokens": 100, + "temperature": 0.5 + } @@ -63,8 +166,2 @@ When providing multi modal inputs, you will need to include the appropriate plac - modelId=model_id, - body=json.dumps({ - 'prompt': prompt, - 'temperature': temperature, - 'max_gen_len': max_tokens, - 'top_p': top_p, - 'images': ["base64 encoded image"] - }), + modelId='your-model-arn', + body=json.dumps(payload), @@ -74,0 +172,33 @@ When providing multi modal inputs, you will need to include the appropriate plac + response_body = json.loads(response['body'].read()) +