AWS bedrock documentation change
Summary
Restructured documentation to focus exclusively on the Invoke API, removed references to other APIs (Converse, Messages, OpenAI), and added comprehensive code examples for various inference scenarios including text/image/video inputs, embeddings, and streaming responses.
Security assessment
The changes are purely documentation improvements showing API usage patterns. No security vulnerabilities, weaknesses, or incident responses are mentioned. Optional guardrail parameters are referenced but not explained as security features.
Diff
diff --git a/bedrock/latest/userguide/inference-api.md b/bedrock/latest/userguide/inference-api.md index a7a774963..59c3718bc 100644 --- a//bedrock/latest/userguide/inference-api.md +++ b//bedrock/latest/userguide/inference-api.md @@ -7 +7 @@ -# Submit prompts and generate responses using the API +Submit a single prompt @@ -9 +9 @@ -Amazon Bedrock offers the followingAPI operations for carrying out model inference: +# Inference using Invoke API @@ -11 +11 @@ Amazon Bedrock offers the followingAPI operations for carrying out model inferen - * [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) – Submit a prompt and generate a response. The request body is model-specific. To generate streaming responses, use [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html). +The Invoke API is available on the `bedrock-runtime` endpoint only. Amazon Bedrock offers the following API operations for carrying out model inference with the Invoke API: @@ -13 +13 @@ Amazon Bedrock offers the followingAPI operations for carrying out model inferen - * [Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) – Submit a prompt and generate responses with a structure unified across all models. Model-specific request fields can be specified in the `additionalModelRequestFields` field. You can also include system prompts and previous conversation for context. To generate streaming responses, use [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html). + * [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) – Submit a prompt and generate a response. The request body is model-specific. To generate streaming responses, use [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html). @@ -19,2 +18,0 @@ Amazon Bedrock offers the followingAPI operations for carrying out model inferen - * OpenAI Chat completions API – Use the [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create) with models supported by Amazon Bedrock to generate a response. - @@ -45 +43,438 @@ Custom model | A model whose parameters are shifted from a foundation model base -Select a topic to learn how to use the model invocation APIs. +## Submit a single prompt with InvokeModel + +You run inference on a single prompt by using the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) API operations and specifying a model. Amazon Bedrock models differ in whether they accept text, image, or video inputs and whether they can produce outputs of text, image, or embeddings. Some models can return the response in a stream. To check model support for input, output, and streaming support, do one of the following: + + * Check the value in the **Input modalities** , **Output modalities** , or **Streaming supported** columns for a model at [Supported foundation models in Amazon Bedrock](./models-supported.html). + + * Send a [GetFoundationModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetFoundationModel.html) request with the model ID and check the values in the `inputModalities`, `outputModalities`, and `responseStreamingSupported` field. + + + + +Run model inference on a prompt by sending an [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) request with an [Amazon Bedrock runtime endpoint](https://docs.aws.amazon.com/general/latest/gr/bedrock.html#br-rt). + +###### Note + +Restrictions apply to the following operations: `InvokeModel`, `InvokeModelWithResponseStream`, `Converse`, and `ConverseStream`. See [API restrictions](./inference-api-restrictions.html) for details. + +The following fields are required: + +Field | Use case +---|--- +modelId | To specify the model, inference profile, or prompt from Prompt management to use. To learn how to find this value, see [Inference using Invoke API](./inference-api.html). +body | To specify the inference parameters for a model. To see inference parameters for different models, see [Inference request parameters and response fields for foundation models](./model-parameters.html). If you specify a prompt from Prompt management in the `modelId` field, omit this field (if you include it, it will be ignored). + +The following fields are optional: + +Field | Use case +---|--- +accept | To specify the media type for the request body. For more information, see **Media Types** on the [Swagger website](https://swagger.io/specification/). +contentType | To specify the media type for the response body. For more information, see **Media Types** on the [Swagger website](https://swagger.io/specification/). +performanceConfigLatency | To specify whether to optimize a model for latency. For more information, see [Optimize model inference for latency](./latency-optimized-inference.html). +guardrailIdentifier | To specify a guardrail to apply to the prompt and response. For more information, see [Test your guardrail](./guardrails-test.html). +guardrailVersion | To specify a guardrail to apply to the prompt and response. For more information, see [Test your guardrail](./guardrails-test.html). +trace | To specify whether to return the trace for the guardrail you specify. For more information, see [Test your guardrail](./guardrails-test.html). +serviceTier | To specify the service tier for a request. For more information, see [Service tiers for optimizing performance and cost](./service-tiers-inference.html). + +### Invoke model code examples + +This topic provides some basic examples for running inference using a single prompt with the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) API. For more examples with different models, visit the following resources: + + * Pick an example under the [Code examples for Amazon Bedrock Runtime using AWS SDKs](./service_code_examples_bedrock-runtime.html) topic. + + * Visit the inference parameter reference for the desired model at [Inference request parameters and response fields for foundation models](./model-parameters.html). + + + + +The following examples assume that you've set up programmatic access such that you automatically authenticate to the AWS CLI and the SDK for Python (Boto3) in a default AWS Region when you run these examples. For information on setting up programmating access, see [Get started with the API](./getting-started-api.html). + +###### Note + +Review the following points before trying out the examples: + + * You should test these examples in US East (N. Virginia) (us-east-1), which supports all the models used in the examples. + + * The `body` parameter can be large, so for some CLI examples, you'll be asked to create a JSON file and provide that file into the `--body` argument instead of specifying it in the command line. + + * For the image and video examples, you'll be asked to use your own image and video. The examples assume that your image file is named `image.png` and that your video file is named `video.mp4`. + + * You might have to convert images or videos into a base64-encoded string or upload them to an Amazon S3 location. In the examples, you'll have to replace the placeholders with the actual base64-encoded string or S3 location. + + + + +#### Generate text with a text prompt + +The following examples generate a text response to a text prompt using the Amazon Titan Text Premier model. Choose the tab for your preferred method, and then follow the steps: + +CLI + + +Run the following command in a terminal and find the generated response in a file called `invoke-model-output.txt`. + + + aws bedrock-runtime invoke-model \ + --model-id amazon.titan-text-premier-v1:0 \ + --body '{ + "inputText": "Describe the purpose of a 'hello world' program in one line.", + "textGenerationConfig": { + "maxTokenCount": 512, + "temperature": 0.5 + } + }' \ + --cli-binary-format raw-in-base64-out \ + invoke-model-output.txt + +Python + + +Run the following Python code example to generate a text response: + + + # Use the native inference API to send a text message to Amazon Titan Text. + + import boto3 + import json + + from botocore.exceptions import ClientError + + # Create a Bedrock Runtime client in the AWS Region of your choice. + client = boto3.client("bedrock-runtime", region_name="us-east-1") + + # Set the model ID, e.g., Titan Text Premier. + model_id = "amazon.titan-text-premier-v1:0" + + # Define the prompt for the model. + prompt = "Describe the purpose of a 'hello world' program in one line." + + # Format the request payload using the model's native structure. + native_request = { + "inputText": prompt, + "textGenerationConfig": { + "maxTokenCount": 512, + "temperature": 0.5, + }, + } + + # Convert the native request to JSON. + request = json.dumps(native_request) + + try: + # Invoke the model with the request. + response = client.invoke_model(modelId=model_id, body=request) + + except (ClientError, Exception) as e: + print(f"ERROR: Can't invoke '{model_id}'. Reason: {e}") + exit(1) + + # Decode the response body. + model_response = json.loads(response["body"].read()) + + # Extract and print the response text. + response_text = model_response["results"][0]["outputText"] + print(response_text) + + + +#### Generate text with a text prompt using service tier + +The following examples generate a text response to a text prompt using the OpenAI GPT model with a service tier to prioritize the request. Choose the tab for your preferred method, and then follow the steps: + +CLI + + +Run the following command in a terminal and validate the service tier in the response. + + + aws bedrock-runtime invoke-model \ + --model-id openai.gpt-oss-120b-1:0 \ + --body '{ + "messages": [ + { + "role": "user", + "content": "Describe the purpose of a '\''hello world'\'' program in one line." + } + ], + "max_tokens": 512, + "temperature": 0.7 + }' \ + --content-type application/json \ + --accept application/json \ + --service-tier priority \ + --cli-binary-format raw-in-base64-out + +Python + + +Run the following Python code example to generate a text response with service tier: + + + import boto3 + import json + + # Create a Bedrock Runtime client + bedrock_runtime = boto3.client( + service_name="bedrock-runtime", + region_name="us-east-1" + ) +