AWS bedrock documentation change
Summary
Expanded documentation about tool calling mechanisms, adding detailed explanations of client-side vs server-side tool calling, code examples for multiple APIs, and new sections about Lambda integration and AWS-provided tools
Security assessment
The change adds documentation about server-side tool calling security features including IAM policy enforcement, VPC access controls, and compliance standards (ISO/SOC/HIPAA). While it describes security-related capabilities, there's no evidence of addressing a specific vulnerability.
Diff
diff --git a/bedrock/latest/userguide/tool-use.md b/bedrock/latest/userguide/tool-use.md index 488627185..4c790373d 100644 --- a//bedrock/latest/userguide/tool-use.md +++ b//bedrock/latest/userguide/tool-use.md @@ -9 +9 @@ You can use the Amazon Bedrock API to give a model access to tools that can help -###### Note +In Amazon Bedrock, the model doesn't directly call the tool. Rather, when you send a message to a model, you also supply a definition for one or more tools that could potentially help the model generate a response. In this example, you would supply a definition for a tool that returns the most popular song for a specified radio station. If the model determines that it needs the tool to generate a response for the message, the model, depending on the API used to invoke the model, can perform either client-side calling or ask Bedrock to call the tool using server-side tool calling. Let us discuss these two options in more detail. @@ -11 +11 @@ You can use the Amazon Bedrock API to give a model access to tools that can help -Tool use with models is also known as _Function calling_. +**Client-side tool calling** @@ -13 +13 @@ Tool use with models is also known as _Function calling_. -In Amazon Bedrock, the model doesn't directly call the tool. Rather, when you send a message to a model, you also supply a definition for one or more tools that could potentially help the model generate a response. In this example, you would supply a definition for a tool that returns the most popular song for a specified radio station. If the model determines that it needs the tool to generate a response for the message, the model responds with a request for you to call the tool. It also includes the input parameters (the required radio station) to pass to the tool. +If you use the Responses API, Chat Completions API, Converse API, or InvokeModel API to send the request, then the model uses client-side tool calling. This means that in your code, you call the tool on the model's behalf. In this scenario, assume the tool implementation is an API. The tool could just as easily be a database, Lambda function, or some other software. You decide how you want to implement the tool. You then continue the conversation with the model by supplying a message with the result from the tool. Finally, the model generates a response for the original message that includes the tool results that you sent to the model. @@ -15 +15 @@ In Amazon Bedrock, the model doesn't directly call the tool. Rather, when you se -In your code, you call the tool on the model's behalf. In this scenario, assume the tool implementation is an API. The tool could just as easily be a database, Lambda function, or some other software. You decide how you want to implement the tool. You then continue the conversation with the model by supplying a message with the result from the tool. Finally the model generates a response for the original message that includes the tool results that you sent to the model. +Let us define the tool we will use for tool use. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station. @@ -17 +16,0 @@ In your code, you call the tool on the model's behalf. In this scenario, assume -To use tools with a model you can use the Converse API ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) or [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). The example code in this topic uses the Converse API to show how to use a tool that gets the most popular song for a radio station. For general information about calling the Converse API, see [Carry out a conversation with the Converse API operations](./conversation-inference.html). @@ -19 +18,7 @@ To use tools with a model you can use the Converse API ([Converse](https://docs. -It is possible to use tools with the base inference operations ([InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html)). To find the inference parameters that you pass in the request body, see the [inference parameters](./model-parameters.html) for the model that you want to use. We recommend using the Converse API as it provides a consistent API, that works with all Amazon Bedrock models that support tool use. + def get_most_popular_song(station_name: str) -> str: + stations = { + "Radio Free Mars": "Starman – David Bowie", + "Neo Tokyo FM": "Plastic Love – Mariya Takeuchi", + "Cloud Nine Radio": "Blinding Lights – The Weeknd", + } + return stations.get(station_name, "Unknown Station – No chart data available") @@ -21 +26 @@ It is possible to use tools with the base inference operations ([InvokeModel](ht -For information about models that support tool calling, see [Supported models and model features](./conversation-inference-supported-models-features.html). +**Using Responses API for client-side tooling** @@ -23 +28 @@ For information about models that support tool calling, see [Supported models an -###### Topics +You can use the [Function calling](https://platform.openai.com/docs/guides/function-calling) feature provided by OpenAI to call this tool. Responses API is OpenAI's preferred API. Here is the Python code for Responses API for client-side tooling: @@ -25 +29,0 @@ For information about models that support tool calling, see [Supported models an - * [Call a tool with the Converse API](./tool-use-inference-call.html) @@ -27 +31,2 @@ For information about models that support tool calling, see [Supported models an - * [Converse API tool use examples](./tool-use-examples.html) + from openai import OpenAI + import json @@ -28,0 +34 @@ For information about models that support tool calling, see [Supported models an + client = OpenAI() @@ -29,0 +36,21 @@ For information about models that support tool calling, see [Supported models an + response = client.responses.create( + model="oss-gpt-120b", + input="What is the most popular song on Radio Free Mars?", + tools=[ + { + "type": "function", + "name": "get_most_popular_song", + "description": "Returns the most popular song on a radio station", + "parameters": { + "type": "object", + "properties": { + "station_name": { + "type": "string", + "description": "Name of the radio station" + } + }, + "required": ["station_name"] + } + } + ] + ) @@ -30,0 +58,432 @@ For information about models that support tool calling, see [Supported models an + if response.output and response.output[0].content: + tool_call = response.output[0].content[0] + args = json.loads(tool_call["arguments"]) + result = get_most_popular_song(args["station_name"]) + + final_response = client.responses.create( + model="oss-gpt-120b", + input=[ + { + "role": "tool", + "tool_call_id": tool_call["id"], + "content": result + } + ] + ) + + print(final_response.output_text) + +**Using Chat Completions API for client-side tooling** + +You can use the Chat Completions API also. Here is the Python code for using Chat Completions: + + + from openai import OpenAI + import json + + client = OpenAI() + + completion = client.chat.completions.create( + model="oss-gpt-120b", + messages=[{"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}], + tools=[{ + "type": "function", + "function": { + "name": "get_most_popular_song", + "description": "Returns the most popular song on a radio station", + "parameters": { + "type": "object", + "properties": { + "station_name": {"type": "string", "description": "Name of the radio station"} + }, + "required": ["station_name"] + } + } + }] + ) + + message = completion.choices[0].message + + if message.tool_calls: + tool_call = message.tool_calls[0] + args = json.loads(tool_call.function.arguments) + result = get_most_popular_song(args["station_name"]) + + followup = client.chat.completions.create( + model="oss-gpt-120b", + messages=[ + {"role": "user", "content": "What is the most popular song on Neo Tokyo FM?"}, + message, + {"role": "tool", "tool_call_id": tool_call.id, "content": result} + ] + ) + + print(followup.choices[0].message.content) + + +For more details on the using Function Calling on Responses API and Chat Completions API, see [Function Calling](https://platform.openai.com/docs/guides/function-calling) in OpenAI. + +**Using Converse API for client-side tooling** + +You can use the [Converse API](https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html) to let a model use a tool in a conversation. The following Python examples show how to use a tool that returns the most popular song on a fictional radio station. + + + # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. + # SPDX-License-Identifier: Apache-2.0 + """Shows how to use tools with the Converse API and the Cohere Command R model.""" + + import logging + import json + import boto3 + from botocore.exceptions import ClientError + + + class StationNotFoundError(Exception): + """Raised when a radio station isn't found.""" + pass + + + logger = logging.getLogger(__name__) + logging.basicConfig(level=logging.INFO) + + + def get_top_song(call_sign): + """Returns the most popular song for the requested station. + + Args: + call_sign (str): The call sign for the station for which you want + the most popular song. + + Returns: + response (json): The most popular song and artist. + """ + song = "" + artist = "" + + if call_sign == 'WZPZ': + song = "Elemental Hotel" + artist = "8 Storey Hike" + else: + raise StationNotFoundError(f"Station {call_sign} not found.") + + return song, artist + + + def generate_text(bedrock_client, model_id, tool_config, input_text): + """Generates text using the supplied Amazon Bedrock model. If necessary, + the function handles tool use requests and sends the result to the model. + + Args: + bedrock_client: The Boto3 Bedrock runtime client. + model_id (str): The Amazon Bedrock model ID. + tool_config (dict): The tool configuration. + input_text (str): The input text. + + Returns: + Nothing. + """ + logger.info("Generating text with model %s", model_id) + + # Create the initial message from the user input. + messages = [{"role": "user", + "content": [{"text": input_text}]}] + + response = bedrock_client.converse(modelId=model_id, + messages=messages, + toolConfig=tool_config)