AWS bedrock medium security documentation change
Summary
Major restructuring of Computer Use documentation including updated terminology, new implementation details, risk warnings, and replacement of code examples
Security assessment
The changes explicitly highlight security risks associated with the Computer Use API, including warnings about internet exposure and malicious content. The updated risk section emphasizes heightened dangers compared to standard APIs, and adds concrete mitigation guidance like limiting internet access to specific domains.
Diff
diff --git a/bedrock/latest/userguide/computer-use.md b/bedrock/latest/userguide/computer-use.md index 56b07e59f..2f9129dd1 100644 --- a//bedrock/latest/userguide/computer-use.md +++ b//bedrock/latest/userguide/computer-use.md @@ -7 +7 @@ -Example codeExample response +Use computer use on Amazon BedrockExample request @@ -9 +9 @@ Example codeExample response -# Use a computer use tool to complete an Amazon Bedrock model response +# Use computer use tools to automate GUI tasks with Amazon Bedrock models @@ -11 +11,3 @@ Example codeExample response -Computer use is an Anthropic Claude model capability (in beta) available with Anthropic Claude 3.7 Sonnet and Claude 3.5 Sonnet v2 only. With computer use, Claude can help you automate tasks through basic GUI actions. +Computer use is an Anthropic Claude tool family (in beta) for automating graphical user interface (GUI) tasks. The model returns `tool_use` actions such as `screenshot`, `left_click`, `type`, and `scroll` in JSON. Your application executes those actions on a desktop or container and returns the resulting screenshot to Claude on the next turn. + +For the underlying tool protocol, the full list of actions, and prompt-engineering guidance, see [Computer use](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) in the Anthropic documentation. This page covers the Amazon Bedrock-specific usage on top of that protocol. @@ -15 +17 @@ Computer use is an Anthropic Claude model capability (in beta) available with An -Computer use feature is made available to you as a ‘Beta Service’ as defined in the AWS Service Terms. It is subject to your Agreement with AWS and the AWS Service Terms, and the applicable model EULA. Please be aware that the Computer Use API poses unique risks that are distinct from standard API features or chat interfaces. These risks are heightened when using the Computer Use API to interact with the Internet. To minimize risks, consider taking precautions such as: +Computer use is made available to you as a "Beta Service" as defined in the AWS Service Terms. It is subject to your Agreement with AWS, the AWS Service Terms, and the applicable model EULA. The Computer Use API poses unique risks distinct from standard API features or chat interfaces. These risks are heightened when using the Computer Use API to interact with the Internet. To minimize risks, consider taking precautions such as: @@ -21 +23 @@ Computer use feature is made available to you as a ‘Beta Service’ as defined - * Limiting the computer use API’s internet access to required domains to reduce exposure to malicious content. + * Limit the computer use API's internet access to required domains to reduce exposure to malicious content. @@ -30 +32 @@ Any content that you enable Claude to see or access can potentially override ins -The computer use API offers several pre-defined computer use tools (_computer_20241022_ , _bash_20241022_ , and _text_editor_20241022_) for you to use. You can then create a prompt with your request, such as “send an email to Ben with the notes from my last meeting” and a screenshot (when required). The response contains a list of `tool_use` actions in JSON format (for example, scroll_down, left_button_press, screenshot). Your code runs the computer actions and provides Claude with screenshot showcasing outputs (when requested). +###### Topics @@ -32 +34 @@ The computer use API offers several pre-defined computer use tools (_computer_20 -The tools parameter has been updated to accept polymorphic tool types; a new `tool.type` property is being added to distinguish them. `type` is optional; if omitted, the tool is assumed to be a custom tool (previously the only tool type supported). Additionally, a new parameter, `anthropic_beta`, has been added, with a corresponding enum value: `computer-use-2024-10-22`. Only requests made with this parameter and enum can use the new computer use tools. It can be specified as follows: `"anthropic_beta": ["computer-use-2024-10-22"] `. + * Use computer use on Amazon Bedrock @@ -34 +36 @@ The tools parameter has been updated to accept polymorphic tool types; a new `to -To use computer use with Anthropic Claude 3.5 Sonnet v2 you can use the Converse API ([Converse](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) or [ConverseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html)). You specify the computer use specific fields in the `additionalModelRequestFields` field. For general information about calling the Converse API, see [Inference using Converse API](./conversation-inference.html). + * Example request @@ -36 +37,0 @@ To use computer use with Anthropic Claude 3.5 Sonnet v2 you can use the Converse -It is possible to use tools with the base inference operations ([InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) or [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html)). To find the inference parameters that you pass in the request body, see the [Anthropic Claude Messages API](./model-parameters-anthropic-claude-messages.html). @@ -38 +38,0 @@ It is possible to use tools with the base inference operations ([InvokeModel](ht -For more information, see [Computer use (beta)](https://docs.anthropic.com/en/docs/build-with-claude/computer-use) in the Anthropic documentation. @@ -40 +39,0 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do -###### Topics @@ -42 +41 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do - * Example code +## Use computer use on Amazon Bedrock @@ -44 +43 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do - * Example response +Computer use is supported on both the `bedrock-runtime` and `bedrock-mantle` endpoints. To find which models support computer use on each endpoint, see the _Capabilities and Features_ table in each [](./model-cards.html). Submitting a request to a model that doesn't support computer use, or pairing a tool type with an unsupported model, returns a `400 invalid_request_error`. @@ -45,0 +45 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do +To enable computer use on a request, include both: @@ -46,0 +47 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do + * An `anthropic_beta` entry naming a computer-use beta version (for example, `"anthropic_beta": ["computer-use-2025-11-24"]` in the request body for `bedrock-runtime`, or the `anthropic-beta: computer-use-2025-11-24` HTTP header for `bedrock-mantle`). @@ -47,0 +49 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do + * One or more Anthropic pre-defined tool entries in the `tools` array, with a `type` field that matches the beta version (for example, `computer_20251124`, `bash_20250124`, or `text_editor_20250124`). @@ -49 +50,0 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do -## Example code @@ -51 +51,0 @@ For more information, see [Computer use (beta)](https://docs.anthropic.com/en/do -The following code shows how to call the computer use API. The input is an image of the AWS console. @@ -54,2 +54 @@ The following code shows how to call the computer use API. The input is an image - with open('test_images/console.png', 'rb') as f: - png = f.read() +For the beta-version-to-tool-type pairings, additional Anthropic-defined tools (bash and text editor), and the full request and response field documentation, see [Computer use (Beta)](./model-parameters-anthropic-claude-messages-tool-use.html#model-parameters-anthropic-claude-messages-computer-use). @@ -57,32 +56 @@ The following code shows how to call the computer use API. The input is an image - response = bedrock.converse( - modelId='anthropic.claude-3-5-sonnet-20241022-v2:0', - messages=[ - { - 'role': 'user', - 'content': [ - { - 'text': 'Go to the bedrock console' - }, - { - 'image': { - 'format': 'png', - 'source': { - 'bytes': png - } - } - } - ] - } - ], - additionalModelRequestFields={ - "tools": [ - { - "type": "computer_20241022", - "name": "computer", - "display_height_px": 768, - "display_width_px": 1024, - "display_number": 0 - }, - { - "type": "bash_20241022", - "name": "bash", +## Example request @@ -90,24 +58 @@ The following code shows how to call the computer use API. The input is an image - }, - { - "type": "text_editor_20241022", - "name": "str_replace_editor", - } - ], - "anthropic_beta": ["computer-use-2024-10-22"] - }, - toolConfig={ - 'tools': [ - { - 'toolSpec': { - 'name': 'get_weather', - 'inputSchema': { - 'json': { - 'type': 'object' - } - } - } - } - ] - }) - - print(json.dumps(response, indent=4)) +The following Python example sends a Messages API request through the `bedrock-runtime` endpoint with the computer use tool. Replace the `modelId` with one that supports computer use (see [](./model-cards.html)). @@ -115 +59,0 @@ The following code shows how to call the computer use API. The input is an image -## Example response @@ -117 +61,2 @@ The following code shows how to call the computer use API. The input is an image -The example code emits output similar to the following. + import boto3 + import json @@ -118,0 +64 @@ The example code emits output similar to the following. + client = boto3.client("bedrock-runtime", region_name="us-east-1") @@ -119,0 +66,10 @@ The example code emits output similar to the following. + response = client.invoke_model( + modelId="us.anthropic.claude-opus-4-7", + body=json.dumps({ + "anthropic_version": "bedrock-2023-05-31", + "anthropic_beta": ["computer-use-2025-11-24"], + "max_tokens": 1024, + "messages": [ + {"role": "user", "content": "Take a screenshot of the desktop."} + ], + "tools": [ @@ -121,11 +77 @@ The example code emits output similar to the following. - "id": "msg_bdrk_01Ch8g9MF3A9FTrmeywrwfMZ", - "type": "message", - "role": "assistant", - "content": [ - { - "type": "text", - "text": "I can see from the screenshot that we're already in the AWS Console. To go to the Amazon Bedrock console specifically, I'll click on the Amazon Bedrock service from the \"Recently Visited\" section." - }, - { - "type": "tool_use", - "id": "toolu_bdrk_013sAzs1gsda9wLrfD8bhYQ3", + "type": "computer_20251124", @@ -133,3 +79,2 @@ The example code emits output similar to the following. - "input": { - "action": "screenshot" - } + "display_width_px": 1024, + "display_height_px": 768, @@ -138,7 +83,8 @@ The example code emits output similar to the following. - "stop_reason": "tool_use", - "stop_sequence": null, - "usage": { - "input_tokens": 3710, - "output_tokens": 97 - } - } + }), + ) + + result = json.loads(response["body"].read()) + print(result["stop_reason"]) # "tool_use" when the model issues a computer action + print(result["content"]) # contains the tool_use block describing the action + +For the equivalent `bedrock-mantle` Messages API request shape, see [Inference using Anthropic Messages API](./inference-messages-api.html). @@ -154 +100 @@ Playgrounds (console) -Chat Completions API (legacy) +Structured Outputs