AWS bedrock documentation change
Summary
Updated Provisioned Throughput documentation with new prerequisites, model-specific guidance (Amazon Nova Lite), and restructured code examples. Added steps for finding model IDs/ARNs and configuring inference requests.
Security assessment
Added prerequisite about requesting model access via IAM policies ('Add or remove access to Amazon Bedrock foundation models'), which relates to security controls. However, there's no evidence of addressing a specific vulnerability or security incident.
Diff
diff --git a/bedrock/latest/userguide/prov-thru-code-examples.md b/bedrock/latest/userguide/prov-thru-code-examples.md index bf6559b21..43d0fb259 100644 --- a//bedrock/latest/userguide/prov-thru-code-examples.md +++ b//bedrock/latest/userguide/prov-thru-code-examples.md @@ -7 +7 @@ -The following code examples demonstrate how to create, use, and manage a Provisioned Throughput with the AWS CLI and the Python SDK. +The following code examples demonstrate how to create a Provisioned Throughput and how to manage and invoke it, using the AWS CLI and the Python SDK. You can create a Provisioned Throughput from a foundation model or from a model that you've already customized. Before you get started, carry out the following prerequisites: @@ -9 +9 @@ The following code examples demonstrate how to create, use, and manage a Provisi -AWS CLI +**Prerequisites** @@ -10,0 +11 @@ AWS CLI +The following examples use the Amazon Nova Lite model, whose model ID is `amazon.nova-lite-v1:0:24k`. If you haven't already, request access to the Amazon Nova Lite by following the steps at [Add or remove access to Amazon Bedrock foundation models](./model-access-modify.html). @@ -12 +13 @@ AWS CLI -Create a no-commitment Provisioned Throughput called `MyPT` based off a custom model called `MyCustomModel` that was customized from the Anthropic Claude v2.1 model by running the following command in a terminal. +If you want to purchase Provisioned Throughput for a different foundation model or a custom model, you'll have to do the following: @@ -13,0 +15 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m + 1. Find the model's ID (for foundation models), name (for custom models), or ARN (for either) by doing one of the following: @@ -15,4 +17 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m - aws bedrock create-provisioned-model-throughput \ - --model-units 1 \ - --provisioned-model-name MyPT \ - --model-id arn:aws:bedrock:us-east-1::custom-model/anthropic.claude-v2:1:200k/MyCustomModel + * If you're purchasing a Provisioned Throughput for a foundation model, find the ID or Amazon Resource Name (ARN) of a model that supports provisioning in one of the following ways: @@ -20 +19 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m -The response returns a `provisioned-model-arn`. Allow some time for the creation to complete. To check its status, provide the name or ARN of the provisioned model as the `provisioned-model-id` in the following command. + * Look up the value in the table at [Supported Region and models for Provisioned Throughput](./pt-supported.html). @@ -21,0 +21 @@ The response returns a `provisioned-model-arn`. Allow some time for the creation + * Send a [ListFoundationModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html) request and specify the `byInferenceType` value as `PROVISIONED` to see a list of models that support provisioning. Find the value in the `modelId` or `modelArn` field. @@ -23,2 +23,7 @@ The response returns a `provisioned-model-arn`. Allow some time for the creation - aws bedrock get-provisioned-model-throughput \ - --provisioned-model-id MyPT + * If you're purchasing a Provisioned Throughput for a custom model, find the name or Amazon Resource Name (ARN) of the model that you customized in one of the following ways: + + * In the Amazon Bedrock console, choose **Custom models** from the left navigation pane. Find the name of your customized model in the **Models** list or select it and find the **Model ARN** in the **Model details**. + + * Send a [ListCustomModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListCustomModels.html) request and find the `modelName` or `modelArn` value of your custom model in the response. + + 2. Modify the `body` of the [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) request in the examples below to match the format of the body of the model by finding it in [Inference request parameters and response fields for foundation models](./model-parameters.html). @@ -26 +30,0 @@ The response returns a `provisioned-model-arn`. Allow some time for the creation -Change the name of the Provisioned Throughput and associate it with a different model customized from Anthropic Claude v2.1. @@ -29,4 +32,0 @@ Change the name of the Provisioned Throughput and associate it with a different - aws bedrock update-provisioned-model-throughput \ - --provisioned-model-id MyPT \ - --desired-provisioned-model-name MyPT2 \ - --desired-model-id arn:aws:bedrock:us-east-1::custom-model/anthropic.claude-v2:1:200k/MyCustomModel2 @@ -34 +34,16 @@ Change the name of the Provisioned Throughput and associate it with a different -Run inference with your updated provisioned model with the following command. You must provide the ARN of the provisioned model, returned in the `UpdateProvisionedModelThroughput` response, as the `model-id`. The output is written to a file named `output.txt` in your current folder. +Choose the tab for your preferred method, and then follow the steps: + +AWS CLI + + + 1. Send a [CreateProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateProvisionedModelThroughput.html) request to create a no-commitment Provisioned Throughput called `MyPT`, by running the following command in a terminal: + + aws bedrock create-provisioned-model-throughput \ + --model-units 1 \ + --provisioned-model-name MyPT \ + --model-id amazon.nova-lite-v1:0:24k + + 2. The response returns a `provisioned-model-arn`. Allow some time for the creation to complete. To check its status, send a [GetProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetProvisionedModelThroughput.html) request and provide the name or ARN of the provisioned model as the `provisioned-model-id`, by running the following command: + + aws bedrock get-provisioned-model-throughput \ + --provisioned-model-id ${provisioned-model-arn} @@ -35,0 +51 @@ Run inference with your updated provisioned model with the following command. Yo + 3. Run inference with your provisioned model by sending an [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) request. Provide the ARN of the provisioned model that was returned in the `CreateProvisionedModelThroughput` response, as the `model-id`. The output is written to a file named `output.txt` in your current folder. @@ -39 +55,11 @@ Run inference with your updated provisioned model with the following command. Yo - --body '{"inputText": "What is AWS?", "textGenerationConfig": {"temperature": 0.5}}' \ + --body '{ + "messages": [{ + "role": "user", + "content": [{ + "text": "Hello" + }] + }], + "inferenceConfig": { + "temperature":0.7 + } + }' \ @@ -43,2 +69 @@ Run inference with your updated provisioned model with the following command. Yo -Delete the Provisioned Throughput using the following command. You'll no longer be charged for the Provisioned Throughput. - + 4. Send a [DeleteProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_DeleteProvisionedModelThroughput.html) request to delete the Provisioned Throughput using the following command. You'll no longer be charged for the Provisioned Throughput. @@ -47 +72,4 @@ Delete the Provisioned Throughput using the following command. You'll no longer - --provisioned-model-id MyPT2 + --provisioned-model-id MyPT + + + @@ -52 +80 @@ Python (Boto) -Create a no-commitment Provisioned Throughput called `MyPT` based off a custom model called `MyCustomModel` that was customized from the Anthropic Claude v2.1 model by running the following code snippet. +The following code snippets walk you through creating a Provisioned Throughput getting information about it, and invoking the Provisioned Throughput. @@ -53,0 +82 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m + 1. To create a no-commitment Provisioned Throughput called `MyPT` and assign the ARN of the Provisioned Throughput to a variable called `provisioned_model_arn`, send the following [CreateProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateProvisionedModelThroughput.html) request: @@ -56,0 +86,2 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m + provisioned_model_name = 'MyPT' + @@ -58 +89 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m - bedrock.create_provisioned_model_throughput( + response = bedrock.create_provisioned_model_throughput( @@ -60,2 +91,2 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m - provisionedModelName='MyPT', - modelId='arn:aws:bedrock:us-east-1::custom-model/anthropic.claude-v2:1:200k/MyCustomModel' + provisionedModelName=provisioned_model_name, + modelId='amazon.nova-lite-v1:0:24k' @@ -64 +95 @@ Create a no-commitment Provisioned Throughput called `MyPT` based off a custom m -The response returns a `provisionedModelArn`. Allow some time for the creation to complete. You can check its status with the following code snippet. You can provide either the name of the Provisioned Throughput or the ARN returned from the [CreateProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateProvisionedModelThroughput.html) response as the `provisionedModelId`. + provisioned_model_arn = response['provisionedModelArn'] @@ -65,0 +97 @@ The response returns a `provisionedModelArn`. Allow some time for the creation t + 2. Allow some time for the creation to complete. You can check its status with the following code snippet. You can provide either the name of the Provisioned Throughput or the ARN returned from the [CreateProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateProvisionedModelThroughput.html) response as the `provisionedModelId`. @@ -67,14 +99 @@ The response returns a `provisionedModelArn`. Allow some time for the creation t - bedrock.get_provisioned_model_throughput(provisionedModelId='MyPT') - -Change the name of the Provisioned Throughput and associate it with a different model customized from Anthropic Claude v2.1. Then send a [GetProvisionedModelThroughput](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_GetProvisionedModelThroughput.html) request and save the ARN of the provisioned model to a variable to use for inference. - - - bedrock.update_provisioned_model_throughput( - provisionedModelId='MyPT', - desiredProvisionedModelName='MyPT2', - desiredModelId='arn:aws:bedrock:us-east-1::custom-model/anthropic.claude-v2:1:200k/MyCustomModel2' - ) - - arn_MyPT2 = bedrock.get_provisioned_model_throughput(provisionedModelId='MyPT2').get('provisionedModelArn') - -Run inference with your updated provisioned model with the following command. You must provide the ARN of the provisioned model as the `modelId`. + bedrock.get_provisioned_model_throughput(provisionedModelId=provisioned_model_name) @@ -81,0 +101 @@ Run inference with your updated provisioned model with the following command. Yo + 3. Run inference with your updated provisioned model with the following command and using the ARN of the provisioned model as the `modelId`. @@ -143 +163 @@ Run inference with your updated provisioned model with the following command. Yo - model_id = arn_myPT2 + model_id = provisioned-model-arn @@ -174 +194,4 @@ Run inference with your updated provisioned model with the following command. Yo -Delete the Provisioned Throughput with the following code snippet. You'll no longer be charged for the Provisioned Throughput. + 4. Delete the Provisioned Throughput with the following code snippet. You'll no longer be charged for the Provisioned Throughput. + + bedrock.delete_provisioned_model_throughput(provisionedModelId=provisioned_model_name) + @@ -177 +199,0 @@ Delete the Provisioned Throughput with the following code snippet. You'll no lon - bedrock.delete_provisioned_model_throughput(provisionedModelId='MyPT2')