AWS nova high security documentation change
Summary
Removed 'Video understanding limitations' and 'Video understanding examples' sections, restructured content headings
Security assessment
The removed 'Video understanding limitations' section contained security-related content including: 1) Explicit statement about refusing to process inappropriate content per Acceptable Use Policy 2) Healthcare application warnings 3) People identification restrictions. Removing this documentation could lead to insecure implementations if users are unaware of these limitations.
Diff
diff --git a/nova/latest/userguide/modalities-video.md b/nova/latest/userguide/modalities-video.md index 44b1f13e4..479dacd09 100644 --- a//nova/latest/userguide/modalities-video.md +++ b//nova/latest/userguide/modalities-video.md @@ -5 +5 @@ -Video size informationVideo tokensVideo understanding limitationsVideo understanding examples +Video size informationVideo tokens @@ -56,157 +55,0 @@ Estimated token count | 2,880 | 8,640 | 276,480 | 345,600 | 518,400 | 777,600 | -## Video understanding limitations - -The following are key model limitations, where model accuracy and performance might not be guaranteed. - - * **One video per request:** currently the model supports only 1 video per request. Some frameworks and libraries use memory to keep track of previous interactions. There might be a video that was added in a previous context. - - * **No audio support:** The models are currently trained to process and understand video content solely based on the visual information in the video. They do not possess the capability to analyze or comprehend any audio components that are present in the video. - - * **Temporal causality:** The model has limited understanding of event causality across the progression of the video. Although it answers well to point in time questions, it does not perform as well on answers that depends on understanding a sequence of events - - * **Multilingual image understanding:** The models have limited understanding of multilingual images and video frames. They might struggle or hallucinate on similar tasks. - - * **People identification** : The Amazon Nova models do not support the capability to identify or name individuals in images, documents, or videos. The models will refuse to perform such tasks. - - * **Spatial reasoning** : The Amazon Nova models have limited spatial reasoning capabilities. They may struggle with tasks that require precise localization or layout analysis. - - * **Small text in images or videos** : If the text in the image or video is too small, consider increasing relative size of the text in the image by cropping to the relevant section while preserving necessary content. - - * **Counting** : The Amazon Nova models can provide approximate counts of objects in an image, but might not always be precisely accurate, especially when dealing with large numbers of small objects. - - * **Inappropriate content** : The Amazon Nova models will not process inappropriate or explicit images that violate the Acceptable Use Policy - - * **Healthcare applications** : Due to the sensitive nature of these artifacts, even though Amazon Nova models can give general analysis on healthcare images or videos, we do not recommend that you interpret complex diagnostic scans. The response of Amazon Nova should never be considered a substitute for professional medical advice. - - - - -## Video understanding examples - -The following examples show how to send video prompts to Amazon Nova models using different input methods. - -The following example shows how to send a video prompt to Amazon Nova Model with [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html). - - - # Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. - # SPDX-License-Identifier: Apache-2.0 - import base64 - import boto3 - import json - # Create a Bedrock Runtime client in the AWS Region of your choice. - client = boto3.client( - "bedrock-runtime", - region_name="us-east-1", - ) - - MODEL_ID = "us.amazon.nova-lite-v1:0" - # Open the image you'd like to use and encode it as a Base64 string. - with open("media/cooking-quesadilla.mp4", "rb") as video_file: - binary_data = video_file.read() - base_64_encoded_data = base64.b64encode(binary_data) - base64_string = base_64_encoded_data.decode("utf-8") - # Define your system prompt(s). - system_list= [ - { - "text": "You are an expert media analyst. When the user provides you with a video, provide 3 potential video titles" - } - ] - # Define a "user" message including both the image and a text prompt. - message_list = [ - { - "role": "user", - "content": [ - { - "video": { - "format": "mp4", - "source": { - "bytes": video // Binary array (Converse API) or Base64-encoded string (Invoke API) - }, - } - }, - { - "text": "Provide video titles for this clip." - }, - ], - } - ] - # Configure the inference parameters. - inf_params = {"maxTokens": 300, "topP": 0.1, "topK": 20, "temperature": 0.3} - - native_request = { - "schemaVersion": "messages-v1", - "messages": message_list, - "system": system_list, - "inferenceConfig": inf_params, - } - # Invoke the model and extract the response body. - response = client.invoke_model(modelId=MODEL_ID, body=json.dumps(native_request)) - model_response = json.loads(response["body"].read()) - # Pretty print the response JSON. - print("[Full Response]") - print(json.dumps(model_response, indent=2)) - # Print the text content for easy readability. - content_text = model_response["output"]["message"]["content"][0]["text"] - print("\n[Response Content Text]") - print(content_text) - -The following example shows how to send a video using an Amazon S3 location to Amazon Nova with [InvokeModel](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html). - - - import base64 - import boto3 - import json - # Create a Bedrock Runtime client in the AWS Region of your choice. - client = boto3.client( - "bedrock-runtime", - region_name="us-east-1", - ) - - MODEL_ID = "us.amazon.nova-lite-v1:0" - # Define your system prompt(s). - system_list = [ - { - "text": "You are an expert media analyst. When the user provides you with a video, provide 3 potential video titles" - } - ] - # Define a "user" message including both the image and a text prompt. - message_list = [ - { - "role": "user", - "content": [ - { - "video": { - "format": "mp4", - "source": { - "s3Location": { - "uri": "s3://my_bucket/my_video.mp4", - "bucketOwner": "111122223333" - } - } - } - }, - { - "text": "Provide video titles for this clip." - } - ] - } - ] - # Configure the inference parameters. - inf_params = {"maxTokens": 300, "topP": 0.1, "topK": 20, "temperature": 0.3} - - native_request = { - "schemaVersion": "messages-v1", - "messages": message_list, - "system": system_list, - "inferenceConfig": inf_params, - } - # Invoke the model and extract the response body. - response = client.invoke_model(modelId=MODEL_ID, body=json.dumps(native_request)) - model_response = json.loads(response["body"].read()) - # Pretty print the response JSON. - print("[Full Response]") - print(json.dumps(model_response, indent=2)) - # Print the text content for easy readability. - content_text = model_response["output"]["message"]["content"][0]["text"] - print("\n[Response Content Text]") - print(content_text) - @@ -219 +62 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Image understanding +Image understanding examples @@ -221 +64 @@ Image understanding -Document understanding +Video understanding limitations