AWS nova documentation change
Summary
Expanded documentation on cross-modal input with implementation details, event structure examples, and added note about DTMF integration requiring client-side handling
Security assessment
The change adds a security consideration regarding DTMF input handling (credit card numbers), explicitly stating Nova Sonic doesn't process DTMF natively and requiring implementers to handle sensitive data conversion. While this highlights security responsibilities, it documents a security-relevant implementation pattern rather than addressing a specific vulnerability.
Diff
diff --git a/nova/latest/nova2-userguide/sonic-cross-modal.md b/nova/latest/nova2-userguide/sonic-cross-modal.md index eedb0763a..495377079 100644 --- a//nova/latest/nova2-userguide/sonic-cross-modal.md +++ b//nova/latest/nova2-userguide/sonic-cross-modal.md @@ -9 +9,16 @@ How it worksEvent structure -Cross-modal input enables users to send text messages during an active voice conversation. This feature is particularly useful for scenarios where typing is more appropriate than speaking, such as sharing URLs or email addresses, providing sensitive information like passwords or account numbers, sending structured data like addresses or phone numbers, or clarifying spelling of names or technical terms. +Amazon Nova 2 Sonic now supports cross-modal input, allowing you to send text messages in addition to voice input during a conversation session. While speech remains the primary mode of interaction, text input provides flexibility for scenarios where typing is more convenient or appropriate. + +**Continuous streaming required** : Cross-modal input requires an active streaming session to function properly. The session must maintain continuous streaming like a regular voice session, otherwise standard session timeouts will be applied and the connection will be terminated. + +Sensitivity levels in cross-modal text input is useful for scenarios such as: + + * Client-side app integration (web and mobile): Allows users to interact with the application using both text and voice, supporting seamless multimodal experiences. + + * "Model-start-first" use case: A text message can be sent immediately after the session starts to prompt the model to begin speaking. + + * Guiding the model during async tool calling: When a toolUse event is triggered and the system begins processing tool calls, the client can send a text message to Sonic to provide a natural response while waiting — such as, “Hold on a second while I process your information. In the meantime, is there anything else I can assist with?” + + * Telephony DTMF integration: Customer uses phone keypad to enter sensitive information (such as credit card numbers). Note: Amazon Nova Sonic does not process DTMF tones natively. To support DTMF input, your system must detect the tones, convert them to text (such as "1234"), and send to Nova 2 Sonic. + + + @@ -13 +28,12 @@ Cross-modal input enables users to send text messages during an active voice con -To send cross-modal text input, you use the same event structure as other content types, but with the `interactive` parameter set to `true` in the `InputContentStartEvent`. This signals to Amazon Nova 2 Sonic that the text input is being sent during an active voice session. +Cross-modal input uses a three-event sequence similar to audio input: + + 1. **Content Start Event:** Signals the beginning of text input + + 2. **Text Input Event:** Contains the actual text message + + 3. **Content End Event:** Signals the completion of text input + + + + +All three events must use the same promptName and contentName to maintain the sequence. A new UUID should be generated for contentName each time you send text input to ensure proper multi-turn conversation tracking. @@ -17 +43,70 @@ To send cross-modal text input, you use the same event structure as other conten -Cross-modal input follows a three-event pattern: +Initiates the text input sequence with configuration details: + + + { + "event": { + "contentStart": { + "promptName": "<prompt_name>", + "contentName": "<new_content_name>", + "role": "USER", + "type": "TEXT", + "interactive": true, + "textInputConfiguration": { + "mediaType": "text/plain" + } + } + } + } + +Key Parameters: + + * `promptName`:The name of your conversation prompt (consistent across the session) + + * `contentName`: A unique identifier for this text input (generate a new UUID for each message) + + * `role`: Set to `"USER"` to indicate user input + + * `type`: Set to `"TEXT"` for text input + + * `interactive`: Set to `true` to enable interactive mode + + * `mediaType`: Set to `"text/plain"` for plain text content + + + + +Contains the actual text message content: + + + { + "event": { + "textInput": { + "promptName": "<prompt_name>", + "contentName": "<new_content_name>", + "content": "<your_text_message>" + } + } + } + +Key Parameters: + + * `promptName`: Must match the value from Content Start Event + + * `contentName`: Must match the value from Content Start Event + + * `role`: Your text message string + + + + +Signals the completion of the text input: + + + { + "event": { + "contentEnd": { + "promptName": "<prompt_name>", + "contentName": "<new_content_name>" + } + } + } @@ -19 +114 @@ Cross-modal input follows a three-event pattern: - 1. **Content Start Event:** Set `interactive: true` and `role: "USER"` +Key Parameters: @@ -21 +116 @@ Cross-modal input follows a three-event pattern: - 2. **Text Input Event:** Provide the text content + * `promptName`: Must match the value from previous events @@ -23 +118 @@ Cross-modal input follows a three-event pattern: - 3. **Content End Event:** Close the content segment + * `contentName`: Must match the value from previous events