AWS quicksuite documentation change
Summary
Added support for agent/research hours logging, updated log schema to snake_case, introduced new fields for monitoring blocked queries and usage metrics, and restructured documentation sections.
Security assessment
Changes focus on operational monitoring enhancements (new log types, field renaming, usage tracking) without addressing vulnerabilities or security controls. No evidence of patching exploits, access control changes, or security incident response. Added 'status_code' for blocked queries improves diagnostics but isn't a security feature.
Diff
diff --git a/quicksuite/latest/userguide/monitoring-quicksuite-chat-feedback-cloudwatch.md b/quicksuite/latest/userguide/monitoring-quicksuite-chat-feedback-cloudwatch.md index 9906262a1..dbcc3067f 100644 --- a//quicksuite/latest/userguide/monitoring-quicksuite-chat-feedback-cloudwatch.md +++ b//quicksuite/latest/userguide/monitoring-quicksuite-chat-feedback-cloudwatch.md @@ -5 +5 @@ -Supported log destinationsPrerequisitesConfigure loggingLog schema and formatSecurity considerations +Supported log destinationsPrerequisitesConfigure loggingLog schema and formatAgent/Research hours LogsSecurity considerations @@ -7 +7 @@ Supported log destinationsPrerequisitesConfigure loggingLog schema and formatSec -# Monitoring Amazon Quick Suite chat and feedback using CloudWatch Logs +# Monitoring Amazon Quick Suite usage using CloudWatch Logs @@ -9 +9 @@ Supported log destinationsPrerequisitesConfigure loggingLog schema and formatSec -You can use [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) to deliver user conversations and response feedback in Amazon Quick Suite for you to analyze. These logs can be delivered to multiple destinations, such as CloudWatch, Amazon S3, or (standard rates apply). We recommend that you set up conversation and feedback logging shortly after creating your Amazon Quick Suite chat agent. +You can use [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) to deliver chat conversations, user feedback and agent/research hours usage in Amazon Quick Suite for you to analyze. These logs can be delivered to multiple destinations, such as CloudWatch, Amazon S3, or (standard rates apply). We recommend that you set up conversation and feedback logging shortly after enabling Amazon Quick Suite AI features. @@ -11 +11 @@ You can use [Amazon CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatc -The following are examples of tasks you can complete with logs from conversations and response feedback in Amazon Quick Suite: +The following are examples of tasks you can complete with logs from Amazon Quick Suite: @@ -20,0 +21,4 @@ The following are examples of tasks you can complete with logs from conversation + * Identify and Analyze cases where the chat returned no answer or the user query was blocked + + * Monitor agent and research hours usage + @@ -108 +112 @@ For example IAM policies with all the required permissions for your specific log -Create a delivery source with the [PutDeliverySource](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliverySource.html) CloudWatch Logs API operation. Give the delivery source a name and for `resourceArn`, specify the ARN of your application. For `logType`, specify `CHAT_LOGS` or `FEEDBACK_LOGS` +Create a delivery source with the [PutDeliverySource](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_PutDeliverySource.html) CloudWatch Logs API operation. Give the delivery source a name and for `resourceArn`, specify the ARN of your application. For `logType`, specify `CHAT_LOGS`, `AGENT_HOURS_LOGS` or `FEEDBACK_LOGS` @@ -123,0 +128,7 @@ Create a delivery source with the [PutDeliverySource](https://docs.aws.amazon.co + + { + "logType": "AGENT_HOURS_LOGS", + "name": "my-quick-suite-delivery-source", + "resourceArn": "arn:aws:quicksight:your-region:your-account-id:account/account-id" + } + @@ -153 +164 @@ All log events include these common fields: - * `resourceArn` \- Resource ARN of your Amazon Quick Suite account (for example, `arn:aws:quicksight:us-east-1:111122223333:account/111122223333:`) + * `resource_arn` \- Resource ARN of your Amazon Quick Suite account (for example, `arn:aws:quicksight:us-east-1:111122223333:account/111122223333:`) @@ -155 +166 @@ All log events include these common fields: - * `eventTimestamp` \- ISO 8601 timestamp of the event (for example, `1763532110061`) + * `event_timestamp` \- ISO 8601 timestamp of the event (for example, `1763532110061`) @@ -161 +172 @@ All log events include these common fields: - * `userArn` \- Amazon Quick Suite user ARN associated with the event (for example, `"arn:aws:quicksight:us-west-2:111122223333:user/default/user"`) + * `user_arn` \- Amazon Quick Suite user ARN associated with the event (for example, `"arn:aws:quicksight:us-west-2:111122223333:user/default/user"`) @@ -163 +173,0 @@ All log events include these common fields: - * `userType` \- Amazon Quick Suite user type associated with the event (for example, `ADMIN_PRO`) @@ -165 +174,0 @@ All log events include these common fields: - * `nameSpace` \- Amazon Quick Suite namespace for the event (for example, `default`) @@ -167 +175,0 @@ All log events include these common fields: - * `statusCode` \- Status of the event delivery (for example, `Success`, `request_blocked`, `no_answer_found`) @@ -168,0 +177 @@ All log events include these common fields: +### Chat logs @@ -169,0 +179 @@ All log events include these common fields: +Chat logs capture conversation interactions and contains below fields: @@ -170,0 +181 @@ All log events include these common fields: + * `status_code` \- Status of the chat request (for example, `Success, request_blocked, no_answer_found` ) @@ -172 +183 @@ All log events include these common fields: -### Chat logs + * `namespace*` \- Amazon Quick Suite namespace for the event (for example, `default`) @@ -174 +185 @@ All log events include these common fields: -Chat logs capture conversation interactions and contains below fields: + * `user_type` \- Amazon Quick Suite user type associated with the event (for example, `ADMIN_PRO`) @@ -176 +187 @@ Chat logs capture conversation interactions and contains below fields: - * `conversationId` \- Unique ID for the user conversation + * `conversation_id` \- Unique ID for the user conversation @@ -178 +189 @@ Chat logs capture conversation interactions and contains below fields: - * `systemMessageId` \- System-generated message ID + * `system_message_id` \- System-generated message ID @@ -180 +191 @@ Chat logs capture conversation interactions and contains below fields: - * `latency` \- Chat message latency in milliseconds + * `latency*` \- Chat message latency in milliseconds @@ -182 +193 @@ Chat logs capture conversation interactions and contains below fields: - * `timeToFirstToken` \- Time in milliseconds of first response token + * `time_to_first_token*` \- Time in milliseconds of first response token @@ -184 +195 @@ Chat logs capture conversation interactions and contains below fields: - * `messageScope` \- Scope of the message + * `message_scope` \- Scope of the message (for example, `all_resources, specific_resources, no_resources` ) @@ -186 +197 @@ Chat logs capture conversation interactions and contains below fields: - * `userMessageId` \- Unique ID of the user message + * `user_message_id` \- Unique ID of the user message @@ -188 +199 @@ Chat logs capture conversation interactions and contains below fields: - * `userMessage` \- user message in the conversation + * `user_message` \- user message in the conversation @@ -190 +201 @@ Chat logs capture conversation interactions and contains below fields: - * `agentId` \- Unique ID of the chat agent + * `agent_id` \- Unique ID of the chat agent @@ -192 +203 @@ Chat logs capture conversation interactions and contains below fields: - * `flowId` \- Unique ID of the Amazon Quick Suite Flow + * `flow_id` \- Unique ID of the Amazon Quick Suite Flow @@ -194 +205 @@ Chat logs capture conversation interactions and contains below fields: - * `systemTextMessage` \- Text message in the conversation + * `system_text_message` \- System response in the conversation @@ -196 +207 @@ Chat logs capture conversation interactions and contains below fields: - * `surfaceType` \- Application being used for the conversation + * `surface_type*` \- Application being used for the conversation @@ -198 +209 @@ Chat logs capture conversation interactions and contains below fields: - * `webSearch` \- Web search required or not + * `web_search*` \- Web search enabled or not @@ -200 +211 @@ Chat logs capture conversation interactions and contains below fields: - * `userSelectedResources`\- List of resources selected by user + * `user_selected_resources`\- List of resources selected by user @@ -202 +213 @@ Chat logs capture conversation interactions and contains below fields: - * `actionConnectors` \- List of action connectors + * `action_connectors` \- List of action connectors @@ -204 +215 @@ Chat logs capture conversation interactions and contains below fields: - * `citedResource` \- List of cited resources + * `cited_resource` \- List of cited resources @@ -206 +217 @@ Chat logs capture conversation interactions and contains below fields: - * `fileAttachment` \- List of files attached by user + * `file_attachment` \- List of files attached by user @@ -215,2 +226,5 @@ The following is an example of chat logs: - "conversationId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "systemMessageId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "status_code": "success", + "namespace": "default", + "user_type": "ADMIN_PRO", + "conversation_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "system_message_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", @@ -218,13 +232,13 @@ The following is an example of chat logs: - "timeToFirstToken": "10000", - "messageScope": "General Knowledge, ALL etc", - "userMessageId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "userMessage": "What is the status of my project?", - "agentId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "flowId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d?", - "systemTextMessage": "What is the status of my project?", - "surfaceType": "Slack, WebApp etc.", - "webSearch": "TRUE" - "userSelectedResources": [{"resource_type": "Dashboard","resource_id": "146abs-1222-534894"},{"resource_type": "Space","resource_id": "123abs-1234-534894"}], - "actionConnectors": [{"action_connector_id": "quicksight-website"},{"action_connector_id": "123abs-1234-534894"}] - "citedResource": [{"cited_resource_name": "Dashboard","cited_resource_id": "146abs-1222-534894","cited_resource_name": "ds1"},{"cited_resource_name": "Space","cited_resource_id": "123abs-1234-534894","cited_resource_name": "space1"}], - "fileAttachment": [{"file_attachmet_type": "pdf","file_attachment_name": "file1.pdf"},{"file_attachmet_type": "txt","file_attachment_name": "file2.txt"}] + "time_to_first_token": "10000", + "message_scope": "all_resources", + "user_message_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "user_message": "Hi chat", + "agent_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "flow_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d?", + "system_text_message": "Hello user", + "surface_type": "WEB_EXPERIENCE", + "web_search": "true" + "user_selected_resources": [{"resource_type": "Dashboard","resource_id": "146abs-1222-534894"},{"resource_type": "Space","resource_id": "123abs-1234-534894"}], + "action_connectors": [{"action_connector_id": "quicksight-website"},{"action_connector_id": "123abs-1234-534894"}] + "cited_resource": [{"cited_resource_name": "Dashboard","cited_resource_id": "146abs-1222-534894","cited_resource_name": "ds1"},{"cited_resource_name": "Space","cited_resource_id": "123abs-1234-534894","cited_resource_name": "space1"}], + "file_attachment": [{"file_attachmet_type": "pdf","file_attachment_name": "file1.pdf"},{"file_attachmet_type": "txt","file_attachment_name": "file2.txt"}] @@ -237 +251 @@ Feedback logs capture user feedback on chat and contains below fields: - * `conversationId` \- Unique ID of the conversation + * `status_code` \- Status of the event delivery @@ -239 +253 @@ Feedback logs capture user feedback on chat and contains below fields: - * `researchId` \- Unique ID of the research + * `namespace*` \- Amazon Quick Suite namespace for the event (for example, `default`) @@ -241 +255 @@ Feedback logs capture user feedback on chat and contains below fields: - * `systemMessageId` \- System generated message ID + * `user_type` \- Amazon Quick Suite user type associated with the event (for example, `ADMIN_PRO`) @@ -243 +257 @@ Feedback logs capture user feedback on chat and contains below fields: - * `userMessageId` \- Unique ID of user message + * `conversation_id` \- Unique ID of the conversation @@ -245 +259 @@ Feedback logs capture user feedback on chat and contains below fields: - * `feedback_type` \- Type of feedback + * `system_message_id` \- System generated message ID @@ -247 +261 @@ Feedback logs capture user feedback on chat and contains below fields: - * `feedback_reason` \- Reason for the feedback + * `user_message_id` \- Unique ID of user message @@ -249 +263 @@ Feedback logs capture user feedback on chat and contains below fields: - * `feedback_details` \- Text message in the feedback + * `feedback_type` \- Type of feedback (for example, `Not Useful, Useful` ) @@ -251 +265,3 @@ Feedback logs capture user feedback on chat and contains below fields: - * `rating` \- Rating provided by the user + * `feedback_reason` \- Feedback reason selected by the user + + * `feedback_details` \- (Optional) Additional details provided by the user @@ -260,4 +276,6 @@ The following is an example of feedback logs: - "conversationId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "researchId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "systemMessageId": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", - "userMessageId" : "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "status_code": "success", + "namespace": "default", + "user_type": "ADMIN_PRO", + "conversation_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "system_message_id": "a11b2bbc-c123-3abc-a12b-12a34b5c678d", + "user_message_id" : "a11b2bbc-c123-3abc-a12b-12a34b5c678d", @@ -267 +285,31 @@ The following is an example of feedback logs: - "rating" : "thumbsUp,thumbsDown,ease_of_use etc." + +## Agent/Research hours Logs + +This log type captures the usage logs for different agents within your Quick Suite account used for pricing: + + * `subscription_type` \- ENTERPRISE or PROFESSIONAL + + * `reporting_service` \- Service corresponding to the agent: RESEARCH, FLOWS OR AUTOMATIONS + + * `usage_group` \- `Included or Extra` based on the subscription type and usage so far + + * `usage_hours` \- Decimal value indicating the usage hours for the particular log instance + + * `service_resource_arn` \- ARN of the corresponding Agent’s service + + + + +The following is an example of Agent Hours logs: