AWS sagemaker documentation change
Summary
Added four new metrics (MidStreamErrors, FirstChunkLatency, FirstChunkModelLatency, FirstChunkOverheadLatency) for monitoring streaming inference requests
Security assessment
While monitoring improvements can aid operational security, these metrics focus on performance measurement rather than directly addressing security vulnerabilities or documenting security controls.
Diff
diff --git a/sagemaker/latest/dg/monitoring-cloudwatch.md b/sagemaker/latest/dg/monitoring-cloudwatch.md index ec038572a..62875659c 100644 --- a//sagemaker/latest/dg/monitoring-cloudwatch.md +++ b//sagemaker/latest/dg/monitoring-cloudwatch.md @@ -97,0 +98,4 @@ Endpoint invocation metrics Metric | Description +`MidStreamErrors` | The number of errors that occur during response streaming after the initial response has been sent to the customer. Units: None Valid statistics: Average, Sum +`FirstChunkLatency` | The time elapsed from when the request arrives at SageMaker AI endpoint until the first chunk of the response is sent to the customer. This metric applies to bidirectional streaming inference requests. Units: Microseconds Valid statistics: Average, Sum, Min, Max, Sample Count, Percentiles +`FirstChunkModelLatency` | The time taken by the model container to process the request and return the first chunk of the response. This is measured from when the request is sent to the model container until the first byte is received from the model. This metric applies to bidirectional streaming inference requests. Units: Microseconds Valid statistics: Average, Sum, Min, Max, Sample Count, Percentiles +`FirstChunkOverheadLatency` | The overhead latency for the first chunk, excluding model processing time. This is calculated as `FirstChunkLatency` minus `FirstChunkModelLatency`, representing the time spent in routing, preprocessing, and postprocessing operations within SageMaker AI platform. Overhead latency can vary depending on multiple factors, including request frequency, load and authentication/authorization of the request. This metric applies to bidirectional streaming inference requests. Units: Microseconds Valid statistics: Average, Sum, Min, Max, Sample Count, Percentile