AWS AmazonS3 documentation change
Summary
Added new section 'Optimizing for high-request rate workloads' with strategies for handling high request rates, including prefix distribution, retry logic, monitoring, and S3 Express One Zone recommendation
Security assessment
The changes focus on performance optimization and scaling strategies. While they mention handling HTTP 503 errors and retries, these are related to throttling and system capacity rather than security vulnerabilities. No security vulnerabilities or security-specific features are addressed.
Diff
diff --git a/AmazonS3/latest/userguide/optimizing-performance-design-patterns.md b/AmazonS3/latest/userguide/optimizing-performance-design-patterns.md index 588bcb2ee..2989e4799 100644 --- a//AmazonS3/latest/userguide/optimizing-performance-design-patterns.md +++ b//AmazonS3/latest/userguide/optimizing-performance-design-patterns.md @@ -5 +5 @@ -Caching frequently accessed contentTimeouts and retries for latency-sensitive appsHorizontal scaling and request parallelizationOptimizing for high-request rate workloads Accelerating geographically disparate data transfers +Caching frequently accessed contentTimeouts and retries for latency-sensitive appsHorizontal scaling and request parallelizationOptimizing for high-request rate workloads Accelerating geographically disparate data transfersOptimizing for high-request rate workloads @@ -24,0 +25,2 @@ To optimize performance, you can use the following design patterns. + * Optimizing for high-request rate workloads + @@ -75,0 +78,23 @@ You can set up Transfer Acceleration on new or existing buckets. You can use a s +## Optimizing for high-request rate workloads + +Applications that generate high request rates to Amazon S3 require specific design patterns to achieve optimal performance. When your application consistently generates more than 3,500 PUT/COPY/POST/DELETE or 5,500 GET/HEAD requests per second per prefix, you should implement strategies to distribute requests and handle scaling behavior. + +Amazon S3 automatically scales to accommodate higher request rates, but this scaling happens gradually. During the scaling process, you might receive HTTP 503 (Slow Down) responses. These responses are temporary and indicate that Amazon S3 is optimizing its internal systems for your new request pattern. Once scaling is complete, your requests will be served without throttling. + +To optimize performance for high-request rate workloads, consider the following strategies: + + * **Distribute requests across multiple prefixes** – Use a randomized or sequential prefix pattern to spread requests across multiple partitions. For example, instead of using sequential object names like `log-2024-01-01.txt`, use randomized prefixes like `a1b2/log-2024-01-01.txt`. This helps Amazon S3 distribute the load more effectively. + + * **Implement exponential backoff for 503 errors** – When you receive HTTP 503 responses, implement retry logic with exponential backoff. Start with a short delay and gradually increase the wait time between retries. The AWS SDKs include built-in retry logic that handles this automatically. + + * **Monitor request patterns** – Use Amazon CloudWatch metrics to monitor your request rates and error rates. Pay particular attention to 5xx error metrics, which can indicate when your application is approaching or exceeding current scaling limits. + + * **Gradually ramp up request rates** – When launching new applications or significantly increasing request rates, gradually increase your traffic over time rather than immediately jumping to peak rates. This allows Amazon S3 to scale proactively and reduces the likelihood of throttling. + + * **Use multiple connections** – Distribute your requests across multiple HTTP connections to maximize throughput and reduce the impact of any single connection issues. + + + + +For applications that require consistent high performance, consider using Amazon S3 Express One Zone, which is designed for applications that require single-digit millisecond latencies and can support hundreds of thousands of requests per second. For more information, see [S3 Express One Zone](./directory-bucket-high-performance.html#s3-express-one-zone). +