AWS AmazonS3 documentation change
Summary
Added documentation for pagination behavior when topK exceeds page size, including nextToken handling, SDK paginators, token validity, and limitations on concurrent writes during queries. Updated scale from 'millions' to 'billions' of vectors.
Security assessment
Changes describe query result pagination mechanics and performance scaling. No security vulnerabilities, encryption, access controls, or vulnerabilities are mentioned or addressed.
Diff
diff --git a/AmazonS3/latest/userguide/s3-vectors-query.md b/AmazonS3/latest/userguide/s3-vectors-query.md index b29047171..7d359d366 100644 --- a//AmazonS3/latest/userguide/s3-vectors-query.md +++ b//AmazonS3/latest/userguide/s3-vectors-query.md @@ -11 +11 @@ You can run a similarity query with the [QueryVectors](https://docs.aws.amazon.c -In the response, the vector keys are returned by default. You can optionally include the distance and metadata in the response. +In the response, the vector keys are returned by default. You can optionally include the distance and metadata in the response. When `topK` exceeds the page size, results are returned across multiple pages. The response includes a `nextToken` that you use to retrieve the next page. Continue making `QueryVectors` requests with the same `queryVector`, `topK`, and `filter` parameters, passing the `nextToken` from each response, until `nextToken` is no longer present. Alternatively, you can use the built-in paginators in the AWS SDKs to automatically iterate through all pages without manually handling `nextToken`. Pagination tokens remain valid for several minutes after the query is executed. If a pagination token expires, re-issue the original query to start a new session. Writes to the vector index between page retrievals will not be reflected in that query session. For specific limits on `topK` and page size, see [Limitations and restrictions](./s3-vectors-limitations.html). @@ -17 +17 @@ Furthermore, the open-source Amazon S3 Vectors Embed CLI tool provides a simplif -S3 Vectors delivers sub-second response times for cold queries, leveraging Amazon S3 elastic throughput to efficiently search across millions of vectors. This makes it highly cost-effective for workloads with infrequent queries. For warm queries, S3 Vectors can deliver response times as low as 100ms, benefiting workloads with repeated or frequent query patterns. +S3 Vectors delivers sub-second response times for cold queries, leveraging Amazon S3 elastic throughput to efficiently search across billions of vectors. This makes it highly cost-effective for workloads with infrequent queries. For warm queries, S3 Vectors can deliver response times as low as 100ms, benefiting workloads with repeated or frequent query patterns. When query results are returned across multiple pages, subsequent pages can be accessed immediately. @@ -68,0 +69,31 @@ SDK for Python + # Paginated query with manual nextToken handling. + query_params = dict( + vectorBucketName="media-embeddings", + indexName="movies", + queryVector={"float32": embedding}, + topK=500, + returnDistance=True, + returnMetadata=True, + ) + response = s3vectors.query_vectors(**query_params) + for vector in response["vectors"]: + print(vector) + while response.get("nextToken") is not None: + response = s3vectors.query_vectors(**query_params, nextToken=response["nextToken"]) + for vector in response["vectors"]: + print(vector) + + # Use the built-in paginator to automatically iterate through all pages. + paginator = s3vectors.get_paginator("query_vectors") + + for page in paginator.paginate( + vectorBucketName="media-embeddings", + indexName="movies", + queryVector={"float32": embedding}, + topK=500, + returnDistance=True, + returnMetadata=True, + ): + for vector in page["vectors"]: + print(vector) +