AWS bedrock documentation change
Summary
Added documentation for Amazon S3 Vectors integration including setup instructions, metadata management, encryption options, and compatibility notes
Security assessment
The changes add security documentation about encryption options (SSE-S3/SSE-KMS) and IAM permissions requirements, but there's no evidence of addressing a specific security vulnerability. The encryption guidance and permission requirements represent standard security feature documentation rather than a response to a security incident.
Diff
diff --git a/bedrock/latest/userguide/knowledge-base-setup.md b/bedrock/latest/userguide/knowledge-base-setup.md index 7268ea2c5..eedb6242b 100644 --- a//bedrock/latest/userguide/knowledge-base-setup.md +++ b//bedrock/latest/userguide/knowledge-base-setup.md @@ -7 +7 @@ -To store the vector embeddings that your documents are converted to, you use a vector store. If you prefer for Amazon Bedrock to automatically create a vector index in Amazon OpenSearch Serverless for you, skip this prerequisite and proceed to [Create a knowledge base by connecting to a data source in Amazon Bedrock Knowledge Bases](./knowledge-base-create.html). +To store the vector embeddings that your documents are converted to, you use a vector store. Amazon Bedrock Knowledge Bases supports a quick-create flow for some of the vector stores, so if you prefer for Amazon Bedrock to automatically create a vector index for you in one of those vector stores, skip this prerequisite and proceed to [Create a knowledge base by connecting to a data source in Amazon Bedrock Knowledge Bases](./knowledge-base-create.html). @@ -31,0 +32,4 @@ Select the tab corresponding to the vector store service that you will use to cr +###### Note + +Your choice of embeddings model and vector dimensions can affect the available vector store choices. If you are not able to use your preferred vector store, choose compatible options the embeddings model and vector dimensions. + @@ -205,0 +210,88 @@ Metadata management (second mapping field) | Bedrock-managed metadata field | me +Amazon S3 Vectors + + +Amazon S3 Vectors provides cost-effective vector storage in Amazon S3 that can be used to store and query vector data. It provides durable and elastic storage of large vector datasets with sub-second query performance. Amazon S3 Vectors is best suited for infrequent query workloads, and can help reduce costs when used in retrieval augmented generation (RAG) and semantic search applications. + +###### Important + +The Amazon S3 Vectors integration with Amazon Bedrock Knowledge Bases is in preview release and is subject to change. + +Amazon S3 Vectors introduces S3 vector buckets, which you can query based on semantic meaning and similarity. It can be used to deliver sub-second query response times and reduce costs while storing, accessing, and querying vector data at scale without provisioning any infrastructure. Inside a vector bucket, you can organize your vector data within vector indexes. Your vector bucket can have multiple vector indexes, and each vector index can hold millions of vectors. For more information, see [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html) in the _Amazon S3 User Guide_. + +###### Note + +You can create a knowledge base for Amazon S3 Vectors in all AWS Regions where both Amazon Bedrock and Amazon S3 Vectors are available. For information about regional availability of Amazon S3 Vectors, see [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html) in the _Amazon S3 User Guide_. + +###### Metadata support + +After creating a vector index, when adding vector data to the index, you can attach metadata as key-value pairs to each vector. By default, all metadata attached to a vector is filterable and can be used as filters in a similarity search query. The filterable metadata can be used to filter incoming queries based on a set of conditions, such as dates, categories, or user preferences. + +You can also configure the metadata to be non-filterable when creating the vector index. Amazon S3 vector indexes support string, boolean, and number types. It can support up to a maximum of 40 KB of metadata for each vector. Within this 40 KB of metadata, the filterable metadata can be up to a maximum of 2 KB for each vector. The filterable metadata space can be used to store the embeddings after the knowledge base has been created. + +If the metadata exceeds any of these limits, it results in an error when creating the vector index. For more information, see [Amazon S3 Vectors](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors.html) in the _Amazon S3 User Guide_. + +###### Required permissions + +Make sure that your IAM policy allows Amazon Bedrock to access your vector index in S3 vector bucket. For more information about the required permissions, see [Create a service role for Amazon Bedrock Knowledge Bases](./kb-permissions.html). + +###### Create S3 vector bucket and index + +To use Amazon S3 Vectors with your knowledge base, you need to create an S3 vector bucket and a vector index. You can create a vector bucket and index using the Amazon S3 console, AWS CLI, or AWS SDK. For detailed instructions, see [Create a vector index](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-index-create.html) in the _Amazon S3 User Guide_. + +Note the following considerations when creating your vector bucket and index in the [Amazon S3 console](https://console.aws.amazon.com/s3/vector-buckets#). + + 1. When creating your S3 vector bucket, take note of the following considerations. + + * Provide a unique **Vector bucket name**. + + * (Optional) Amazon S3 will automatically encrypt the data using the default **Server-side encryption with Amazon S3 managed keys (SSE-S3)**. You can choose whether to use this default encryption, or the **Server-side encryption with AWS Key Management Service keys (SSE-KMS)** instead. + +###### Note + +The encryption type can't be changed once the vector bucket has been created. + +For step-by-step instructions, see [Encryption with AWS KMS keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-bucket-encryption.html). + + 2. Once you've created the S3 vector bucket, take note of the **Amazon Resource Name (ARN)** of the vector bucket for when you create the knowledge base. + + 3. Choose the vector bucket that you created and then create a vector index. When creating the vector index, take note of the following considerations. + + * **Vector index name** – Provide a name for the field (for example, `embeddings`). + + * **Dimension** – The number of dimensions in the vector. The dimensions must be a value between 1 and 4096. Refer to the following table to determine how many dimensions the vector should contain based on your selection of the embeddings model: + +Model | Dimensions +---|--- +Titan G1 Embeddings - Text | 1,536 +Titan V2 Embeddings - Text | 1,024, 512, and 256 +Cohere Embed English | 1,024 +Cohere Embed Multilingual | 1,024 + + * ###### Note + +Amazon S3 Vectors only support floating-point embeddings. Binary embeddings are not supported. + +**Distance metric** – The metric used to measure the similarity between vectors. You can use **Cosine** or **Euclidean**. + + 4. Expand the **Additional settings** and provide any non-filterable metadata in the **Non-filterable metadata** field. + +###### Note + +If you expect your text chunks to exceed the 2 KB metadata space, we recommend that you add the text field `AMAZON_BEDROCK_TEXT` as a non-filterable metadata key. Your knowledge base will use this field to store the text chunks. + +You can configure up to a maximum of 10 non-filterable metadata keys. Choose **Add key** and then add `AMAZON_BEDROCK_TEXT` as a key. + + 5. Create the vector index and take note of the **Amazon Resource Name (ARN)** of the vector index for when you create the knowledge base. + + + + +###### Create knowledge base for S3 vector bucket + +After you've gathered this information, you can proceed to [create your knowledge base](./knowledge-base-create.html). When creating your knowledge base with S3 vector bucket, you'll need to provide the ARN of the vector bucket and the vector index. The vector index will store the embeddings that's generated from your data sources. The following table summarizes where you will enter each piece of information: + +Field | Corresponding field in knowledge base setup (Console) | Corresponding field in knowledge base setup (API) | Description +---|---|---|--- +Vector bucket ARN | S3 vector bucket ARN | vectorBucketArn | The Amazon Resource Name (ARN) of your S3 vector bucket. +Vector index ARN | S3 vector index ARN | vectorIndexARN | The Amazon Resource Name (ARN) of the vector index for your S3 vector bucket. +