AWS cli medium security documentation change
Summary
Restructured data source configuration documentation with added details about S3 bucket ownership, web crawler security controls, and authentication requirements. Added explicit security guidance for user-agent headers and URL crawling authorization.
Security assessment
Added security-relevant configurations including: 1) bucketOwnerAccountId field emphasizing S3 bucket ownership validation 2) webConfiguration requiring authorization to crawl URLs 3) userAgentHeader documentation about allowlisting specific crawler agents 4) explicit security warnings about preview features. These changes directly impact access control and authentication mechanisms.
Diff
diff --git a/cli/latest/reference/bedrock-agent/get-data-source.md b/cli/latest/reference/bedrock-agent/get-data-source.md index 9d83bacf1..098cd8c01 100644 --- a//cli/latest/reference/bedrock-agent/get-data-source.md +++ b//cli/latest/reference/bedrock-agent/get-data-source.md @@ -15 +15 @@ - * [AWS CLI 2.28.8 Command Reference](../../index.html) » + * [AWS CLI 2.28.11 Command Reference](../../index.html) » @@ -67 +66,0 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc - --data-source-id <value> @@ -68,0 +68 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc + --data-source-id <value> @@ -93,4 +92,0 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc -`--data-source-id` (string) - -> The unique identifier of the data source. - @@ -100,0 +97,4 @@ See also: [AWS API Documentation](https://docs.aws.amazon.com/goto/WebAPI/bedroc +`--data-source-id` (string) + +> The unique identifier of the data source. + @@ -204 +204 @@ dataSource -> (structure) -> createdAt -> (timestamp) +> knowledgeBaseId -> (string) @@ -206 +206 @@ dataSource -> (structure) ->> The time at which the data source was created. +>> The unique identifier of the knowledge base to which the data source belongs. @@ -208 +208 @@ dataSource -> (structure) -> dataDeletionPolicy -> (string) +> dataSourceId -> (string) @@ -210 +210,18 @@ dataSource -> (structure) ->> The data deletion policy for the data source. +>> The unique identifier of the data source. +> +> name -> (string) +> +>> The name of the data source. +> +> status -> (string) +> +>> The status of the data source. The following statuses are possible: +>> +>> * Available – The data source has been created and is ready for ingestion into the knowledge base. +>> * Deleting – The data source is being deleted. +>> + +> +> description -> (string) +> +>> The description of the data source. @@ -216 +233 @@ dataSource -> (structure) ->> confluenceConfiguration -> (structure) +>> type -> (string) @@ -218 +235,23 @@ dataSource -> (structure) ->>> The configuration information to connect to Confluence as your data source. +>>> The type of data source. +>> +>> s3Configuration -> (structure) +>> +>>> The configuration information to connect to Amazon S3 as your data source. +>>> +>>> bucketArn -> (string) +>>> +>>>> The Amazon Resource Name (ARN) of the S3 bucket that contains your data. +>>> +>>> inclusionPrefixes -> (list) +>>> +>>>> A list of S3 prefixes to include certain files or content. For more information, see [Organizing objects using prefixes](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-prefixes.html) . +>>>> +>>>> (string) +>>> +>>> bucketOwnerAccountId -> (string) +>>> +>>>> The account ID for the owner of the S3 bucket. +>> +>> webConfiguration -> (structure) +>> +>>> The configuration of web URLs to crawl for your data source. You should be authorized to crawl the URLs. @@ -222 +261 @@ dataSource -> (structure) ->>> Confluence data source connector is in preview release and is subject to change. +>>> Crawling web URLs as your data source is in preview release and is subject to change. @@ -224 +263 @@ dataSource -> (structure) ->>> crawlerConfiguration -> (structure) +>>> sourceConfiguration -> (structure) @@ -226 +265 @@ dataSource -> (structure) ->>>> The configuration of the Confluence content. For example, configuring specific types of Confluence content. +>>>> The source configuration details for the web data source. @@ -228 +267 @@ dataSource -> (structure) ->>>> filterConfiguration -> (structure) +>>>> urlConfiguration -> (structure) @@ -230 +269 @@ dataSource -> (structure) ->>>>> The configuration of filtering the Confluence content. For example, configuring regular expression patterns to include or exclude certain content. +>>>>> The configuration of the URL/URLs. @@ -232 +271 @@ dataSource -> (structure) ->>>>> patternObjectFilter -> (structure) +>>>>> seedUrls -> (list) @@ -234 +273 @@ dataSource -> (structure) ->>>>>> The configuration of filtering certain objects or content types of the data source. +>>>>>> One or more seed or starting point URLs. @@ -236 +275 @@ dataSource -> (structure) ->>>>>> filters -> (list) +>>>>>> (structure) @@ -238 +277 @@ dataSource -> (structure) ->>>>>>> The configuration of specific filters applied to your data source content. You can filter out or include certain content. +>>>>>>> The seed or starting point URL. You should be authorized to crawl the URL. @@ -240 +279 @@ dataSource -> (structure) ->>>>>>> (structure) +>>>>>>> url -> (string) @@ -242,21 +281 @@ dataSource -> (structure) ->>>>>>>> The specific filters applied to your data source content. You can filter out or include certain content. ->>>>>>>> ->>>>>>>> exclusionFilters -> (list) ->>>>>>>> ->>>>>>>>> A list of one or more exclusion regular expression patterns to exclude certain object types that adhere to the pattern. If you specify an inclusion and exclusion filter/pattern and both match a document, the exclusion filter takes precedence and the document isn’t crawled. ->>>>>>>>> ->>>>>>>>> (string) ->>>>>>>> ->>>>>>>> inclusionFilters -> (list) ->>>>>>>> ->>>>>>>>> A list of one or more inclusion regular expression patterns to include certain object types that adhere to the pattern. If you specify an inclusion and exclusion filter/pattern and both match a document, the exclusion filter takes precedence and the document isn’t crawled. ->>>>>>>>> ->>>>>>>>> (string) ->>>>>>>> ->>>>>>>> objectType -> (string) ->>>>>>>> ->>>>>>>>> The supported object type or content type of the data source. ->>>>> ->>>>> type -> (string) ->>>>> ->>>>>> The type of filtering that you want to apply to certain objects or content of the data source. For example, the `PATTERN` type is regular expression patterns you can apply to filter your content. +>>>>>>>> A seed or starting point URL. @@ -264 +283 @@ dataSource -> (structure) ->>> sourceConfiguration -> (structure) +>>> crawlerConfiguration -> (structure) @@ -266 +285 @@ dataSource -> (structure) ->>>> The endpoint information to connect to your Confluence data source. +>>>> The Web Crawler configuration details for the web data source. @@ -268 +287 @@ dataSource -> (structure) ->>>> authType -> (string) +>>>> crawlerLimits -> (structure) @@ -270 +289,9 @@ dataSource -> (structure) ->>>>> The supported authentication type to authenticate and connect to your Confluence instance. +>>>>> The configuration of crawl limits for the web URLs. +>>>>> +>>>>> rateLimit -> (integer) +>>>>> +>>>>>> The max rate at which pages are crawled, up to 300 per minute per host. +>>>>> +>>>>> maxPages -> (integer) +>>>>> +>>>>>> The max number of web pages crawled from your source URLs, up to 25,000 pages. If the web pages exceed this limit, the data source sync will fail and no web pages will be ingested. @@ -272 +299 @@ dataSource -> (structure) ->>>> credentialsSecretArn -> (string) +>>>> inclusionFilters -> (list) @@ -274 +301,3 @@ dataSource -> (structure) ->>>>> The Amazon Resource Name of an Secrets Manager secret that stores your authentication credentials for your Confluence instance URL. For more information on the key-value pairs that must be included in your secret, depending on your authentication type, see [Confluence connection configuration](https://docs.aws.amazon.com/bedrock/latest/userguide/confluence-data-source-connector.html#configuration-confluence-connector) . +>>>>> A list of one or more inclusion regular expression patterns to include certain URLs. If you specify an inclusion and exclusion filter/pattern and both match a URL, the exclusion filter takes precedence and the web content of the URL isn’t crawled. +>>>>> +>>>>> (string) @@ -276 +305 @@ dataSource -> (structure) ->>>> hostType -> (string) +>>>> exclusionFilters -> (list) @@ -278 +307,3 @@ dataSource -> (structure) ->>>>> The supported host type, whether online/cloud or server/on-premises. +>>>>> A list of one or more exclusion regular expression patterns to exclude certain URLs. If you specify an inclusion and exclusion filter/pattern and both match a URL, the exclusion filter takes precedence and the web content of the URL isn’t crawled. +>>>>> +>>>>> (string) @@ -280 +311 @@ dataSource -> (structure) ->>>> hostUrl -> (string) +>>>> scope -> (string) @@ -282 +313,11 @@ dataSource -> (structure) ->>>>> The Confluence host URL or instance URL. +>>>>> The scope of what is crawled for your URLs. +>>>>> +>>>>> You can choose to crawl only web pages that belong to the same host or primary domain. For example, only web pages that contain the seed URL “<https://docs.aws.amazon.com/bedrock/latest/userguide/>” and no other domains. You can choose to include sub domains in addition to the host or primary domain. For example, web pages that contain “aws.amazon.com” can also include sub domain “docs.aws.amazon.com”. +>>>> +>>>> userAgent -> (string) +>>>> +>>>>> Returns the user agent suffix for your web crawler. +>>>> +>>>> userAgentHeader -> (string) +>>>> +>>>>> A string used for identifying the crawler or bot when it accesses a web server. The user agent header value consists of the `bedrockbot` , UUID, and a user agent suffix for your crawler (if one is provided). By default, it is set to `bedrockbot_UUID` . You can optionally append a custom suffix to `bedrockbot_UUID` to allowlist a specific user agent permitted to access your source URLs. @@ -284 +325 @@ dataSource -> (structure) ->> s3Configuration -> (structure) +>> confluenceConfiguration -> (structure) @@ -286,5 +327 @@ dataSource -> (structure) ->>> The configuration information to connect to Amazon S3 as your data source. ->>> ->>> bucketArn -> (string) ->>> ->>>> The Amazon Resource Name (ARN) of the S3 bucket that contains your data. +>>> The configuration information to connect to Confluence as your data source. @@ -292 +329 @@ dataSource -> (structure) ->>> bucketOwnerAccountId -> (string) +>>> ### Note @@ -294 +331 @@ dataSource -> (structure) ->>>> The account ID for the owner of the S3 bucket. +>>> Confluence data source connector is in preview release and is subject to change. @@ -296 +333 @@ dataSource -> (structure)