AWS bedrock-agentcore documentation change

Service: bedrock-agentcore · 2026-06-16 · Documentation low

File: bedrock-agentcore/latest/devguide/ab-testing-config-bundle.md

Summary

Added example section for A/B testing tool descriptions with configuration bundles

Security assessment

Added documentation about non-security feature (A/B testing workflow). No security mechanisms or vulnerabilities addressed.

Diff

diff --git a/bedrock-agentcore/latest/devguide/ab-testing-config-bundle.md b/bedrock-agentcore/latest/devguide/ab-testing-config-bundle.md
index 0ebe33feb..573731e7c 100644
--- a//bedrock-agentcore/latest/devguide/ab-testing-config-bundle.md
+++ b//bedrock-agentcore/latest/devguide/ab-testing-config-bundle.md
@@ -7 +7 @@
-Step 1: Create the projectStep 2: Add the runtimeStep 3: Update the agent code and deployStep 4: Create configuration bundlesStep 5: Create an online evaluation configurationStep 6: Create the A/B testStep 7: Send traffic through the AgentCore GatewayStep 8: Get resultsStep 9: Confirm results and stop the A/B testStep 10: Deploy the winnerTroubleshooting
+Step 1: Create the projectStep 2: Add the runtimeStep 3: Update the agent code and deployStep 4: Create configuration bundlesStep 5: Create an online evaluation configurationStep 6: Create the A/B testStep 7: Send traffic through the AgentCore GatewayStep 8: Get resultsStep 9: Confirm results and stop the A/B testStep 10: Deploy the winnerExample: A/B testing tool descriptionsTroubleshooting
@@ -477,0 +478,89 @@ After deploying the winner:
+## Example: A/B testing tool descriptions
+
+You can use the same configuration bundle pattern to test optimized tool descriptions. Unlike system prompt A/B tests where the agent reads the bundle directly, tool description overrides are applied by the AgentCore Gateway. When the agent calls `tools/list` through the gateway, the gateway reads the configuration bundle and returns tool descriptions with overrides applied. No agent code changes are needed.
+
+For details on how the gateway applies tool description overrides, see [Behavior on MCP targets](./gateway-rules-propagation.html#gateway-rules-propagation-mcp).
+
+### Configuration bundles
+
+Control bundle — current tool descriptions:
+    
+    
+    agentcore add config-bundle \
+      --name toolDescControl \
+      --components '{
+        "arn:aws:bedrock-agentcore:us-west-2:123456789012:runtime/csAgent-abc123": {
+          "configuration": {
+            "tools": {
+              "lookup_order": {
+                "description": "Look up an order by ID."
+              },
+              "initiate_return": {
+                "description": "Initiate a return for an order."
+              },
+              "apply_discount": {
+                "description": "Apply a discount to an order."
+              }
+            }
+          }
+        }
+      }'
+    
+    agentcore deploy
+
+Treatment bundle — optimized tool descriptions from a recommendation:
+    
+    
+    agentcore add config-bundle \
+      --name toolDescTreatment \
+      --components '{
+        "arn:aws:bedrock-agentcore:us-west-2:123456789012:runtime/csAgent-abc123": {
+          "configuration": {
+            "tools": {
+              "lookup_order": {
+                "description": "Look up order details including status, item, and total by order ID. Use when the customer asks about an order or references an order number."
+              },
+              "initiate_return": {
+                "description": "Start a return process for an order. Use only when the customer explicitly requests a return or exchange, not for order status inquiries."
+              },
+              "apply_discount": {
+                "description": "Apply a percentage discount to an order. Use when compensating for service issues such as delivery delays. Requires a reason."
+              }
+            }
+          }
+        }
+      }'
+    
+    agentcore deploy
+
+### How it works
+
+  1. When the agent calls `tools/list` through the gateway (MCP targets), the A/B test assigns each session to a variant (control or treatment) on the gateway and resolves the corresponding configuration bundle.
+
+  2. Gateway reads the configuration bundle and returns tool descriptions with overrides applied.
+
+  3. The agent uses the returned descriptions for tool selection — no agent code changes required.
+
+
+
+
+### Create the A/B test
+    
+    
+    agentcore add ab-test \
+      --mode config-bundle \
+      --name toolDescTest \
+      --runtime csAgent \
+      --control-bundle toolDescControl \
+      --control-version <control-bundle-version-id> \
+      --treatment-bundle toolDescTreatment \
+      --treatment-version <treatment-bundle-version-id> \
+      --online-eval customerSupportEval \
+      --control-weight 80 \
+      --treatment-weight 20 \
+      --enable
+    
+    agentcore deploy
+
+The remaining steps (send traffic, get results, deploy the winner) are identical to the preceding system prompt example.
+