AWS wellarchitected documentation change
Summary
Added guidance on introducing determinism to model responses through keyed options, reorganized response length control steps, and updated related content titles.
Security assessment
Changes focus on optimizing response length and cost efficiency through deterministic outputs. No security vulnerabilities, incidents, or security features are mentioned. The modifications are operational improvements without security implications.
Diff
diff --git a/wellarchitected/latest/generative-ai-lens/gencost03-bp02.md b/wellarchitected/latest/generative-ai-lens/gencost03-bp02.md index 6bef57f54..5b79b156a 100644 --- a//wellarchitected/latest/generative-ai-lens/gencost03-bp02.md +++ b//wellarchitected/latest/generative-ai-lens/gencost03-bp02.md @@ -20,0 +21,8 @@ Model response length should be kept as concise as possible, so long as it satis +In scenarios where a full-text response is unnecessary, consider introducing determinism to the model. You might instruct the model to evaluate its response against a set of keyed options, returning the key which maps to the model's response. For example: + +End of prompt template +--- +If after carefully evaluating all of the information available to you that you respond in the affirmative, simply respond with the word True. Otherwise, respond False, providing a detailed explanation for your decision. + +Such behavior as the one shown above encourages model responses to be succinct. Moreover, this behavior has the added benefit introducing determinism into the system for _True_ responses. + @@ -27 +35,7 @@ Model response length should be kept as concise as possible, so long as it satis - 3. Set a hard limit on the response length by configuring the response length hyperparameter accordingly. + 3. Introduce a response length control to limit response tokens. + + * Set a hard limit on the response length by configuring the response length hyperparameter accordingly. + + * Extend the prompt template to encourage deterministic responses. + + 4. Set a hard limit on the response length by configuring the response length hyperparameter accordingly. @@ -29 +43 @@ Model response length should be kept as concise as possible, so long as it satis - 4. Continue testing and optimizing the model's response to verify it satisfies the workload requirements. + 5. Continue testing and optimizing the model's response to verify it satisfies the workload requirements. @@ -36 +50 @@ Model response length should be kept as concise as possible, so long as it satis -**Related practices:** +**Related best practices:** @@ -43 +57 @@ Model response length should be kept as concise as possible, so long as it satis -**Related guides, videos, and documentation:** +**Related documents:** @@ -63 +77 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -GENCOST03-BP01 Reduce prompt token length +GENCOST03-BP01 Optimize prompt token length @@ -65 +79 @@ GENCOST03-BP01 Reduce prompt token length -Cost-informed vector stores +GENCOST03-BP03 Implement prompt caching to reduce token costs