AWS wellarchitected medium security documentation change
Summary
Expanded guidance on implementing prompt catalogs with detailed governance, version control, testing frameworks, and security controls. Added specific requirements for access control, audit trails, and backup processes.
Security assessment
The change introduces security controls including access control requirements, audit trails, and backup processes. These additions directly address security concerns by implementing governance and protection mechanisms for sensitive AI assets.
Diff
diff --git a/wellarchitected/latest/generative-ai-lens/genrel04-bp01.md b/wellarchitected/latest/generative-ai-lens/genrel04-bp01.md index 66f9f7c69..bb9a13064 100644 --- a//wellarchitected/latest/generative-ai-lens/genrel04-bp01.md +++ b//wellarchitected/latest/generative-ai-lens/genrel04-bp01.md @@ -19 +19 @@ Prompt catalogs store and manage prompts and prompt versions. They act as a reli -Prompt catalogs function as a centralized system for developing, testing, and managing prompts. Customers should implement a prompt catalog to maintain different version of prompts. Prompts should be released to a live version once passing the appropriate testing thresholds and benchmarks. In the case where a prompt results in unexpected or undesirable behavior, a prompt catalog enables the ability to roll back to the previous version. +Prompt catalogs function as a centralized system for developing, testing, and managing prompts. Implement a prompt catalog to maintain different versions of prompts. Prompts should be released to a live version once passing the appropriate testing thresholds and benchmarks. In the case where a prompt results in unexpected or undesirable behavior, a prompt catalog enables the ability to roll back to the previous version. @@ -21 +21 @@ Prompt catalogs function as a centralized system for developing, testing, and ma -Amazon Bedrock Prompt Management helps customers maintain prompts and prompt versions. Additionally, Prompt Management through Amazon Bedrock maintains versioned information on hyperparameter rangers for a prompt. Prompt behavior can change drastically when tuning hyperparameters such as temperature, top_p, or top_k. Value ranges for these hyperparameters should be paired with and validated against prompt versions as part of the prompt engineering process. +Additionally, maintain versioned information on hyperparameter ranges for prompts. Prompt behavior can change drastically when tuning hyperparameters such as temperature, top_p, or top_k. Value ranges for these hyperparameters should be paired with and validated against prompt versions as part of the prompt engineering process. @@ -23 +23 @@ Amazon Bedrock Prompt Management helps customers maintain prompts and prompt ver -Prompt catalogs should maintain test results for a prompt against several model versions. A given foundation model can have several versions, and prompt test results for each model version can vary accordingly. Consider using Bedrock Prompt Management or a similar capability to maintain prompt versions for each of the available models. +Prompt catalogs should maintain test results for a prompt against several model versions. A given foundation model can have several versions, and prompt test results for each model version can vary accordingly. Consider developing a catalog that maintains prompt versions for each of the available models. @@ -27 +27 @@ Prompt catalogs should maintain test results for a prompt against several model - 1. Create a prompt, and identify variables and hyperparameter ranges to test. + 1. Design catalog structure: @@ -29 +29 @@ Prompt catalogs should maintain test results for a prompt against several model - 2. Test the prompt against several models and model versions. + * Define prompt metadata schema (like version, author, and purpose) @@ -31 +31 @@ Prompt catalogs should maintain test results for a prompt against several model - 3. Publish the best performing prompt for the given model for use in your application stack. + * Create categorization system for different prompt types @@ -33 +33,45 @@ Prompt catalogs should maintain test results for a prompt against several model - * Integrate prompt versions into your application CI/CD process to maintain continual performance evaluation and tracking. + * Establish naming conventions and tagging standards + + * Define access control requirements + + 2. Implement version control: + + * Set up version tracking for prompts + + * Create changelog management process + + * Define rollback procedures + + * Establish backup and recovery processes + + 3. Create testing framework: + + * Define success criteria for prompts + + * Establish validation procedures + + * Create test suites for different use cases + + * Set up automated testing pipelines + + 4. Configure prompt metadata: + + * Document hyperparameter ranges + + * Track performance metrics + + * Record model compatibility + + * Maintain usage statistics + + 5. Establish governance processes: + + * Define approval workflows + + * Create audit trails + + * Set up review procedures + + * Implement quality controls + + * Codify in your organizations AI usage or policy document @@ -40 +84 @@ Prompt catalogs should maintain test results for a prompt against several model -**Related practices:** +**Related best practices:** @@ -51 +95 @@ Prompt catalogs should maintain test results for a prompt against several model -**Related guides, videos, and documentation:** +**Related documents:**