AWS prescriptive-guidance documentation change
Summary
Refined test data guidance and added reference to Amazon Bedrock Guardrails documentation
Security assessment
Added link to Bedrock Guardrails documentation, which provides security controls for AI systems. While not addressing a specific vulnerability, it expands security-related guidance.
Diff
diff --git a/prescriptive-guidance/latest/generative-ai-nlp-healthcare/evaluation.md b/prescriptive-guidance/latest/generative-ai-nlp-healthcare/evaluation.md index 41e3e1338..e424f2f63 100644 --- a//prescriptive-guidance/latest/generative-ai-nlp-healthcare/evaluation.md +++ b//prescriptive-guidance/latest/generative-ai-nlp-healthcare/evaluation.md @@ -19 +19 @@ Medical NLP tasks commonly use medical corpora (such as PubMed) or patient infor - * When using a pretrained LLM solution, make sure that you have an adequate amount of test data. The test data should be an exact match or closely resemble the actual medical data. Depending on the task, this can range from 20 to more than 100 records. + * When using a pretrained LLM solution, make sure that you have an adequate amount of test data. The test data should closely resemble the actual medical data. Depending on the task, this can range from 20 to more than 100 records. @@ -21 +21 @@ Medical NLP tasks commonly use medical corpora (such as PubMed) or patient infor - * When fine-tuning an LLM, collect a sufficient number of labeled (ground truth) records from a variety of SMEs of the targeted medical domain. A general starting point is at least 100 high-quality records, and we recommend no more than 20 records from each SME. However, given the complexity of the task and your accuracy acceptance criteria, more records might be required. + * When fine-tuning an LLM, collect a sufficient number of labeled (ground truth) records from a variety of SMEs of the targeted medical domain. A general starting point is at least 100 high-quality records. However, given the complexity of the task and your accuracy acceptance criteria, more records might be required. @@ -28 +28 @@ Medical NLP tasks commonly use medical corpora (such as PubMed) or patient infor -Many AI research and development companies, such as Anthropic, have already implemented guardrails in their foundation models to avoid toxicity. You can use toxicity detection to check input prompts and the output responses from LLMs. For more information, see [Toxicity detection](https://docs.aws.amazon.com/comprehend/latest/dg/trust-safety.html#toxicity-detection) in the Amazon Comprehend documentation. +Many AI research and development companies, such as Anthropic, have already implemented guardrails in their foundation models to avoid toxicity. You can use toxicity detection to check input prompts and the output responses from LLMs. For more information, see [Toxicity detection](https://docs.aws.amazon.com/comprehend/latest/dg/trust-safety.html#toxicity-detection) in the Amazon Comprehend documentation and see [Guardrails](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) in the Amazon Bedrock documentation.