AWS prescriptive-guidance documentation change
Summary
Comprehensive rewrite expanding security guidance for AI workloads, adding sections on multi-account architecture, defense-in-depth strategies, model risks, application layer risks, data governance, and model evaluation.
Security assessment
The changes significantly enhance security documentation by detailing AI-specific threats (prompt injection, data exfiltration, model manipulation) and corresponding AWS mitigations (Guardrails, WAF, Macie, KMS). However, there's no evidence of addressing a specific existing vulnerability or incident - this is proactive guidance expansion.
Diff
diff --git a/prescriptive-guidance/latest/security-reference-architecture-generative-ai/gen-ai-model-inference.md b/prescriptive-guidance/latest/security-reference-architecture-generative-ai/gen-ai-model-inference.md index 403899680..492c4b61e 100644 --- a//prescriptive-guidance/latest/security-reference-architecture-generative-ai/gen-ai-model-inference.md +++ b//prescriptive-guidance/latest/security-reference-architecture-generative-ai/gen-ai-model-inference.md @@ -3 +3 @@ -[Documentation](/index.html)[AWS Prescriptive Guidance](https://aws.amazon.com/prescriptive-guidance/)[AWS Security Reference Architecture (AWS SRA) – generative AI](introduction.html) +[Documentation](/index.html)[AWS Prescriptive Guidance](https://aws.amazon.com/prescriptive-guidance/)[AWS Security Reference Architecture (AWS SRA) – AI security](introduction.html) @@ -5 +5 @@ -RationaleSecurity considerationsRemediationsRecommended AWS services +RationaleSecurity considerationsMulti-account architecture for AI workloadsDefense in depthModel evaluation and validation @@ -9 +9 @@ RationaleSecurity considerationsRemediationsRecommended AWS services -The following architecture diagram illustrates the AWS services recommended for the Generative AI account for this capability. The scope of this capability is to give users access to foundation models (FMs) for chat and image generation. +Organizations building AI-powered applications must understand the fundamental differences between traditional AI systems and generative AI foundation models (FMs). Traditional AI systems perform classification, prediction, or optimization tasks with consistent outputs. Generative AI creates new content (text, images, code, or other media) based on learned patterns from training data. FMs are large-scale neural networks trained on vast datasets that generate probabilistic outputs, meaning identical inputs can produce different responses across invocations. This non-deterministic behavior requires security architectures that account for output variability while maintaining consistent protection. @@ -11,3 +11 @@ The following architecture diagram illustrates the AWS services recommended for - - -The Generative AI account is dedicated to securing generative AI functionality through the use of Amazon Bedrock. We will build out this account (and the architecture diagram) with functionality throughout this guide. The account includes services for storing conversations for users and maintaining a prompt store. The account also includes security services to implement security guardrails and centralized security governance. Users can gain federated access by using an identity provider (IdP) to securely access a virtual private cloud (VPC) in the Generative AI account. AWS PrivateLink supports private connectivity from your VPC to Amazon Bedrock endpoint services. You should create an Amazon S3 gateway endpoint for the model invocation logs and prompt store bucket in Amazon S3 that the VPC environment is configured to access. You should also create an Amazon CloudWatch Logs gateway endpoint for the CloudWatch logs that the VPC environment is configured to access. +Building applications that integrate generative AI FMs and agent capabilities enables advanced functionality, including natural language processing (NLP), image generation, automated reasoning, and intelligent decision support. This integration drives organizational innovation by allowing developers to build solutions that improve productivity and competitive positioning. However, the probabilistic nature of AI outputs demands security controls that function effectively regardless of model response variability. @@ -17 +15 @@ The Generative AI account is dedicated to securing generative AI functionality t -Granting users access to generative AI FMs enables them to use advanced models for tasks such as natural language processing, image generation, and enhancing efficiency and decision making. This access fosters innovation within an organization because employees can experiment with new applications and develop cutting-edge solutions, which ultimately improves productivity and provides competitive advantages. This use case corresponds to Scope 3 of the [Generative AI Security Scoping Matrix](https://aws.amazon.com/blogs/security/securing-generative-ai-an-introduction-to-the-generative-ai-security-scoping-matrix/). In Scope 3, your organization builds a generative AI application by using a pre-trained FM, such as those offered in Amazon Bedrock. In this scope, you control your application and any customer data used by your application, whereas the FM provider controls the pre-trained model and its training data. For data flows pertaining to various application scopes and information about the shared responsibility between you and the FM provider, see the AWS blog post [Securing generative AI: Applying relevant security controls](https://aws.amazon.com/blogs/security/securing-generative-ai-applying-relevant-security-controls/). +This use case corresponds to Scope 3 of the [Generative AI Security Scoping Matrix](https://aws.amazon.com/blogs/security/securing-generative-ai-an-introduction-to-the-generative-ai-security-scoping-matrix/). In Scope 3, your organization builds an application or feature that integrates generative AI by using pre-trained FMs, such as those offered on Amazon Bedrock. You control your application and any customer data used by your application, whereas the FM provider controls the pre-trained model and its training data. For data flows pertaining to various application scopes and information about the shared responsibility between you and the FM provider, see [Securing generative AI: Applying relevant security controls](https://aws.amazon.com/blogs/security/securing-generative-ai-applying-relevant-security-controls/) (AWS blog post). @@ -19 +17 @@ Granting users access to generative AI FMs enables them to use advanced models f -When you give users access to the generative AI FMs in Amazon Bedrock, you should address these key security considerations: +Organizations can also implement custom AI solutions using [Amazon SageMaker AI](https://docs.aws.amazon.com/sagemaker/latest/dg/whatis.html) for model development, training, and deployment. This approach introduces additional security considerations including secure model development environments, protection of training data and model artifacts, and governance of the entire machine learning lifecycle. @@ -21 +19 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul - * Secure access to the model invocation, conversation history, and prompt store +Custom models require enhanced monitoring for model drift, bias detection, and performance degradation that could indicate security issues or model compromise. When you customize FMs with your own training data (Scope 4) or train models from scratch (Scope 5), training data security becomes critical. Malicious or poisoned training data can compromise model behavior, introduce bias, or cause models to leak sensitive information during inference. For detailed guidance on securing model customization and training data, see [Capability 2](./gen-ai-rag.html). @@ -23 +21 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul - * Encryption of conversations and the prompt store +The security architecture must address both the non-deterministic nature of AI systems and the autonomous capabilities of AI agents. The security architecture must implement layered defenses that maintain effectiveness across the spectrum of possible AI behaviors and outputs. @@ -25 +23 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul - * Monitoring for potential security risks such as prompt injection or sensitive information disclosure +## Security considerations @@ -26,0 +25 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul +AI workloads introduce unique attack vectors and operational risks that traditional security controls don't address. Unlike conventional applications with predictable input-output relationships, AI systems process natural language and generate probabilistic responses that attackers can influence through carefully crafted inputs. @@ -27,0 +27 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul +### Model-specific risk @@ -28,0 +29 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul +These risks target the AI model itself, exploiting the probabilistic nature of neural networks and their training methodologies. Attackers can manipulate model behavior without traditional code injection, instead using carefully crafted natural language inputs to achieve malicious outcomes. Risks include the following: @@ -30 +31 @@ When you give users access to the generative AI FMs in Amazon Bedrock, you shoul -The next section discusses these security considerations and generative AI functionality. + * Resource exhaustion through crafted prompts that trigger excessive token generation @@ -32 +33,148 @@ The next section discusses these security considerations and generative AI funct -## Security considerations + * Data exfiltration through prompt engineering techniques that extract training data or fine-tuning information + + * Model behavior manipulation through adversarial inputs designed to bypass safety mechanisms + + + + +### Application layer risks + +AI applications face unique challenges in validating and securing the interface between human users, AI models, and downstream systems. Traditional application security assumes deterministic behavior with predictable input-output relationships, but AI outputs require dynamic validation strategies that can assess content quality, safety, and appropriateness in real-time. Applications must handle scenarios where models generate syntactically valid but semantically problematic outputs. Examples of such outputs include hallucinated information presented as fact, biased responses that reflect training data patterns, or outputs that inadvertently reveal system architecture details. + +The integration of AI into existing application workflows introduces risks when downstream systems consume model outputs without proper validation. This situation can potentially lead to automated execution of flawed recommendations or propagation of incorrect information through business processes. Additionally, conversational AI applications maintain complex session state across multiple interactions, creating opportunities for session manipulation, context poisoning, and unauthorized access to conversation history containing sensitive information. + +A systems-thinking approach reveals deeper interdependencies where AI application risks cascade across system boundaries. Model outputs influence not just immediate application behavior but also training data for future models, decision-making processes, and user trust relationships. Security failures at the application layer can create feedback loops where compromised outputs become trusted inputs, gradually degrading system integrity over time. + +The temporal nature of AI interactions means that security decisions must account for both immediate threats and long-term systemic impacts. These impacts include how model behaviors evolve through user interactions and how application-level vulnerabilities might be exploited across multiple sessions or user contexts, such as: + + * Unvalidated model outputs being passed to downstream systems + + * Context injection where malicious content in Retrieval Augmented Generation (RAG) sources influences model behavior + + * Session hijacking in conversational AI applications with inadequate state management + + * Missing rate limiting enabling resource exhaustion and denial of service attacks + + * Inadequate authentication and authorization for model access endpoints + + * Insecure storage of conversation history and user interaction data + + * Cascading failures when AI-generated content triggers errors in downstream business logic + + * Model output caching creating stale or contextually inappropriate responses + + * Feedback loop contamination where AI outputs become training data without validation + + * Compound security issues where multiple minor issues combine to create potential security issues + + + + +### Data governance risks + +AI systems process and generate data in ways that challenge traditional data classification and protection mechanisms. Models can inadvertently memorize and reproduce sensitive information from training data, while their outputs may contain synthetic but realistic personal information. Risks include the following: + + * Sensitive data leakage through model memorization and regurgitation from custom foundation models + + * Compliance violations when personal data is processed without proper controls such as overly permissive agents + + * Data poisoning in fine-tuning scenarios where malicious training data affects model behavior + + * Cross-tenant data exposure in multi-tenant AI applications + + + + +## Multi-account architecture for AI workloads + +Organizations implementing AI at scale should adopt a multi-account strategy that provides clear separation of concerns, enhanced security boundaries, and simplified governance across different AI lifecycle phases. As shown in the following diagram, this architectural approach isolates inference workloads from training activities while maintaining centralized security oversight and cross-account collaboration capabilities: + + * **AI development account** – Sandbox for experimentation and prototyping with non-sensitive data + + * **AI inference account** – Production environment for AI model consumption and application hosting + + * **AI training account** – Secured environment for handling sensitive training data and production model development + + + + + + +### AI development account + +The development account provides a sandbox environment for AI experimentation, prototyping, and initial model development using non-sensitive data. This account enables data scientists and developers to explore AI capabilities, test new approaches, and develop proof-of-concept solutions without access to production or sensitive training datasets. + +Deploy Amazon Macie [automated data discovery](https://docs.aws.amazon.com/macie/latest/user/discovery-asdd.html) to help security and data science teams identify and classify data in development environments. Configure Macie to scan Amazon Simple Storage Service (Amazon S3) buckets regularly and alert when sensitive data appears in the development account. This approach enables teams to remediate data classification issues before they reach production. + +Structure this account with permissive development policies that encourage experimentation while maintaining clear boundaries that prevent access to sensitive data or production systems. Implement cost controls and resource limits to manage experimental workloads and use [AWS Budgets](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html) to monitor spending on development activities. + +Deploy [Amazon SageMaker Studio](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated.html) for collaborative development environments, with shared notebooks and experiment tracking capabilities. Configure automated cleanup policies that remove unused resources and temporary datasets, maintaining a clean development environment while controlling costs. + +### AI inference account + +The inference accounts serve as production environments for AI model consumption and application hosting. Organizations typically deploy multiple inference accounts to maintain workload isolation, for example, separate accounts for different business units, applications, or security boundaries. Each inference account contains Amazon Bedrock endpoints, agent orchestration services, and user-facing applications that consume foundation models or custom models deployed from the training account. Security controls in these accounts focus on runtime protection, user access management, and real-time monitoring of AI interactions. + +Configure each inference account with restrictive IAM policies that prevent model training activities, while enabling comprehensive inference capabilities. Implement [Amazon Cognito](https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html) or [AWS IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html) for user authentication, and with fine-grained permissions that control access to specific models. Deploy [Amazon Bedrock Guardrails](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html) and [AWS WAF](https://docs.aws.amazon.com/waf/latest/developerguide/waf-chapter.html) to filter inputs and outputs, ensuring that AI interactions meet organizational security standards. + +Establish cross-account trust relationships that allow inference accounts to access approved model artifacts from the training account through secure, audited mechanisms. Use [AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html) endpoints to maintain private connectivity to AI services while implementing comprehensive logging through [AWS CloudTrail](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-user-guide.html) and [Amazon CloudWatch](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/WhatIsCloudWatch.html) to monitor all inference activities. + +Use Amazon GuardDuty [Malware Protection for S3](https://docs.aws.amazon.com/guardduty/latest/ug/gdu-malware-protection-s3.html) to scan untrusted files that users submit for processing, such as document uploads, images, or data files that AI workloads analyze. This protection is particularly important for applications that process user-submitted content like mortgage documents, resumes, or customer support attachments. + +### AI training account + +The training account serves as a highly secured staging environment specifically designed for handling sensitive training data and production model development. This account implements the strictest security controls because of the potential presence of personally identifiable information (PII), proprietary datasets, and other sensitive information used in model training processes. Models developed in the development account are promoted to the training account for production-grade training with real datasets before deployment to inference accounts. + +Establish secure model promotion workflows that move models from development through training to inference environments with appropriate security validations at each stage. Implement automated security scanning of model artifacts and comprehensive approval processes before any model deployment to production inference systems. + +Implement enhanced data protection measures including mandatory encryption at rest and in transit. Use AWS Key Management Service (AWS KMS) [customer managed keys](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html) that provide granular access control over sensitive training datasets. Deploy Amazon Macie with continuous monitoring to identify and classify sensitive data, to help make sure that all training materials are properly protected and access is appropriately restricted. If possible, redact sensitive data before using it for training to minimize exposure risk. + +Configure Amazon SageMaker with private VPC deployments that eliminate internet access for training jobs, using VPC endpoints for necessary AWS service communication. Implement strict IAM policies that limit access to authorized personnel only, with multi-factor authentication requirements and session-based access controls for all training activities. + +Establish secure data ingestion pipelines that validate and sanitize incoming training data while maintaining comprehensive audit trails of all data access and processing activities. Use Amazon S3 with [Object Lock](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html) and versioning to help ensure training data integrity and provide immutable audit records of all training dataset modifications. + +Implement temporary elevated access management for access to training data when feasible, granting time-limited permissions that automatically expire after use. Log all user activity through CloudTrail and configure CloudWatch alarms to detect anomalous access patterns to sensitive training datasets. + +### Cross-account security and governance + +Implement centralized security monitoring through [AWS Security Hub](https://docs.aws.amazon.com/securityhub/latest/userguide/what-is-securityhub-v2.html) and Amazon GuardDuty deployed across all three account types, with findings aggregated in a dedicated security account. Use [AWS Config](https://docs.aws.amazon.com/config/latest/developerguide/WhatIsConfig.html) to enforce consistent security baselines while allowing account-specific security enhancements, particularly for the training account's heightened security requirements. + +Configure cross-account logging aggregation that forwards all AI-related logs to a centralized log archive account, with enhanced retention and protection for training account logs due to their potential sensitivity. Use [Amazon EventBridge rules](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-rules.html) to orchestrate security responses across all accounts while maintaining appropriate isolation between environments. + +## Defense in depth + +As shown in the following diagram, a defense-in-depth strategy implements security controls at different layers within each account to protect AI workloads. This section details security controls in the Application, Data, and Network layers. + + + +### Application security layer + +Deploy [AWS WAF](https://docs.aws.amazon.com/waf/latest/developerguide/waf-chapter.html) as the first line of defense against malicious requests targeting your AI applications. Configure rate limiting to prevent resource exhaustion attacks and implement AWS Managed Rules for the [Core rule set](https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-baseline.html#aws-managed-rule-groups-baseline-crs) and [Known bad inputs](https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-baseline.html#aws-managed-rule-groups-baseline-known-bad-inputs) managed rule groups. Create custom AWS WAF rules to detect common prompt injection patterns such as instruction override attempts, delimiter manipulation, and context escape sequences. For applications handling critical business functions or experiencing high request volumes, enhance this protection with [AWS Shield Advanced](https://docs.aws.amazon.com/waf/latest/developerguide/ddos-advanced-summary.html) to guard against DDoS attacks. + +Implement comprehensive input validation through [Amazon API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html) request validators. Configure validators to enforce JSON schema requirements and establish appropriate character limits for prompts and metadata fields. This validation prevents malformed requests from reaching your AI models and helps mitigate prompt injection attacks. + +Strengthen authentication and authorization by deploying [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) authorizers that validate user context and session state. Alternatively, implement [Amazon Verified Permissions](https://docs.aws.amazon.com/verifiedpermissions/latest/userguide/what-is-avp.html) for policy-based authorization that evaluates fine-grained permissions dynamically based on user attributes, resource context, and request parameters before model invocation. This approach enables centralized policy management and consistent authorization decisions across your AI applications. + +Configure response transformation to strip sensitive metadata from model outputs, helping to ensure that internal system information never reaches end users. This approach includes removing debug information, internal identifiers, and system prompts that could reveal application architecture or security controls. + +Monitor the effectiveness of these controls through CloudWatch [custom metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/publishingMetrics.html) that track prompt characteristics, response times, and error rates. Create [CloudWatch alarms](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html) to identify anomalous patterns that potentially indicate attacks or system degradation, enabling rapid response to emerging threats. + +### Data security + +Deploy Amazon Macie [automated data discovery](https://docs.aws.amazon.com/macie/latest/user/discovery-asdd.html) to identify and classify sensitive data in your AI inference workloads. Configure Macie to scan Amazon S3 buckets that contain the following: + + * User prompts and conversation logs + + * Model responses and generated content + + * RAG knowledge base documents + + * Agent memory and session data + + * Application configuration and prompt templates + + + + +Enhance detection capabilities with custom data identifiers that recognize your organization's specific sensitive data patterns. Review Macie findings regularly and establish automated remediation workflows using EventBridge to alert security teams when sensitive data appears in unexpected locations. + +Implement encryption using AWS KMS with customer managed keys for all inference-related data at rest. Organize your encryption strategy by using separate keys for the following: @@ -34 +182 @@ The next section discusses these security considerations and generative AI funct -Generative AI workloads face unique risks, including prompt injection attacks during model inference. Threat actors could craft malicious queries that force continuous output, leading to excessive resource consumption, or craft prompts that result in inappropriate model responses. Additionally, end users might inadvertently misuse these systems by inputting sensitive information in prompts. Amazon Bedrock offers robust security controls for data protection, access control, network security, logging and monitoring and input/output validation that can help mitigate these risks. These are discussed in the following sections. For more information about the risks associated with generative AI workloads, see [OWASP Top 10 for Large Language Model Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/) on the Open Worldwide Application Security Project (OWASP) website and [MITRE ATLAS](https://atlas.mitre.org/matrices/ATLAS) on the MITRE website. + * Conversation history and session data @@ -36 +184 @@ Generative AI workloads face unique risks, including prompt injection attacks du -## Remediations + * RAG knowledge base documents @@ -38 +186 @@ Generative AI workloads face unique risks, including prompt injection attacks du -### Identity and access management