AWS wellarchitected: Rewrote ADVSUS05-BP01 from data redundancy to privacy workload efficiency
Summary
The best practice page was completely rewritten: the old guidance on identifying and removing redundant ad data (S3 lifecycle rules, Storage Lens, AWS Config, ECR lifecycle policies) was replaced with sustainability guidance for privacy-enhanced data collaboration (Clean Rooms), covering time-shiftable batch scheduling, columnar formats, Graviton, serverless matching, auto scaling, and Regional aggregation. Title, risk level, related services, and related best practices sections were updated accordingly.
Security assessment
The change is a sustainability/carbon-efficiency rewrite of a Well-Architected best practice; it mentions privacy-preserving collaboration (AWS Clean Rooms, suppression, aggregation) but contains no vulnerability fix, credential/encryption/authz change, or incident reference. The only security-adjacent content is general privacy framing, which is not new security feature documentation.
Evidence
+# ADVSUS05-BP01 Optimize privacy workload processing patterns and resource allocation for sustainability
Diff
diff --git a/wellarchitected/latest/video-streaming-advertising-lens/advsus05-bp01.md b/wellarchitected/latest/video-streaming-advertising-lens/advsus05-bp01.md index 75f6a28..b8ce28a 100644 --- a//wellarchitected/latest/video-streaming-advertising-lens/advsus05-bp01.md +++ b//wellarchitected/latest/video-streaming-advertising-lens/advsus05-bp01.md @@ -7 +7 @@ -Implementation guidance +Implementation guidanceResources @@ -9 +9 @@ Implementation guidance -# ADVSUS05-BP01 Identify and remove redundant data across storage +# ADVSUS05-BP01 Optimize privacy workload processing patterns and resource allocation for sustainability @@ -11 +11,36 @@ Implementation guidance -Participants in the real-time advertising supply chain can accrue large volumes of data. Consider how you use data and data preservation as outlined in ADPERF04-BP01. Don't keep data that has no purpose, can easily be recreated, and expedite the removal of low value or short-lived data. Remove unwanted advertisement video, images, files, and any other associated data that is no longer needed. +Privacy-enhanced collaboration introduces sustainability considerations for advertising workloads that combine first-party and third-party customer data. The processing patterns and resource allocation you choose for data matching and cleansing influence the energy and carbon footprint of the workload. + +**Desired outcome:** + + * Privacy-preserving data collaboration workloads run on right-sized, energy-efficient compute that scales with actual demand. + + * Compute-intensive matching and cleansing operations are scheduled and located to minimize energy use and data movement. + + * Data is stored and exchanged in compact, columnar formats that reduce processing and transfer overhead. + + + + +**Common anti-patterns:** + + * Running privacy computations continuously on persistent, over-provisioned instances regardless of collaboration demand. + + * Processing raw, uncompressed, row-oriented data and moving full datasets across Regions before any aggregation. + + * Defaulting to general-purpose x86 instances without evaluating energy-efficient processor options for matching workloads. + + + + +**Benefits of establishing this best practice:** + + * Lower energy consumption and carbon footprint for privacy-enhanced collaboration workloads. + + * Reduced cost through right-sized, demand-driven compute and lower data transfer volumes. + + * Improved processing efficiency from compact data formats and purpose-fit compute. + + + + +**Level of risk exposed if this best practice is not established:** Medium @@ -15 +50,55 @@ Participants in the real-time advertising supply chain can accrue large volumes - * Optimize data handling needs based on workload requirements, and verify that it reflects the nature of the business and short-lived advertising content (delete or archive based on data class). +With privacy-enhanced collaboration, advertisers and publishers derive shared insight from combined first-party and third-party customer data without exposing individual records. In video-streaming advertising, this collaboration underpins audience overlap analysis, suppression, and campaign measurement, and its role is growing as third-party identifiers decline and more targeting relies on matched first-party signals. + +The work is a chain of operations: + + * Matching resolves and conglomerates the records that describe the same person or household across parties + + * Cleansing normalizes, deduplicates, and discards invalid records + + * Aggregation organizes detail into non-identifying groups + + + + +Each stage is data-intensive and bursty, driven by campaign planning and reporting cycles rather than a steady stream of viewer requests. Infrastructure provisioned for constant demand spends most of its life idle, generating carbon emissions with no delivered output. + +This workload's results are consumed on analysis and reporting timescales, not in the request path of ad delivery. A planner sizing a campaign or an analyst studying audience overlap tolerates minutes or hours of latency, a different constraint from the milliseconds an ad-serving decision allows. That latency tolerance is the main sustainability opportunity, because deferrable work can be shifted in time and consolidated instead of held ready on standby. Grid carbon intensity varies by hour and by location, so the same computation finished during a cleaner window or in a lower-carbon Region has a smaller estimated footprint for identical output. Treating deferrable work as latency-critical prevents time-shifting and requires continuously running, over-provisioned capacity. + +A collaboration cycle's energy use depends as much on how much data it touches and how far that data travels as on the algorithm it runs. Privacy rules heighten the trade-off by constraining where records may reside and how they must be transformed. This can force less efficient data movement or replication. Instrument the resource-utilization proxies you can see, such as compute time, bytes transferred, and storage footprint. Base decisions on those proxies, treating a reduction in them as a reduction in estimated energy cost. + +### Implementation steps + + 1. **Schedule for lower carbon intensity:** Run intensive privacy computations as scheduled, time-shiftable batch jobs rather than continuously, and prefer lower-carbon AWS Regions. To target genuinely lower-carbon periods, drive scheduling from a carbon-intensity data source (a third-party grid carbon-intensity API or published Region-level carbon data), since AWS doesn't expose a built-in real-time grid carbon-intensity signal. + + 2. **Use batch processing:** Process data cleansing and matching operations as batch jobs to avoid persistent resource consumption. + + 3. **Adopt efficient data formats:** Implement efficient data compression and use compact columnar formats such as Parquet for processing and exchange. + + 4. **Use energy-efficient compute:** Run matching and cleansing workloads on AWS Graviton processors to lower energy use per operation. + + 5. **Prefer serverless for matching:** Use serverless architectures for matching operations where possible to reduce idle capacity. + + 6. **Scale to actual demand:** Implement auto scaling driven by actual collaboration workload patterns so capacity tracks real usage. + + 7. **Aggregate data regionally first:** Configure Regional data aggregation before central processing to reduce cross-Region data transfer needs. + + + + +## Resources + +**Related best practices:** + + * [ADVSUS05-BP02 Optimize fraud detection systems for resource efficiency](./advsus05-bp02.html) + + * [ADVSUS05-BP03 Optimize content moderation systems for sustainable operation](./advsus05-bp03.html) + + + + +**Related documents:** + + * [Hardware and services](https://docs.aws.amazon.com/wellarchitected/latest/sustainability-pillar/hardware-and-services.html) + + * [AWS Clean Rooms](https://docs.aws.amazon.com/clean-rooms/latest/userguide/what-is.html) + @@ -17 +105,0 @@ Participants in the real-time advertising supply chain can accrue large volumes - * Consider if duplicate ad files or versions of ad files are be being saved that can be easily recreated. @@ -19 +106,0 @@ Participants in the real-time advertising supply chain can accrue large volumes - * Use [Amazon S3 storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) rules to automate the expiration (and deletion) of draft ad content versions. For content that should be preserved for historical purposes, use Amazon S3 storage lifecycles to transition content to another storage class, such as Amazon Glacier. @@ -21 +108 @@ Participants in the real-time advertising supply chain can accrue large volumes - * [Amazon S3 Storage Lens](https://docs.aws.amazon.com/AmazonS3/latest/userguide/storage-lens-optimize-storage.html) can identify incomplete multipart uploads, buckets that have numerous noncurrent versions, and if lifecycle rules are not present. Storage Lens can also provide activity metrics to identify ad objects or even prefixes that are infrequently used. +**Related services:** @@ -23 +110 @@ Participants in the real-time advertising supply chain can accrue large volumes - * [AWS Config](https://aws.amazon.com/config/) can also identify if you have unused resources, such as [EBS volumes](https://aws.amazon.com/ebs/). + * [AWS Lambda](https://aws.amazon.com/lambda/) @@ -25 +112 @@ Participants in the real-time advertising supply chain can accrue large volumes - * Use [Amazon ECR lifecycle policies](https://docs.aws.amazon.com/AmazonECR/latest/userguide/LifecyclePolicies.html) to expire old images used for real-time bidding containers. + * [AWS Graviton](https://aws.amazon.com/ec2/graviton/) @@ -27 +114 @@ Participants in the real-time advertising supply chain can accrue large volumes - * Evaluate how users are using data to eliminate use cases, dimensions, and queries that no longer provide value. + * [AWS Auto Scaling](https://aws.amazon.com/autoscaling/) @@ -38 +125 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Data management +Trust, safety, and privacy workload efficiency @@ -40 +127 @@ Data management -Hardware and services +ADVSUS05-BP02 Optimize fraud detection systems for resource efficiency