AWS amazondynamodb documentation change
Summary
Expanded documentation about global table write modes (no primary/single primary/mixed primary) with detailed use cases, diagrams, and guidance for MREC/MRSC tables. Added explanations of idempotency requirements, conflict avoidance strategies, and disaster recovery considerations.
Security assessment
Changes focus on architectural patterns for data consistency and conflict resolution rather than addressing security vulnerabilities. While idempotency and conflict avoidance improve data integrity, there's no evidence of addressing specific security vulnerabilities or introducing security features.
Diff
diff --git a/amazondynamodb/latest/developerguide/bp-global-table-design.prescriptive-guidance.writemodes.md b/amazondynamodb/latest/developerguide/bp-global-table-design.prescriptive-guidance.writemodes.md index 556bc6c5d..e33dac9d5 100644 --- a//amazondynamodb/latest/developerguide/bp-global-table-design.prescriptive-guidance.writemodes.md +++ b//amazondynamodb/latest/developerguide/bp-global-table-design.prescriptive-guidance.writemodes.md @@ -9 +9 @@ No primarySingle primaryMixed primary -Global tables are always active-active at the table level. However, you might want to treat them as active-passive by controlling how you route write requests. For example, you might decide to route write requests to a single Region to avoid potential write conflicts. +Global tables are always active-active at the table level. However, especially for MREC tables, you might want to treat them as active-passive by controlling how you route write requests. For example, you might decide to route write requests to a single Region to avoid potential write conflicts that can happen with MREC tables. @@ -11 +11 @@ Global tables are always active-active at the table level. However, you might wa -There are three main categorizations of managed write patterns: +There are three main managed write patterns, as explained in the next three sections. You should consider which write pattern fits your use case. This choice affects how you route requests, evacuate a Region, and handle disaster recovery. The guidance in later sections depends on your application’s write mode. @@ -13 +13 @@ There are three main categorizations of managed write patterns: - * Write to any Region mode (no primary) +## Write to any Region mode (no primary) @@ -15 +15 @@ There are three main categorizations of managed write patterns: - * Write to one Region mode (single primary) +The _write to any Region_ mode, illustrated in the following diagram, is fully active-active and doesn’t impose restrictions on where a write may occur. Any Region may accept a write at any time. This is the simplest mode, but it can only be used with some types of applications. This mode is suitable for all MRSC tables. It’s also suitable for MREC tables when all writers are idempotent, and therefore safely repeatable so that concurrent or repeated write operations across Regions are not in conflict. For example, when a user updates their contact data. This mode also works well for a special case of being idempotent, an append-only dataset where all writes are unique inserts under a deterministic primary key. Lastly, this mode is suitable for MREC where the risk of conflicting writes would be acceptable. @@ -17 +17 @@ There are three main categorizations of managed write patterns: - * Write to your Region mode (mixed primary) + @@ -18,0 +19 @@ There are three main categorizations of managed write patterns: +The _write to any Region_ mode is the most straightforward architecture to implement. Routing is easier because any Region can be the write target at any time. Failover is easier, because with MRSC tables, the items are always synchronized, and with MRSC tables, any recent writes can be replayed any number of times to any secondary Region. Where possible, you should design for this write mode. @@ -19,0 +21 @@ There are three main categorizations of managed write patterns: +For example, several video streaming services use global tables for tracking bookmarks, reviews, watch status flags, and so on. These deployments use MREC tables because they need replicas scattered around the world, with each replica providing low-latency read and write operations. These deployments can use the _write to any Region_ mode as long as they ensure that every write operation is idempotent. This will be the case if every update―for example, setting a new latest time code, assigning a new review, or setting a new watch status―assigns the user’s new state directly, and the next correct value for an item doesn’t depend on its current value. If, by chance, the user’s write requests are routed to different Regions, the last write operation will persist and the global state will settle according to the last assignment. Read operations in this mode will eventually become consistent, delayed by the latest `ReplicationLatency` value. @@ -20,0 +23 @@ There are three main categorizations of managed write patterns: +In another example, a financial services firm uses global tables as part of a system to maintain a running tally of debit card purchases for each customer, to calculate that customer’s cash-back rewards. They want to keep a `RunningBalance` item per customer. This write mode is not naturally idempotent because as transactions stream in, they modify the balance by using an `ADD` expression where the new correct value depends on the current value. By using MRSC tables they can still _write to any Region_ , because every `ADD` call always operates against the very latest value of the item. @@ -22 +25 @@ There are three main categorizations of managed write patterns: -You should consider which write pattern fits your use case. This choice affects how you route requests, evacuate a Region, and handle disaster recovery. Overall best practices can differ depending on your application’s write mode. +A third example involves a company that provides online ad placement services. This company decided that a low risk of data loss would be acceptable to achieve the design simplifications of the _write to any Region_ mode. When they serve ads, they have just a few milliseconds to retrieve enough metadata to determine which ad to show, and then to record the ad impression so they don’t repeat the same ad soon. They use global tables to get both low-latency read operations for end users across the world and low-latency write operations. They record all ad impressions for a user within a single item, which is represented as a growing list. They use one item instead of appending to an item collection, so they can remove older ad impressions as part of each write operation without paying for a delete operation. This write operation is not idempotent; if the same end user sees ads served out of multiple Regions at approximately the same time, there’s a chance that one write operation for an ad impression could overwrite another. The risk is that a user might see an ad repeated once in a while. They decided that this is acceptable. @@ -24 +27 @@ You should consider which write pattern fits your use case. This choice affects -## Write to any Region mode (no primary) +## Write to one Region (single primary) @@ -26 +29 @@ You should consider which write pattern fits your use case. This choice affects -The _write to any Region_ mode is fully active-active and doesn’t impose restrictions on where a write may occur. Any Region may accept a write at any time. This is the simplest mode. It can be used only with some type of applications. It’s suitable when all writers are idempotent, and therefore safely repeatable so that concurrent or repeated write operations across Regions are not in conflict. For example, when a user updates their contact data. This mode also works well for a special case of being idempotent, an append-only dataset where all writes are unique inserts under a deterministic primary key. Lastly, this mode is suitable where the risk of conflicting writes would be acceptable. +The _write to one Region_ mode, illustrated in the following diagram, is active-passive and routes all table writes to a single active region. Note that DynamoDB doesn’t have a notion of a single active region; the application routing outside DynamoDB manages this. The _write to one Region_ mode works well for MREC tables that need to avoid write conflicts by ensuring that write operations flow only to one Region at a time. This write mode helps when you want to use conditional expressions and can't use MRSC for some reason, or when you need to perform transactions. These expressions aren’t possible unless you know that you’re acting against the latest data, so they require sending all write requests to a single Region that has the latest data. @@ -28 +31 @@ The _write to any Region_ mode is fully active-active and doesn’t impose restr - +When you use an MRSC table, you might choose to generally write to one Region for convenience. For example, this can help minimize your infrastructure build-out beyond DynamoDB. The write mode would still be write to any Region because with MRSC you could safely write to any Region at any time without concern of conflict resolution that would cause MREC tables to choose to _write to one Region_. @@ -30 +33 @@ The _write to any Region_ mode is fully active-active and doesn’t impose restr -The _write to any Region_ mode is the most straightforward architecture to implement. Routing is easier because any Region can be the write target at any time. Failover is easier, because any recent writes can be replayed any number of times to any secondary Region. Where possible, you should design for this write mode. +Eventually consistent reads can go to any replica Regions to achieve lower latencies. Strongly consistent reads must go to the single primary Region. @@ -32 +35 @@ The _write to any Region_ mode is the most straightforward architecture to imple -For example, video streaming services often use global tables for tracking bookmarks, reviews, watch status flags, and so on. These deployments can use the _write to any Region_ mode as long as they ensure that every write is idempotent and the next correct value for an item doesn’t depend on its current value. This will be the case for user updates which assign the user’s new state directly, such as setting a new latest time code, assigning a new review, or setting a new watch status. If the user’s write requests are routed to different Regions, the last write operation will persist and the global state will settle according to the last assignment. Read operations in this mode will eventually become consistent, after being delayed by the latest `ReplicationLatency` value. + @@ -34 +37 @@ For example, video streaming services often use global tables for tracking bookm -In another example, a financial services firm uses global tables as part of a system to maintain a running tally of debit card purchases for each customer, to calculate that customer’s cash-back rewards. New transactions stream in from around the world and go to multiple Regions. For their current design that doesn't take advantage of global tables, they use a single Running Balance item per customer. Customer actions update the balance with an ADD expression, which is not idempotent because the new correct value depends on the current value. This means the balance got out of sync if there were two write operations to the same balance at around the same time in different Regions. +It’s sometimes necessary to change the active Region in response to a Regional failure. Some users change the currently active Region on a regular schedule, such as implementing a follow-the-sun deployment. This places the active Region near the geography that has the most activity (usually where it’s daytime, thus the name), which results in the lowest latency read and write operations. It also has the side benefit of calling the Region-changing code daily and making sure that it’s well tested before any disaster recovery. @@ -36 +39 @@ In another example, a financial services firm uses global tables as part of a sy -This same firm could achieve a _write to any Region_ mode through a careful redesign with DynamoDB’s global tables. The new design could follow an “event streaming” model - essentially a ledger with an append-only workflow. Each customer action appends a new item to the item collection maintained for that customer. The item collection is the set of items that share a primary key, having different sort keys. Each write action that appends the customer action is an idempotent insert, using the customer ID as the partition key and transaction ID as the sort key. This design makes the calculation of the balance more involved, because it requires a `Query` to pull the items followed by some client-side math. But the advantage is that it makes all writes idempotent, which provides significant routing and failover simplifications. For more information see [Request routing with DynamoDB global tables](./bp-global-table-design.prescriptive-guidance.request-routing.html). +The passive Region(s) may keep a downscaled set of infrastructure surrounding DynamoDB that gets built up only if it becomes the active Region. This guide doesn’t cover pilot light and warm standby designs. For a more information, see [ Disaster Recovery (DR) Architecture on AWS, Part III: Pilot Light and Warm Standby](https://aws.amazon.com/blogs/architecture/disaster-recovery-dr-architecture-on-aws-part-iii-pilot-light-and-warm-standby/). @@ -38 +41 @@ This same firm could achieve a _write to any Region_ mode through a careful rede -For a third example, let’s say there’s a customer doing online ad placement. They’ve decided a low risk of data loss would be acceptable to achieve the design simplifications of the _write to any Region_ mode. When they serve ads, they have just a few milliseconds to retrieve enough metadata to determine what ad to show, and then to record the ad impression so the same ad won't repeated to that user. With global tables they can get both low-latency reads for end users across the world and low-latency writes. They can record all ad impressions for a user within a single item, and represent that as a growing List. They can use one item instead of appending to an item collection, because this way they can remove older ad impressions as part of each write without paying for a delete. This write operation is NOT idempotent, so if the same end user sees ads served out of multiple Regions at approximately the same time there’s a chance one ad impression write could overwrite the other. For online ad placement, the risk that a user might occasionally see a repeated ad is worth having this simpler and more efficient design. +Using the _write to one Region_ mode works well when you use global tables for low-latency globally distributed read operations. An example is a large social media company that needs to have the same reference data available in every Region around the world. They don’t update the data often, but when they do, they write to only one Region to avoid any potential write conflicts. Read operations are always allowed from any Region. @@ -40 +43 @@ For a third example, let’s say there’s a customer doing online ad placement. -## Single primary (“Write to one Region”) +As another example, consider the financial services company discussed earlier that implemented the daily cash-back calculation. They used _write to any Region_ mode to calculate the balance but _write to one Region_ mode to track payments. This work requires transactions, which aren't supported in MRSC tables, so it works better with a separate MREC table and _write to one Region_ mode. @@ -42 +45 @@ For a third example, let’s say there’s a customer doing online ad placement. -The _write to one Region_ mode is active-passive and routes all table writes to a single active region. Note that DynamoDB doesn’t have a notion of a single active region; the application routing outside DynamoDB manages this. The _write to one Region_ mode avoids write conflicts by ensuring writes only flow to one Region at a time. This write mode is helpful when you want to use conditional expressions or transactions, because they won't work unless you know you’re acting against the latest data. So using conditional expressions and transactions requires sending all write requests to the one Region with the latest data. +## Write to your Region (mixed primary) @@ -44 +47 @@ The _write to one Region_ mode is active-passive and routes all table writes to -Eventually consistent reads can go to any replica Regions to achieve lower latencies. Strongly consistent reads must go to the single primary region. +The _write to your Region_ write mode, illustrated in the following diagram, works with MREC tables. It assigns different data subsets to different home Regions and allows write operations to an item only through its home Region. This mode is active-passive but assigns the active Region based on the item. Every Region is primary for its own non-overlapping dataset, and write operations must be guarded to ensure proper locality. @@ -46 +49 @@ Eventually consistent reads can go to any replica Regions to achieve lower laten - +This mode is similar to _write to one Region_ except that it enables lower-latency write operations, because the data associated with each user can be placed in closer network proximity to that user. It also spreads the surrounding infrastructure more evenly between Regions and requires less work to build out infrastructure during a failover scenario, because all Regions have a portion of their infrastructure already active. @@ -48 +51 @@ Eventually consistent reads can go to any replica Regions to achieve lower laten -It’s sometimes necessary to change the active Region in response to a Regional failure, to help with data. [Evacuating a Region with DynamoDB global tables](./bp-global-table-design.prescriptive-guidance.evacuation.html) is one example of this use case. Some customers will change the currently active Region on a regular schedule, such as a "follow- the-sun" deployment. This places the active Region near the geography with the most activity, giving it the lowest latency reads and writes. It also has the side benefit of calling the Region- changing code path on a daily basis, making sure it’s well tested before any disaster recovery. + @@ -50 +53 @@ It’s sometimes necessary to change the active Region in response to a Regional -The passive Region(s) may keep a downscaled set of infrastructure surrounding DynamoDB that gets built up only should it become the active region. For a more in-depth discussion of pilot light and warm standby designs see [ Disaster Recovery (DR) Architecture on AWS, Part III: Pilot Light and Warm Standby](https://aws.amazon.com/blogs/architecture/disaster-recovery-dr-architecture-on-aws-part-iii-pilot-light-and-warm-standby/). +You can determine the home Region for items in several ways: @@ -52 +55 @@ The passive Region(s) may keep a downscaled set of infrastructure surrounding Dy -Using the _write to one Region_ mode works well when leveraging global tables for low-latency globally distributed reads. For example, a large social media company has millions of users and billions of posts. Each user gets assigned to a Region at time of account creation, placed geographically near to their location. Into that non-global table goes all their data. The company uses a separate global table to hold the mapping of users to their home regions, using a _write to one Region_ mode. It keeps read-only copies around the world to help locate each user’s data directly with minimum added latency. Updates are rare (only when moving a user’s home Region from one to another) and always go through one Region for writing, to avoid any chance of write conflicts. + * **Intrinsic:** Some aspect of the data, such as a special attribute or a value embedded within its partition key, makes its home Region clear. This technique is described in the blog post [Use Region pinning to set a home Region for items in an Amazon DynamoDB global table](https://aws.amazon.com/blogs/database/use-region-pinning-to-set-a-home-region-for-items-in-an-amazon-dynamodb-global-table/). @@ -54 +57 @@ Using the _write to one Region_ mode works well when leveraging global tables fo -As another example, consider a financial services customer who implemented a daily cash back calculation. They use _write to any Region_ mode for calculating the balance but use _write to one Region_ mode for tracking the actual cash back payments. If they want to reward a customer 1 penny for every $10 spent a day, they will need to `Query` for all transactions from the previous day, calculate the total spent, write the cash back decision to a new table, delete the queried set of items to mark them as consumed, and replace them with a singular item storing any remainder amount that should go into the next day’s calculations. This work requires transactions, and so will work better with the _write to one Region_ mode. An application may mix write modes, even on the same table, as long as the workloads have no chance of overlapping. + * **Negotiated:** The home Region of each dataset is negotiated in some external manner, such as with a separate global service that maintains assignments. The assignment may have a finite duration after which it’s subject to renegotiation. @@ -56 +59 @@ As another example, consider a financial services customer who implemented a dai -## Mixed primary (“Write to your Region”) + * **Table-oriented:** Instead of creating a single replicating global table, you create the same number of global tables as replicating Regions. Each table’s name indicates its home Region. In standard operations, all data is written to the home Region while other Regions keep a read-only copy. During a failover, another Region temporarily adopts write duties for that table. @@ -58 +60,0 @@ As another example, consider a financial services customer who implemented a dai -The _write to your Region_ mode assigns different data subsets to different Regions and allows write operations only to items through its home region. this mode is active-passive but assigns the active Region based on the item. Every Region is primary for its own non-overlapping data set, and writes must be guarded to ensure proper locality. @@ -60 +61,0 @@ The _write to your Region_ mode assigns different data subsets to different Regi -This mode is similar to _write to one Region_ except that it enables lower latency writes, because the data associated with each end user can be placed in closer network proximity to that user. It also spreads the surrounding infrastructure more evenly between Regions and requires less work to build out infrastructure during a failover scenario, because all regions will have a portion of their infrastructure already active. @@ -62 +62,0 @@ This mode is similar to _write to one Region_ except that it enables lower laten - @@ -64 +64 @@ This mode is similar to _write to one Region_ except that it enables lower laten -Determining the home region for items can be done in a variety of ways: +For example, imagine that you’re working for a gaming company. You need low-latency read and write operations for all gamers around the world. You assign each gamer to the Region that’s closest to them. That Region takes all their read and write operations, ensuring strong read-after-write consistency. However, when a gamer travels or if their home Region suffers an outage, a complete copy of their data is available in alternative Regions, and the gamer can be assigned to a different home Region. @@ -66 +66 @@ Determining the home region for items can be done in a variety of ways: - * **Intrinsic:** Some aspect of the data makes clear to what Region it’s homed, like its partition key. For example, a customer and all data about that customer would be marked within the customer data as homed to a certain region. This technique is described in [Use Region pinning to set a home Region for items in an Amazon DynamoDB global table](https://aws.amazon.com/blogs/database/use-region-pinning-to-set-a-home-region-for-items-in-an-amazon-dynamodb-global-table/) +As another example, imagine that you’re working at a video conferencing company. Each conference call’s metadata is assigned to a particular Region. Callers can use the Region that’s closest to them for lowest latency. If there’s a Region outage, using global tables allows quick recovery because the system can move the processing of the call to a different Region where a replicated copy of the data already exists. @@ -68 +68 @@ Determining the home region for items can be done in a variety of ways: - * **Negotiated:** The home region of each data set is negotiated in some external manner, such as with a separate global service that maintains assignments. The assignment may have a finite duration after which it’s subject to renegotiation. +###### To summarize @@ -70 +70 @@ Determining the home region for items can be done in a variety of ways: - * **Table-oriented:** Instead of a single replicating global table, have as many global tables as there are replicating Regions. Each table’s name indicates its home Region. In standard operations, all data is written to the home Region while other Regions keep a read-only copy. During a failover, another Region will temporarily adopt write duties for that table. + * Write to any Region mode is suitable for MRSC tables and idempotent calls to MREC tables. @@ -71,0 +72 @@ Determining the home region for items can be done in a variety of ways: + * Write to one Region mode is suitable for non-idempotent calls to MREC tables. @@ -72,0 +74 @@ Determining the home region for items can be done in a variety of ways: + * Write to your Region mode is suitable for non-idempotent calls to MREC tables, where it's important to have clients write to a Region that’s close to them. @@ -75 +76,0 @@ Determining the home region for items can be done in a variety of ways: -For example, imagine you’re working for a gaming company. You need low latency reads and writes for all gamers around the world. You can home each gamer to the Region closest them. That region takes all their reads and writes, ensuring there’s always strong read-after-write consistency. However, if that gamer travels or their home Region suffers an outage, a complete copy of their data will be available in alternative Regions. So the gamer can be assigned to different home Region as is useful. @@ -77 +77,0 @@ For example, imagine you’re working for a gaming company. You need low latency -As another example, imagine you’re working at a video conferencing company. Each conference call's metadata gets assigned to a particular Region. Callers can use the Region that's closest to them for lowest latency. If there’s a Region outage, using global tables allows quick recovery because the system can move the processing of the call to a different Region where there’s already a replicated copy of the data. @@ -85 +85 @@ To use the Amazon Web Services Documentation, Javascript must be enabled. Please -Global table design +Using global tables @@ -87 +87 @@ Global table design -Request routing +Routing strategies in DynamoDB