AWS documentdb documentation change
Summary
Added documentation for resuming change streams using postBatchResumeToken, including examples and performance considerations
Security assessment
The change introduces a new method for resuming change streams but does not address security vulnerabilities. It focuses on performance optimization by avoiding oplog scanning.
Diff
diff --git a/documentdb/latest/developerguide/change_streams.md b/documentdb/latest/developerguide/change_streams.md index 41632e804..1399a9ccd 100644 --- a/documentdb/latest/developerguide/change_streams.md +++ b/documentdb/latest/developerguide/change_streams.md @@ -5 +5 @@ -Supported operationsBillingLimitationsEnabling change streamsUsing change streams with PythonFull document lookupResuming a change streamResuming with startAtOperationTimeTransactionsModifying log retention durationChange streams on secondary instances +Supported operationsBillingLimitationsEnabling change streamsUsing change streams with PythonFull document lookupResuming a change streamResuming with startAtOperationTimeResuming with postBatchResumeTokenTransactionsModifying log retention durationChange streams on secondary instances @@ -39,0 +40,2 @@ Applications can use change streams to subscribe to data changes on individual c + * Resuming a change stream with postBatchResumeToken + @@ -450,0 +453,46 @@ The ability to use `startAtOperationTime` is available in Amazon DocumentDB 4.0+ +## Resuming a change stream with `postBatchResumeToken` + +Amazon DocumentDB change stream now returns an additional field called `postBatchResumeToken`. This field is returned from the `$changestream` command and `getMore` command. + +Example of the `$changestream` command in Python: + + + db.command({"aggregate": "sales", "pipeline": [{ "$changeStream": {}}], "cursor": {"batchSize": 1} + +Expected output: + + + cursor" : { + "firstBatch" : [ ], + "postBatchResumeToken" : {"_data" : "0167c8cbe60000000004"}, + "id" : NumberLong("9660788144470"), + "ns" : "test.sales" + } + +Example of the `getMore` command in Python: + + + db.command({"getMore": NumberLong(<cursor id>), "collection": "sales", "batchSize": 1 }) + +Expected output + + + cursor" : { + "nextBatch" : [ ], + "postBatchResumeToken" : {"_data" : "0167c8cbe60000000004"}, + "id" : NumberLong("9660788144470"), + "ns" : "test.sales" + } + +The `postBatchResumeToken` field can be used to open new change stream cursors in the `resumeAfter` field, similar to how the resume token is used. + +Open a stream starting after the selected `postBatchResumeToken`: + + + post_batch_resume_token = output['cursor']['postBatchResumeToken'] + stream = db.watch(full_document='updateLookup', resume_after=post_batch_resume_token) + +Unlike a regular resume token that always corresponds to an operations log (oplog) entry that reflects an actual event, `postBatchResumeToken` corresponds to an oplog entry the change stream has scanned up to on the server, which is not necessarily a matching change. + +Attempting to resume with an old regular resume token will force the database to scan all the oplog entries between the specified time stamp and the current time. This may generate a lot of queries internally with each sub-query scanning for a small period of time. This will cause a spike in CPU usage and degrade the database performance. Resuming with the last `postBatchResumeToken` skips the scanning of unmatched oplog entries. +