AWS documentdb: Document expanded events and $changeStreamSplitLargeEvent for change streams
Summary
Adds documentation for DDL event notifications (createCollection, createIndexes) via the showExpandedEvents option and for the new $changeStreamSplitLargeEvent aggregation stage that splits change events larger than 16 MB into fragments, including resume-token behavior and usage constraints.
Security assessment
The change documents new change-stream capabilities (expanded DDL events and event fragmentation) and a 16 MB event-size workaround; it is feature/behavior documentation with no mention of vulnerabilities, credentials, encryption, or access control.
Evidence
+ * Aggregation stages: `$match`, `$project`, `$redact`, `$addFields`, `$replaceRoot`, and `$changeStreamSplitLargeEvent`.
Diff
diff --git a/documentdb/latest/devguide/change_streams.md b/documentdb/latest/devguide/change_streams.md index dfae31c38..a1e23a9f0 100644 --- a//documentdb/latest/devguide/change_streams.md +++ b//documentdb/latest/devguide/change_streams.md @@ -7 +7 @@ -Supported operationsBillingLimitationsEnabling change streamsUsing change streams with PythonFull document lookupResuming a change streamResuming with startAtOperationTimeResuming with postBatchResumeTokenTransactionsModifying log retention durationChange streams on secondary instances +Supported operationsExpanded eventsBillingLimitationsEnabling change streamsUsing change streams with PythonFull document lookupUsing $changeStreamSplitLargeEvent to split large events into fragmentsResuming a change streamResuming with startAtOperationTimeResuming with postBatchResumeTokenTransactionsModifying log retention durationChange streams on secondary instances @@ -27,0 +28,2 @@ Applications can use change streams to subscribe to data changes on individual c + * Expanded events + @@ -37,0 +40,2 @@ Applications can use change streams to subscribe to data changes on individual c + * Using $changeStreamSplitLargeEvent to split large events into fragments + @@ -61 +65 @@ Amazon DocumentDB supports the following operations for change streams: - * Aggregation stages: `$match`, `$project`, `$redact`, and `$addFields`and `$replaceRoot`. + * Aggregation stages: `$match`, `$project`, `$redact`, `$addFields`, `$replaceRoot`, and `$changeStreamSplitLargeEvent`. @@ -66,0 +71,18 @@ Amazon DocumentDB supports the following operations for change streams: + * DDL event notification for `createCollection` using the `showExpandedEvents` parameter (applicable to Amazon DocumentDB 8.0 starting with engine version 8.0.2). + + * DDL event notification for `createIndexes` using the `showExpandedEvents` parameter (applicable to Amazon DocumentDB 8.0 starting with engine version 8.0.2). + + + + +## Expanded events + +###### Note + +Expanded events are available in Amazon DocumentDB 8.0 starting with engine version 8.0.2. + +Change streams support data definition language (DDL) event notifications, `createCollection` and `createIndexes`. To include expanded events, open a change stream cursor with the `showExpandedEvents` option. + +For example: + +mongosh Method @@ -69,0 +92,14 @@ Amazon DocumentDB supports the following operations for change streams: + let cur = db.names.watch( [ ], { + showExpandedEvents: true + } ) + cur.next() + +Aggregation Stage + + + + let cur = db.names.aggregate( [ + { $changeStream: { showExpandedEvents: true } } + ] ) + cur.next() + @@ -90 +126 @@ Change streams have the following limitations in Amazon DocumentDB: - * If the total size of a change streams event (including the change data and full document, if requested) is greater than `16 MB`, the client will experience a read failure on the change streams. + * If the total size of a change streams event (including the change data and full document, if requested) is greater than `16 MB`, the client will experience a read failure on the change stream. To avoid this, use the `$changeStreamSplitLargeEvent` stage to split the event into fragments. @@ -316,0 +353,14 @@ The output of the stream object will look something like this: +## Using $changeStreamSplitLargeEvent to split large events into fragments + +###### Note + +The $changeStreamSplitLargeEvent stage is available in Amazon DocumentDB 8.0 starting with engine version 8.0.2. + +When you use the `$changeStreamSplitLargeEvent` stage, change events larger than 16 MB are split into fragments that are returned sequentially through the change stream cursor. Events smaller than 16 MB are returned unsplit. The stage splits an event along its top-level field boundaries; it does not split within a single field or recurse into subdocuments. Because of this, if any single top-level field is itself larger than 16 MB, the stage can't produce a valid fragment for it and returns an error instead. + +As a best practice, we recommend avoiding the use of `$changeStreamSplitLargeEvent` when possible by reducing the size of the event. + +The `$changeStreamSplitLargeEvent` stage can only be used in a `$changeStream` pipeline, and it must be used only once as the last stage. + +Each fragment has its own `resumeToken` and its own `postBatchResumeToken`. Resuming from a non-final fragment token (using either a regular resume token or a postBatchResumeToken) delivers the subsequent fragment. In the case where the token corresponds to the final fragment of an event, the change stream cursor returns the subsequent event. +