AWS solutions documentation change
Summary
Updated backfill procedures to distinguish between workflow-managed and legacy RFS deployments. Added workflow commands (workflow configure/edit/submit/manage) for document backfill configuration, scaling, and monitoring. Replaced console commands with workflow equivalents and added NextGen references.
Security assessment
The changes are procedural updates for migration operations with no security vulnerabilities addressed. Modifications focus on workflow integration and command syntax without security implications or new security features.
Diff
diff --git a/solutions/latest/migration-assistant-for-amazon-opensearch-service/running-backfill.md b/solutions/latest/migration-assistant-for-amazon-opensearch-service/running-backfill.md index fd2aa381f..cd3d73f6b 100644 --- a//solutions/latest/migration-assistant-for-amazon-opensearch-service/running-backfill.md +++ b//solutions/latest/migration-assistant-for-amazon-opensearch-service/running-backfill.md @@ -11 +11 @@ Starting the backfillScaling the backfillMonitoring the backfillAmazon CloudWatc -After metadata has been migrated, use Reindex-from-Snapshot (RFS) to backfill documents from the source cluster snapshot into the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection. +After metadata has been migrated, use Reindex-from-Snapshot (RFS) to backfill documents from the source cluster snapshot into the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless NextGen collection. @@ -15 +15 @@ After metadata has been migrated, use Reindex-from-Snapshot (RFS) to backfill do -Start the migration from RFS: +Configure RFS under `snapshotMigrationConfigs[].perSnapshotConfig.<snapshot>[].documentBackfillConfig` and submit the workflow: @@ -18 +18,3 @@ Start the migration from RFS: - console backfill start + workflow configure edit + workflow submit + workflow manage @@ -20 +22,3 @@ Start the migration from RFS: -The status reports `Running` even if all shards have been migrated. See Stopping the backfill for how to finalize. +The workflow creates a `SnapshotMigration` resource for each configured snapshot migration. If `documentBackfillConfig` is present, the workflow starts RFS after snapshot and metadata dependencies are ready. If `documentBackfillConfig` is omitted, that migration pass skips document backfill. + +The legacy `console backfill start` command starts a directly configured RFS deployment outside the workflow. @@ -24 +28,2 @@ The status reports `Running` even if all shards have been migrated. See Stopping -To speed up the migration, increase the number of RFS worker pods. Each worker processes shards in parallel from the snapshot in Amazon S3, so scaling up does not add live read load to the source cluster — it only increases the indexing rate on the target. +To speed up workflow-managed backfill, increase `documentBackfillConfig.podReplicas` and resubmit the workflow. Each worker processes shards in parallel from the snapshot in Amazon S3, so scaling up does not add live read load to the source cluster — it only increases the indexing rate on the target. + @@ -25,0 +31,2 @@ To speed up the migration, increase the number of RFS worker pods. Each worker p + documentBackfillConfig: + podReplicas: 5 @@ -27 +33,0 @@ To speed up the migration, increase the number of RFS worker pods. Each worker p - console backfill scale <NUM_WORKERS> @@ -29 +35,4 @@ To speed up the migration, increase the number of RFS worker pods. Each worker p -For example, to scale to 5 workers: + workflow configure edit + workflow submit + +For a legacy directly configured RFS deployment, use: @@ -34 +43 @@ For example, to scale to 5 workers: -It may take a few minutes for additional workers to come online. Scale up gradually while monitoring the health metrics of the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection to avoid oversaturating it. Amazon OpenSearch Service provides [cluster health and performance metrics](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/monitoring.html) that you can use for this monitoring. +It may take a few minutes for additional workers to come online. Scale up gradually while monitoring the health metrics of the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless NextGen collection to avoid oversaturating it. Amazon OpenSearch Service provides [cluster health and performance metrics](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/monitoring.html) that you can use for this monitoring. @@ -38 +47 @@ It may take a few minutes for additional workers to come online. Scale up gradua -Check the status of the backfill: +Check workflow-managed backfill status: @@ -41 +50,2 @@ Check the status of the backfill: - console backfill status + workflow status --resource-view + workflow log resource snapshotmigration.<NAME> -- --tail=100 @@ -43 +53 @@ Check the status of the backfill: -For detailed shard-level progress: +Use `workflow manage` for the interactive view and approval gates. Use `workflow log resource --list` to find the exact `snapshotmigration.<NAME>` resource name. @@ -44,0 +55 @@ For detailed shard-level progress: +For a legacy directly configured RFS deployment, use: @@ -45,0 +57,2 @@ For detailed shard-level progress: + + console backfill status @@ -78 +91,3 @@ Find the dashboards in the Amazon CloudWatch console in the AWS Region where you -To pause a migration without losing progress: +The workflow path does not expose a separate `workflow backfill pause` command. To reduce or pause worker activity for a workflow-managed backfill, lower `documentBackfillConfig.podReplicas` and resubmit the workflow after considering whether the current run is actively writing to the target. + +For a legacy directly configured RFS deployment, use: @@ -83 +98 @@ To pause a migration without losing progress: -This stops all existing workers while leaving the backfill operation in a resumable state. To restart: +This stops all existing legacy workers while leaving the backfill operation in a resumable state. To restart: @@ -94 +109,3 @@ This stops all existing workers while leaving the backfill operation in a resuma -Completing the backfill requires manually stopping the migration. Stopping shuts down all workers and cleans up all coordination metadata. After status checks report that data has been completely migrated, stop the migration: +Workflow-managed backfill does not require a manual `console backfill stop`. The workflow’s RFS completion monitor marks the `SnapshotMigration` resource complete after the eligible shard work is finished, then the workflow proceeds to the next gate or dependent phase. + +For a legacy directly configured RFS deployment, stop the migration only after status checks report that data has been completely migrated: @@ -104 +121 @@ Example output: - Backfill working state archived to: /shared-logs-output/migration-console-default/backfill_working_state/working_state_backup_20241115174822.json + Backfill working state archived to: ./backfill_working_state/working_state_backup_20241115174822.json @@ -108 +125 @@ Example output: -You cannot restart a stopped migration. If you may need to resume later, use `console backfill pause` instead. +You cannot restart a stopped legacy RFS migration. If you may need to resume later, use `console backfill pause` instead. @@ -112 +129 @@ You cannot restart a stopped migration. If you may need to resume later, use `co -After the backfill completes and workers have stopped, verify the contents of your Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection: +After the backfill completes and workers have stopped, verify the contents of your Amazon OpenSearch Service domain or Amazon OpenSearch Serverless NextGen collection: