AWS solutions documentation change
Summary
Removed detailed sections on backfill tuning, replay tuning, cutover/rollback, snapshot creation, metadata migration, backfill operations, and traffic replayer usage. Updated CLI commands and flags, simplified workflow names, and restructured content to focus on core migration workflow commands.
Security assessment
The changes primarily involve removing detailed technical sections and updating command syntax. No security vulnerabilities, weaknesses, or incidents are mentioned. The removed sections included security-related configurations (like auth header handling), but their removal doesn't indicate a security fix - it's documentation simplification.
Diff
diff --git a/solutions/latest/migration-assistant-for-amazon-opensearch-service/use-the-solution.md b/solutions/latest/migration-assistant-for-amazon-opensearch-service/use-the-solution.md index 991c62a1d..d0bc280e0 100644 --- a//solutions/latest/migration-assistant-for-amazon-opensearch-service/use-the-solution.md +++ b//solutions/latest/migration-assistant-for-amazon-opensearch-service/use-the-solution.md @@ -7 +7 @@ -Getting started with the Workflow CLICore commandsApproval gatesStatus symbolsMigration scenariosBackfill tuningReplay tuningCutover and rollbackAssessment and breaking changesCreating a snapshotMigrating metadataRunning backfillUsing Traffic ReplayerWhat is not migrated automatically +Getting started with the Workflow CLICore commandsApproval gatesStatus symbolsMigration scenariosAssessment and breaking changes @@ -197,2 +197,2 @@ After the pilot succeeds, widen the configuration to the full index set and subm - workflow output - workflow output --follow + workflow log all + workflow log all -f @@ -226,3 +226,3 @@ Command | Why you use it -`workflow output` | Shows logs across workflow pods -`workflow output --follow` | Streams logs live -`workflow approve <PATTERN>` | Approves pending gates that match exact names or globs +`workflow log all` | Shows logs across workflow pods +`workflow log all -f` | Streams logs live +`workflow approve step <PATTERN>` | Approves pending step gates that match exact names or globs @@ -243 +243 @@ Flag | Purpose -`--workflow-name <name>` | Name of the workflow to replace if one already exists (default: `migration`) +`--workflow-name <name>` | Name of the workflow to replace if one already exists (default: `migration-workflow`) @@ -252 +252 @@ Flag | Purpose -`--workflow-name <name>` | Show status for a specific workflow (default: `migration`) +`--workflow-name <name>` | Show status for a specific workflow (default: `migration-workflow`) @@ -259 +259 @@ Flag | Purpose -**workflow output** +**workflow log** @@ -266,2 +266,3 @@ Flag | Purpose -`--workflow-name <name>` | Filter logs to a specific workflow (default: `migration`) -`-l` / `--selector <labels>` | Label selector to filter pods (e.g. `source=a,target=b`) +`--workflow-name <name>` | Filter logs to a specific workflow (default: `migration-workflow`) +`--source` / `--target` / `--snapshot` / `--task` / `--from-snapshot-migration <value>` | Filter pods by the corresponding migration label (available on `workflow log filter`) +`--label <key=value>` | Additional raw label selector to filter pods, repeatable (e.g. `--label source=a --label target=b`); available on `workflow log filter` @@ -277 +278 @@ Flag | Purpose -`--workflow-name <name>` | Workflow to manage (default: `migration`) +`--workflow-name <name>` | Workflow to manage (default: `migration-workflow`) @@ -298,2 +299,2 @@ Flag | Purpose -`<TASK_NAMES>` (one or more) | Exact names or glob patterns of pending approval gates to approve (e.g. `*.evaluateMetadata`) -`--workflow-name <name>` | Workflow containing the gates (default: `migration`) +`<NAMES>` (one or more, supplied to a subcommand such as `step`, `change`, or `retry`) | Exact names or glob patterns of pending approval gates to approve (e.g. `workflow approve step 'evaluatemetadata.*'`) +`--workflow-name <name>` | Workflow containing the gates (default: `migration-workflow`) @@ -334 +335 @@ Not every migration step should run without human review. Approval gates let the - workflow approve <STEP_NAME> + workflow approve step <STEP_NAME> @@ -371,78 +371,0 @@ The most comprehensive approach. Capture begins first so no writes are lost, the -## Backfill tuning - -Useful Reindex-from-Snapshot settings include: - - * `podReplicas` — number of RFS pods running in parallel (one shard per pod). - - * `maxConnections` — bulk-indexer concurrency to the target. - - * `documentsPerBulkRequest` — bulk batch size. - - * `maxShardSizeBytes` — maximum supported shard size (default 80 GiB). Larger shards must be reduced before backfill (force-merge or split). - - * `initialLeaseDuration` — ISO-8601 duration each worker holds a shard lease before re-acquisition (default `PT10M`). - - * `allowedDocExceptionTypes` — list of exception class names from the target’s response that should be counted as success rather than retried. - - * `allowLooseVersionMatching` — bypass the strict source/target version compatibility check. - - - - -Because RFS reads from snapshot storage in Amazon S3, increasing worker count does not add live read load to the source cluster. It mainly changes how quickly the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection is driven. - -## Replay tuning - -Useful Traffic Replayer settings include: - - * `podReplicas` — number of replayer pods. - - * `speedupFactor` — default `1.1`. `2.0` means twice the original traffic timeline. - - * `removeAuthHeader` — strips the captured `Authorization` header before replaying. Useful when the captured traffic carries credentials that would not be valid against the target. - - * `authHeaderOverride` — replaces the captured `Authorization` header with a static value. - - * `dependsOnSnapshotMigrations` — ensures replay only starts after backfill completes. - - * `nonRetryableDocExceptionTypes` — list of exception class names that should be counted as failures but not retried. - - - - -###### Warning - -Setting both `replayerConfig.removeAuthHeader: true` and an `authConfig` block on the same target is rejected by the schema. Pick one — either rely on the target’s `authConfig` (the Traffic Replayer applies it for you) or strip the captured header. - -## Cutover and rollback - -Switching traffic to the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection is the cutover step. By this point, capture has already protected writes during backfill, replay has caught the target up, and validation is complete. - -Before you switch: - - * replay has reached the live edge, - - * the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection is healthy, - - * representative application queries work on the target, - - * the application team is ready to move traffic, and - - * the rollback path is still available. - - - - -The exact cutover mechanism depends on your environment, but the principle is always the same: - - 1. Stop pointing clients at the capture proxy. - - 2. Point clients directly at the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection. - - 3. Watch the target closely during the first production traffic window. - - - - -In practice, that usually means updating a DNS record, a load balancer backend, an application connection string, or a service-discovery entry. Keep the source cluster available during a rollback window (typically 24–72 hours) before decommissioning. After the rollback window has passed, see [Uninstall the solution](./uninstall-the-solution.html) to remove the Migration Assistant infrastructure. - @@ -478,597 +400,0 @@ For complex migrations involving multiple transformations, perform a trial migra -## Creating a snapshot - -Once you have your change data capture solution in place or have disabled indexing to your source cluster, create a snapshot. The snapshot captures all the metadata and documents to be migrated to the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection. - -### Create a snapshot - -Run the following command from the Migration Console pod to initiate snapshot creation: - - - console snapshot create - -Migration Assistant automatically generates a snapshot name and configures the necessary Amazon S3 repository. Alternatively, you can use an existing snapshot — see the `snapshot` configuration in your workflow configuration for details. - -### Check snapshot status - -To check the snapshot creation status: - - - console snapshot status - -To retrieve detailed information about the snapshot: - - - console snapshot status --deep-check - -Wait for snapshot creation to complete before proceeding to the metadata migration phase. A completed snapshot returns output similar to: - - - SUCCESS - Snapshot is SUCCESS. - Percent completed: 100.00% - Data GiB done: 29.211/29.211 - Total shards: 40 - Successful shards: 40 - Failed shards: 0 - Start time: 2024-07-22 18:21:42 - Duration: 0h 13m 4s - Anticipated duration remaining: 0h 0m 0s - Throughput: 38.13 MiB/sec - -### Managing slow snapshot speeds - -Depending on the data size in the source cluster and the bandwidth allocated for snapshots, the process can take some time. Use the `--max-snapshot-rate-mb-per-node` option to adjust the maximum rate at which the source cluster’s nodes create the snapshot. Increasing the snapshot rate consumes more source node resources, which may affect the cluster’s ability to handle normal traffic. - -## Migrating metadata - -Metadata migration involves creating a snapshot of your source cluster and then migrating the metadata (index settings, mappings, templates, and aliases) to the Amazon OpenSearch Service domain or Amazon OpenSearch Serverless collection. - -The metadata migration tool gathers information from the source cluster through a snapshot or HTTP requests. These snapshots are fully compatible with the backfill process for Reindex-from-Snapshot (RFS). - -### Evaluating metadata - -Before migrating, use the evaluate command to preview what will be migrated: - - - console metadata evaluate - -This scans the source cluster, applies filtering and modifications, and produces a list of items that will be migrated. Items not listed will not be migrated. Use this as a safety check before modifying the target. - -Example output: - - - Starting Metadata Evaluation - Clusters: - Source: - Remote Cluster: OpenSearch 1.3.16 - - Target: - Remote Cluster: OpenSearch 2.14.0 - - Migration Candidates: - Index Templates: - simple_index_template - - Component Templates: