AWS dms documentation change

Service: dms · 2025-05-13 · Documentation low

File: dms/latest/userguide/CHAP_Troubleshooting.md

Summary

Added troubleshooting section for UTF8 encoding issues between Oracle and PostgreSQL, recommending ReplaceChars settings

Security assessment

The change resolves character encoding compatibility issues during data migration. While encoding errors can impact data integrity, there is no indication this relates to security vulnerabilities or introduces security-specific features. The ReplaceChars setting is a data transformation tool, not a security control.

Diff

diff --git a/dms/latest/userguide/CHAP_Troubleshooting.md b/dms/latest/userguide/CHAP_Troubleshooting.md
index 93586cfd7..62df443e8 100644
--- a//dms/latest/userguide/CHAP_Troubleshooting.md
+++ b//dms/latest/userguide/CHAP_Troubleshooting.md
@@ -280,0 +281,2 @@ Following, you can learn about troubleshooting issues specific to using AWS DMS
+  * Failed to get LOB data
+
@@ -484,0 +487,30 @@ The command displays the AWS DMS Oracle Read Performance Utility options.
+### Failed to get LOB data
+
+LOB (Large Object) lookup failures in AWS DMS occur under specific circumstances during data migration processes. During the full load phase, AWS DMS employs the lookup method for LOB data migration when the task is configured for FULL LOB mode. Notably, during the CDC (Change Data Capture) phase, AWS DMS consistently uses the Lookup method regardless of LOB settings.
+
+AWS DMS first replicates rows without the LOB column, retrieves LOB data using a **SELECT** command, and executes an **UPDATE** command to replicate the LOB field on the target. This sequential INSERT and UPDATE operation characterizes the LOOKUP behavior. The LOB lookup during the CDC phase is not universally applicable across all database engines and depending on data size the tasks may replicate inline rows along with column data.
+
+LOB Lookup process failure is a common issue that can occur during migration displaying the "Failed to get lob data, going to set to null" error message. During this failure the partial table data on the target, particularly the LOB columns appear as NULL values. Various factors can trigger these failures:
+
+  * Source row deletion occurring before DMS completes the Lookup process
+
+  * Intermittent connectivity issues disrupting lookup threads
+
+  * DMS lookup queries entering a waiting state due to source table locking scenarios
+
+
+
+
+To address these LOB Lookup failures, you can do the following:
+
+  * Implement Limited LOB settings during the full load phase to eliminate lookup behavior and enhance performance.
+
+  * Reload affected tables when encountering lookup failure messages and partial data on the target.
+
+  * For issues occuring due to intermittent network or source database availability problems, restart the task to resolve all table data inconsistencies.
+
+
+
+
+These steps to handling LOB Lookup failures ensures more reliable data migration and helps maintain data integrity throughout the process.
+
@@ -769,0 +802,2 @@ Following, you can learn about troubleshooting issues specific to using AWS DMS
+  * Invalid byte sequence for encoding "UTF8"
+
@@ -872,0 +907,8 @@ Sources that support views include the following.
+### Invalid byte sequence for encoding "UTF8"
+
+Data migration from Oracle to PostgreSQL using AWS DMS presents unique challenges due to character set encoding differences between the two databases. A significant issue arises from Oracle's AL32UTF8 character set, which fully supports 4-byte characters, while PostgreSQL's UTF8 character set implementation lacks this capability. This disparity often leads to migration failures, particularly when dealing with tables or columns in the Oracle source that contain 4-byte characters.
+
+During migration attempts, you may encounter error messages in both DMS task logs and PostgreSQL target database logs indicating problems with invalid UTF8 byte sequences. A typical error message "ERROR: invalid byte sequence for encoding "UTF8": 0xed 0xb0 0x86" is displayed. To resolve this issue, AWS DMS provides a solution through the "`ReplaceChars`" settings. It automatically sibtitutes or eliminates invalid characters during the migration process. This approach effectively prevents encoding-related errors without requiring modifications to the source data.
+
+For more information, see _Character set validation and replacement_ bullet point in [Character substitution task settings](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.CharacterSubstitution.html) topic.
+