Login/Register
Replication Slots error while replicating data from RDS Postgres(read replica) to Redshift using AWS DMS
Asked Answered
amazon-web-servicesamazon-redshiftamazon-rdsaws-dms
L

3

2

I'm trying to replicate data from a read replica of RDS Postgres to Amazon redshift using Amazon's Data Migration Service. I was successfully able to perform migration tasks on the same source and destination, but I get the following error on my CloudWatch while doing a replication: Can you help me understand why am I getting this error?

2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:24 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-199231648;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:24 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-148900000;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:24 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-146798752;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-144697504;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-94374048;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-44042400;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-41941152;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-39839904;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=6289248;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:45:25 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:45:25 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=226494304;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:25 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=224393056;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #0 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #1 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #2 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #3 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #4 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #5 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Subtask #6 ended (replicationtask_util.c:925)
2017-07-09T14:45:29 [TASK_MANAGER ]I: Task management thread terminated (replicationtask.c:2679)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' running CDC only in resume mode (replicationtask.c:805)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Creating threads for all components (replicationtask.c:1279)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for QAYMAYIYZXMVMSBSGQRBYUUXQI is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for 4UC3KDSEXMGBDPTTFU2DYBTMNE is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: CDC Control buffer stream size for transaction_sorter is set to 5MB (subtask.c:772)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Threads for all components were created (replicationtask.c:1425)
2017-07-09T14:48:15 [TASK_MANAGER ]I: Task initialization completed successfully (replicationtask.c:2265)
2017-07-09T14:48:15 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1751124128;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:15 [TARGET_APPLY ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Restore bulk state. Last bulk last record id - '0', last applied record id - '0', target confirmed record id - '0' (endpointshell.c:890)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Target endpoint does not support direct insert in batch apply mode (bulk_apply.c:468)
2017-07-09T14:48:16 [TARGET_APPLY ]I: Working in bulk apply mode (endpointshell.c:897)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1749022880;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TASK_MANAGER ]I: All stream components were initialized (replicationtask.c:2110)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Initial position is set to 'now' (postgres_endpoint_util.c:349)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #1 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' does NOT exist. (postgres_endpoint_capture.c:283)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Initial positioning requested is 'now' (postgres_endpoint_capture.c:285)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1695901856;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Failed to execute statement: 'SELECT xlog_position FROM pg_create_logical_replication_slot('6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05', 'test_decoding')' [122502] ODBC general error. (ar_odbc_stmt.c:2184)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #2 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 55000 NativeError: 1 Message: ERROR: logical decoding requires wal_level >= logical;, Error while executing the query [122502] ODBC general error. (ar_odbc_stmt.c:2190)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #3 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Creating replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' failed. [122502] ODBC general error. (postgres_endpoint_capture.c:306)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Error executing source loop [122502] ODBC general error. (streamcomponent.c:1473)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TASK_MANAGER ]E: Stream component failed at subtask 0, component st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI [122502] ODBC general error. (subtask.c:1380)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #4 (replicationtask_util.c:862)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]E: Stream component 'st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI' terminated [122502] ODBC general error. (subtask.c:1545)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #5 (replicationtask_util.c:862)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Starting subtask #6 (replicationtask_util.c:862)
2017-07-09T14:48:16 [TASK_MANAGER ]E: Task error notification received from subtask 0, thread 0 [122502] ODBC general error. (replicationtask.c:1938)
2017-07-09T14:48:16 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' encountered a recoverable error (repository.c:4107)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1465911456;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1415579808;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1413478560;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1411377312;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1361053856;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1310722208;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:16 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1308620960;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1306519712;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:16 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:17 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1260390560;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:17 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1042286752;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:48:17 [TARGET_LOAD ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:48:17 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-1040185504;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #0 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #1 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #2 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #3 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #4 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #5 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Subtask #6 ended (replicationtask_util.c:925)
2017-07-09T14:48:20 [TASK_MANAGER ]I: Task management thread terminated (replicationtask.c:2679)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Task '6UJ63JGAHT7XDURNKAJPQATIXU' running CDC only in resume mode (replicationtask.c:805)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Creating threads for all components (replicationtask.c:1279)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for QAYMAYIYZXMVMSBSGQRBYUUXQI is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for 4UC3KDSEXMGBDPTTFU2DYBTMNE is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: CDC Control buffer stream size for transaction_sorter is set to 5MB (subtask.c:772)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Threads for all components were created (replicationtask.c:1425)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Task initialization completed successfully (replicationtask.c:2265)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-727713952;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Going to start Redshift File Transfer client (redshift_imp.c:925)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Restore bulk state. Last bulk last record id - '0', last applied record id - '0', target confirmed record id - '0' (endpointshell.c:890)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Target endpoint does not support direct insert in batch apply mode (bulk_apply.c:468)
2017-07-09T14:53:45 [TARGET_APPLY ]I: Working in bulk apply mode (endpointshell.c:897)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-725612704;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [TASK_MANAGER ]I: All stream components were initialized (replicationtask.c:2110)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Initial position is set to 'now' (postgres_endpoint_util.c:349)
2017-07-09T14:53:45 [TASK_MANAGER ]I: Starting subtask #1 (replicationtask_util.c:862)
2017-07-09T14:53:45 [SOURCE_UNLOAD ]I: Going to connect to ODBC connection string: Driver={PostgreSQL Unicode(x64)};Server=postgres-replica.1mginfra.com;Port=-672491680;Database=my_sample_db;UseDeclareFetch=1;Fetch=10000;Uid=la_la_username;Pwd=****; (postgres_endpoint_util.c:258)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' does NOT exist. (postgres_endpoint_capture.c:283)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]I: Initial positioning requested is 'now' (postgres_endpoint_capture.c:285)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: Failed to execute statement: 'SELECT xlog_position FROM pg_create_logical_replication_slot('6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05', 'test_decoding')' [122502] ODBC general error. (ar_odbc_stmt.c:2184)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: RetCode: SQL_ERROR SqlState: 55000 NativeError: 1 Message: ERROR: logical decoding requires wal_level >= logical;, Error while executing the query [122502] ODBC general error. (ar_odbc_stmt.c:2190)
2017-07-09T14:53:45 [SOURCE_CAPTURE ]E: Creating replication slot '6uj63jgaht7xdurn_00016400_ca44edb7_78eb_46e7_a992_7ffa63eb3b05' failed. [122502] ODBC general error. (postgres_endpoint_capture.c:306)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #2 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #3 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #4 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Going to connect to server blahblah.redshift.amazonaws.com database test_dms (redshift_imp.c:736)
2017-07-09T14:53:46 [SOURCE_CAPTURE ]E: Error executing source loop [122502] ODBC general error. (streamcomponent.c:1473)
2017-07-09T14:53:46 [TASK_MANAGER ]E: Stream component failed at subtask 0, component st_0_QAYMAYIYZXMVMSBSGQRBYUUXQI [122502] ODBC general error. (subtask.c:1380)
2017-07-09T14:53:46 [TASK_MANAGER ]I: Starting subtask #5 (replicationtask_util.c:862)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Connected to server blahblah.redshift.amazonaws.com database test_dms successfully. (redshift_imp.c:755)
2017-07-09T14:53:46 [TARGET_LOAD ]I: Creating Redshift File Transfer client (redshift_imp.c:867)
Lanilaniard answered 9/7, 2017 at 16:0 Comment(1)
I got the same problem. I assume it may be related to the fact it needs to create objects there and the error is misleading – Navigator
V
2

In order to update the wal_level and get rid of the error you will need to make a change to your RDS configuration. The easiest way to update it is through the console (web site)

  1. Once logged in and on the RDS service page, click on the "Parameter groups"
  2. If you haven't changed anything, you're probably still using the default group. In my case I was using default.aurora-postgresql12. Default groups don't allow changes so you will need to create a new parameter group
  3. Give the group a name, description and copy the settings from you default group
  4. Edit the group and filter on "rds.logical_replication"
  5. Modify that item and set its value to 1 (from 0) and then save your changes
  6. Next update your cluster to use your new parameter group. This will ask you when you want to apply the changes. Applying them now "should" have no impact on the cluster, but you can always wait to apply them. Regardless when you apply the changes you will need to reboot the reader/writer instances (this took me about 60 seconds).
  7. After the reboot, try to restart the DMS task and it should work.
Vide answered 1/12, 2021 at 14:33 Comment(1)
that worked! I want to specify that was necessary to manually reboot the RDS instance from AWS Console. Just select the RDS instance from console -> Actions -> Reboot. After that I restarted the migration task and I could see the status to replication ongoing – Semanteme
N
0

AWS DMS on read replica on PG is not supported . In order to enable CDC you need to trigger wal creation and this done by doing a least one backup , but backup is not supported on Read replica so it is a kind of dead lock.

I've open a case on that in AWS

Navigator answered 25/9, 2017 at 16:53 Comment(0)
H
0

Also it seems replication is supported on 9.5.4 and above. https://forums.aws.amazon.com/message.jspa?messageID=744635

Hydrophone answered 25/4, 2018 at 3:54 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.