--- title: "OMS 数据迁移或数据同步链路遇到报错 [errcode=-4016] request start lsn from all server fail 的原因和解决方法-OceanBase数据库使用指南" description: "了解OceanBase数据库在实际应用中关于OMS 数据迁移或数据同步链路遇到报错 [errcode=-4016] request start lsn from all server fail 的原因和解决方法相关的常见问题和使用技巧，帮助您快速解决OMS 数据迁移或数据同步链路遇到报错 [errcode=-4016] request start lsn from all server fail 的原因和解决方法的难题。" --- 切换语言 - 中文站 - 简体中文 - International - English - 日本站 - 日本語 划线反馈 # OMS 数据迁移或数据同步链路遇到报错 [errcode=-4016] request start lsn from all server fail 的原因和解决方法 更新时间：2026-06-01 09:21 适用版本： V4.2.x、V4.3.x、V4.4.x 内容类型：Troubleshoot ## 问题现象 新建的一条 OceanBase 数据库 Oracle 租户 -> OceanBase 数据库 Oracle 租户的增量同步链路启动后长期卡在【**预检查**】这一步，【**预检查**】结束后没法自动进入到【**增量同步**】这一阶段，最终该链路出现报错 `Failed to complete http request: Connect to 127.0.0.1:8088 [/127.0.0.1] failed: Connection refused (Connection refused)`。 ## 问题原因 通过分析 OMS Web 界面上对应迁移链路右上角的【**查看组件监控**】 -> 【**libobcdcd.log**】。 可以发现如下的日志报错信息： ```shell [2025-12-09 17:33:23.840350] EDIAG [TLOG.FETCHER] get_result (ob_log_start_lsn_locator.cpp:927) [55861][][T0][YDxxxxx-0000000000300001-0-0] [lt=31][errcode=-4016] request start lsn from all server fail(tls_id={tenant_id:1004, ls_id:{id:1}}, start_tstamp_ns=1761113615570552209, svr_cnt=3, svr_list_consumed=3, result_svr_list_idx=-1, svr_list=[{svr:"xxx.xxx.xxx.122:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:xxxxx-xxxxx-xxxxx-xxxxx}, {svr:"xxx.xxx.xxx.182:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:YD9560ACB3C2F-0000000000700911-0-0}, {svr:"xxx.xxx.xxx.183:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:YD9560ACB3C2F-0000000000601222-0-0}]) BACKTRACE:0x1c04c3ac 0x15a13a08 0x15af2b58 0x15af2658 0x15af2480 0x15c09188 0xc446458 0xc33ca24 0xc33c5a8 0xc21f870 0xc21ec50 0xc21e0a8 0xc2c0a58 0xfffff68f7d38 0xfffff683f690 [2025-12-09 17:33:23.840476] EDIAG [TLOG.FETCHER] locate_lsn_ (ob_log_ls_fetch_ctx.cpp:975) [55861][][T0][YDxxxxx-0000000000300001-0-0] [lt=123][errcode=0] start lsn locate fail(start_lsn_locate_req={state:2, tls_id:{tenant_id:1004, ls_id:{id:1}}, start_tstamp_ns:1761113615570552209, svr_cnt:3, svr_list_consumed:3, result_svr_list_idx:-1, cur_max_start_lsn:{lsn:18446744073709551615}, cur_max_start_log_tstamp:-1, succ_locate_count:0, svr_list:[{svr:"xxx.xxx.xxx.122:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:xxxxx-xxxxx-xxxxx-xxxxx}, {svr:"xxx.xxx.xxx.182:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:YD9560ACB3C2F-0000000000700911-0-0}, {svr:"xxx.xxx.xxx.183:2882", rpc_executed:true, rpc_err:0, svr_err:0, ls_err:-4233, start_lsn:{lsn:18446744073709551615}, start_log_tstamp:18446744073709551615, trace_id:YD9560ACB3C2F-0000000000601222-0-0}], fetching_mode:"Integrated Fetching Mode", archive_locate_rs:{start_lsn:{lsn:18446744073709551615}, loc_err:0}}) BACKTRACE:0x1c04c3ac 0x15a13a08 0x15a131a4 0x15a12d70 0x15a12cbc 0x15a12ad8 0xc33e67c 0xc33ca34 0xc33c5a8 0xc21f870 0xc21ec50 0xc21e0a8 0xc2c0a58 0xfffff68f7d38 0xfffff683f690 ``` 其中的关键报错信息如下： ```shell [errcode=-4016] request start lsn from all server fail [errcode=0] start lsn locate fail ls_err:-4233 ``` 这边 -4016 报错的常见原因有： - 对应使用的 OBCDC 的版本和 OceanBase 数据库 的版本不兼容。 - OBCDC 想要拉取的 clog 日志不存在了。 而 -4233 报错一般代表主库的 clog 日志已经被回收了/OBCDC 启动位点对应的 clog 日志被回收了（在数据字典模式下，可能距离 OBCDC 启动位点最近的数据字典基线日志已被回收了）。 检查 OBCDC 的版本发现，这边 OBCDC 使用的版本为 libobcdc 4.2.5.6。 ```shell [2025-12-09 17:24:40.521046] INFO [TLOG] init_logger_ (ob_log_instance.cpp:465) [55739][][T0][Y0-0000000000000000-0-0] [lt=0] ====================libobcdc start==================== [2025-12-09 17:24:40.521157] INFO [TLOG] init_logger_ (ob_log_instance.cpp:466) [55739][][T0][Y0-0000000000000000-0-0] [lt=111] libobcdc 4.2.5.6 ``` 而对应 OceanBase 数据库 的版本为 V4.2.5.5 < libobcdc 4.2.5.6，因此 OBCDC 的版本和 OceanBase 数据库 的版本是兼容的。 进一步排查 libobcdc.log，发现该增量同步链路中 OBCDC 拉取的起始时间位点不对： ```shell start_tstamp_ns=1761113615570552209 ``` start_tstamp_ns=1761113615570552209 对应的可读时间点如下： ```sql MySQL [oceanbase]> select usec_to_time(1761113615570552); +--------------------------------+ | usec_to_time(1761113615570552) | +--------------------------------+ | 2025-10-22 14:13:35.570552 | +--------------------------------+ 1 row in set (0.00 sec) ``` 而迁移链路创建的时间为 2025-12-09，主库也未开启任何的日志归档，2025-10-22 的 clog 日志早就已经被回收掉了，故 OBCDC 拉取 clog 日志失败。 但这边创建该增量同步链路时，使用的起始时间位点为默认的当前的时刻，并未手工设置任何的时间位点为 `2025-10-22 14:13:35`。 利用 `1761113615570552209` 作为关键字去反向查找日志后发现，这个错误的时间位点不是用户输入的起始位点，而是从业务 OceanBase 集群中查出来的： ```sql MySQL [oceanbase]> alter system change tenant tenant_id=1004; Query OK, 0 rows affected (0.003 sec) MySQL [oceanbase]> SELECT SNAPSHOT_SCN, ORA_ROWSCN AS END_SCN, START_LSN, END_LSN FROM __all_data_dictionary_in_log WHERE SNAPSHOT_SCN <= 1765279996000000000 ORDER BY SNAPSHOT_SCN DESC LIMIT 1\G *************************** 1. row *************************** SNAPSHOT_SCN: 1761113615570552209 <<<=== here END_SCN: 1761114495061243000 START_LSN: 75394282680 END_LSN: 75423037019 1 row in set (0.004 sec) ``` 确认该问题为 OceanBase 数据库 V4.2.5 BP1 版本上存在的一个系统定时任务 的问题导致的副作用，该问题会导致 任务一直没有触发自动定时执行，进而导致系统视图 `CDB_OB_DATA_DICTIONARY_IN_LOG` 中对应该租户对应的 `SNAPSHOT_SCN` 不正确，而 OMS 的增量同步链路会使用该 `SNAPSHOT_SCN` 作为增量日志拉取的起始时间位点： **备注：OceanBase 数据库 V4.2.5 BP2 版本上已经修复了该问题**。 ```sql MySQL [oceanbase]> select * from oceanbase.__all_virtual_tenant_scheduler_job where job_name like '%dict%' and job != 0 and tenant_id=1004\G *************************** 1. row *************************** tenant_id: 1004 job_name: SCHEDULED_TRIGGER_DUMP_DATA_DICT job: 1125899906862660 gmt_create: 2025-03-24 18:37:05.316177 gmt_modified: 2025-03-28 16:38:00.103392 lowner: SYS powner: SYS cowner: SYS last_date: 2025-03-28 16:38:00.002283 this_date: NULL next_date: 2048-01-17 02:56:00.000000 <<<=== here total: 6787808 interval#: FREQ=MINUTELY; INTERVAL=120 failures: 0 flag: 0 what: DBMS_DATA_DICT.TRIGGER_DUMP() nlsenv: NULL charenv: NULL field1: NULL scheduler_flags: 0 exec_env: 2151677954,45,46,46,0,,2, job_style: REGULAR program_name: NULL job_type: PLSQL_BLOCK job_action: DBMS_DATA_DICT.TRIGGER_DUMP() number_of_argument: NULL start_date: 2025-03-24 18:38:00.000000 repeat_interval: FREQ=MINUTELY; INTERVAL=120 end_date: 4000-01-01 00:00:00.000000 job_class: DBMS_DATA_DICT_JOB_CLASS enabled: 1 auto_drop: 0 state: SCHEDULED run_count: NULL retry_count: NULL last_run_duration: NULL max_run_duration: 7200 comments: used to dump data_dictionary credential_name: NULL destination_name: NULL interval_ts: 0 user_id: -1 database_id: -1 max_failures: 0 1 row in set (0.023 sec) MySQL [oceanbase]> select scn_to_timestamp(snapshot_scn), * from CDB_OB_DATA_DICTIONARY_IN_LOG where tenant_id = 1004 order by snapshot_scn desc limit 10; +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ | scn_to_timestamp(snapshot_scn) | TENANT_ID | SNAPSHOT_SCN | REPORT_TIME | START_LSN | END_LSN | +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ | 2025-10-22 14:13:35.570552 | 1004 | 1761113615570552209 | 2025-10-22 14:28:15.054786 | 75394282680 | 75423037019 | | 2025-10-22 14:08:55.638602 | 1004 | 1761113335638602434 | 2025-10-22 14:08:55.556791 | 75365779723 | 75393023515 | | 2025-08-19 17:34:05.983417 | 1004 | 1755596045983417000 | 2025-08-19 17:36:51.016882 | 60063536305 | 60086852772 | | 2025-07-29 19:41:08.861165 | 1004 | 1753789268861165834 | 2025-07-29 19:51:32.911839 | 54229556088 | 54257230588 | | 2025-07-09 10:50:58.826104 | 1004 | 1752029458826104207 | 2025-07-09 10:58:54.235626 | 43601096169 | 43638517326 | | 2025-06-26 11:22:28.805926 | 1004 | 1750908148805926000 | 2025-06-26 11:22:29.675429 | 35517503036 | 35532176948 | | 2025-05-10 22:59:15.608098 | 1004 | 1746889155608098000 | 2025-05-10 23:04:28.601148 | 22645275050 | 22656096038 | ... ... ... +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ 10 rows in set (0.087 sec) ``` ## 问题的风险及影响 OMS 数据迁移或数据同步链路无法成功启动。 ## 适用版本 - OMS 所有版本。 - OceanBase 数据库 V4.2.5 BP1（oceanbase-4.2.5.1-101000182024120722）版本以及所有经停 V4.2.5 BP1（oceanbase-4.2.5.1-101000182024120722）升级上来的版本。 ## 解决方法 连接到业务 OceanBase 集群的系统 sys 租户执行如下命令： ```sql alter system set dump_data_dictionary_to_log_interval = '120m' tenant all; ``` 执行完成后再检查最新的 `SNAPSHOT_SCN`： ```sql MySQL [oceanbase]> select scn_to_timestamp(snapshot_scn), * from CDB_OB_DATA_DICTIONARY_IN_LOG where tenant_id = 1004 order by snapshot_scn desc limit 10; +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ | scn_to_timestamp(snapshot_scn) | TENANT_ID | SNAPSHOT_SCN | REPORT_TIME | START_LSN | END_LSN | +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ | 2025-12-10 10:52:44.473431 | 1004 | 1765335164473431000 | 2025-12-10 10:52:48.855053 | 91696362416 | 91729896624 | | 2025-10-22 14:13:35.570552 | 1004 | 1761113615570552209 | 2025-10-22 14:28:15.054786 | 75394282680 | 75423037019 | | 2025-10-22 14:08:55.638602 | 1004 | 1761113335638602434 | 2025-10-22 14:08:55.556791 | 75365779723 | 75393023515 | | 2025-08-19 17:34:05.983417 | 1004 | 1755596045983417000 | 2025-08-19 17:36:51.016882 | 60063536305 | 60086852772 | | 2025-07-29 19:41:08.861165 | 1004 | 1753789268861165834 | 2025-07-29 19:51:32.911839 | 54229556088 | 54257230588 | | 2025-07-09 10:50:58.826104 | 1004 | 1752029458826104207 | 2025-07-09 10:58:54.235626 | 43601096169 | 43638517326 | | 2025-06-26 11:22:28.805926 | 1004 | 1750908148805926000 | 2025-06-26 11:22:29.675429 | 35517503036 | 35532176948 | | 2025-05-10 22:59:15.608098 | 1004 | 1746889155608098000 | 2025-05-10 23:04:28.601148 | 22645275050 | 22656096038 | | 2025-05-05 09:52:01.892010 | 1004 | 1746409921892010000 | 2025-05-05 09:57:43.496891 | 19040754687 | 19051577964 | | 2025-05-04 10:14:30.654399 | 1004 | 1746324870654399110 | 2025-05-04 10:19:32.401826 | 18892428955 | 18903224216 | +--------------------------------+-----------+---------------------+----------------------------+-------------+-------------+ 10 rows in set (0.087 sec) ``` 输出结果中的第一条记录显示最新的 `SNAPSHOT_SCN` 结果已经符合预期了。 最后将之前卡住的迁移链路删掉后再重建下，增量同步都恢复正常了。 ## 规避方式 无。 上一篇 [Oracle 数据库离线迁移至 OceanBase 数据库 Oracle 模式的参考手册](https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000002012507) 下一篇 [OMS 切换子任务 UK&隐藏列删除效率优化](https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000005437836) ![有帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*y6ocSqN8cqsAAAAAAAAAAAAAARQnAQ)![无帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*BG9IQJyLHF8AAAAAAAAAAAAAARQnAQ)![反馈](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*eTWdQKCRKHwAAAAAAAAAAAAAARQnAQ)[AI](https://www.oceanbase.com/obi) 咨询热线