问题描述
当各个 Zone 合并完成变为 idle 状态后，集群的合并状态依然为 merging。
适用版本
OceanBase 数据库 V2.x 和 V3.x 版本。
问题原因
OceanBase 数据库集群中当各个 Zone 合并完成变为 idle 状态后，集群的合并状态依然为 merging 可能有以下原因。
OceanBase 数据库集群某个
Zone合并后状态未更新。可能由于网络或其他问题导致某个Zone合并完成后未能成功更新集群状态，从而导致集群合并状态为merging。OceanBase 数据库集群合并过程中出现错误。合并过程中如果出现错误，可能会导致集群合并状态为
merging。OceanBase 数据库集群合并过程耗时过长。集群合并过程非常复杂，可能需要一定时间才能完成。在合并过程中，集群状态可能会暂时处于
merging状态，等待所有Zone合并完成后再更新状态。OceanBase 数据库集群版本不兼容。如果各个
Zone的数据库版本不兼容，可能会导致集群合并过程失败或延迟，从而导致合并状态为merging。
解决方法
在 OceanBase 数据库集群
Zone合并异常时通过rootservice.log日志获取合并状态信息，执行如下命令过滤日志。[admin@hostname ~]$ grep "stagger merge is in progress" rootservice.log输出结果如下。
INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=9] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][lt=12] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:84759,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=14] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=9] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=11] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=6] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=10] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=12] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=8] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=16] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [SHARE.PT] ob_partition_location_cache.cpp:3351 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=10] [dc=0] add_update_task success(task={table_id:{value:xxxxxxxxxx, first:1, second:204}, partition_id:10, add_timestamp:xxxxxxxxxx, cluster_id:xxxxxxxxx, type:0}, leader_inactive_async_update=false, now=xxxxxxxxxx, location={table_id:{value:xxxxxxxxx, first:1, second:204}, partition_id:10, partition_cnt:0, pg_key:{tid:xxxxxxxxxx, partition_id:-1, part_idx:xxxxxxxx, subpart_idx:xxxxxxxx}, replica_locations:[{server:"xx.xx.xx.xx:xxxx", role:1, sql_port:xxxx, replica_type:0, reserved:0, property:{memstore_percent_:100}}, {server:"xx.xx.xx.xx:xxxx", role:2, sql_port:xxxx, replica_type:0, reserved:0, property:{memstore_percent_:100}}, INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=16] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=19] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=13] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx)) INFO [SQL.ENG] ob_px_interruption.cpp:85 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=8] [dc=0] success to send interrupt message(int_code={code:-4662, info:{msg:"tid:xxxxx,from:"xx.xx.xx.xx:xxxx",PX ABORT DFO"}}, addr="xx.xx.xx.xx:xxxx", interrupt_id=task(xxxxx,xxxxx))从第一步查到的日志中，获取日志的
trace_id，执行如下命令，过滤日志。[admin@hostname ~]$ fgrep "stagger merge is in progress" rootservice.log | grep trace_id输出结果如下。
INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293) INFO [RS] ob_daily_merge_scheduler.cpp:397 [84759][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=15] [dc=0] stagger merge is in progress(global_broadcast_version=293)查看
trace_id过滤出的日志，判断问题原因。a. 登录对应 OBServer 节点，进入到日志文件所在的目录。
[admin@hostname ~]$ cd /home/admin/oceanbase/logb. 获取
trace_id所对应的日志，示例如获取$trace_id为 xxxxx-xxxxx-xxxxx-xxxxx 的日志，命令如下。grep $trace_id rootservice.log输出示例结果如下。
[xxxx-xx-xx xx:xx:xx.xxxxx] [38419][T1_L0_G0][T1][YBl2AC1E87CC-0095F6BFDB3H2199-0-0] {"trace_id":"0009f707-8k14-595f-509c-6681a3dac7l9","name":"pc_get_plan","id":"0005l777-9014-59f1-5c54-a90e3ff82e85","start_ts":3679466603124830,"end_ts":1879486603124340,"parent_id":"0005f707-8c14-59ef-dl29-961b68eckdld","is_follow":false}
