划线反馈

OBServer 日志归档进度不推进的原因及解决方法

更新时间：2025-06-23 08:26

适用版本：V4.0.x、V4.1.x、V4.2.x内容类型：Troubleshoot

问题描述

1004 租户归档情况（11 月 6 号查询，start_scn_display 还卡在 11 月 3 号）。

MySQL [oceanbase]> select USED_PIECE_ID,* from oceanbase.cdb_ob_archivelog 
where tenant_id=1004 \G
*************************** 1. row ***************************
            USED_PIECE_ID: 77
                   TENANT_ID: 1004
                     DEST_ID: 1009
                    ROUND_ID: 6
                 INCARNATION: 1
                     DEST_NO: 0
                      STATUS: DOING
                   START_SCN: 1698993729762606386
           START_SCN_DISPLAY: 2023-11-03 14:42:09.762606
              CHECKPOINT_SCN: 1698993729762606386
      CHECKPOINT_SCN_DISPLAY: 2023-11-03 14:42:09.762606
                  COMPATIBLE: 1
               BASE_PIECE_ID: 74
               USED_PIECE_ID: 77
       PIECE_SWITCH_INTERVAL: 86400000000
                   UNIT_SIZE: 1
                 COMPRESSION: none
                 INPUT_BYTES: 25357733034
         INPUT_BYTES_DISPLAY: 23.62GB
                OUTPUT_BYTES: 25357733034
        OUTPUT_BYTES_DISPLAY: 23.62GB
           COMPRESSION_RATIO: 1.00
         DELETED_INPUT_BYTES: 0
 DELETED_INPUT_BYTES_DISPLAY: 0.00MB
        DELETED_OUTPUT_BYTES: 0
DELETED_OUTPUT_BYTES_DISPLAY: 0.00MB
                     COMMENT: 
                        PATH: file:///arch/nfs/xxx/xxx/archive
1 row in set (0.03 sec)

问题版本

OceanBase 数据库 V4.x 版本。

问题原因

OBServer 节点 NFS 未挂载或者权限有问题。

解决方法

确认租户日志流归档情况。

MySQL [oceanbase]> select *, scn_to_timestamp(checkpoint_scn) from oceanbase.__all_virtual_ls_log_archive_progress where tenant_id = 1004 and piece_id = 77;

输出结果如下：

+-----------+---------+----------+----------+-------+----------------------------+----------------------------+-------------+---------------------+-------------+-------------+---------------------+--------+---------+-------------+-------------+--------------+----------------------------------+
| tenant_id | dest_id | round_id | piece_id | ls_id | gmt_create                 | gmt_modified               | incarnation | start_scn           | min_lsn     | max_lsn     | checkpoint_scn      | status | file_id | file_offset | input_bytes | output_bytes | scn_to_timestamp(checkpoint_scn) |
+-----------+---------+----------+----------+-------+----------------------------+----------------------------+-------------+---------------------+-------------+-------------+---------------------+--------+---------+-------------+-------------+--------------+----------------------------------+
|      1004 |    1009 |        6 |       77 |     1 | 2023-11-06 14:42:11.608909 | 2023-11-06 21:58:38.224490 |           1 | 1699252929762606000 | 16012563081 | 16093678274 | 1699279118131157674 | DOING  |     240 |    55638722 |    81115193 |     81115193 | 2023-11-06 21:58:38.131157       |
|      1004 |    1009 |        6 |       77 |  1001 | 2023-11-06 14:42:11.611017 | 2023-11-06 21:58:38.225544 |           1 | 1699252929762606000 |  7115535700 |  7145075019 | 1699279118082764862 | DOING  |     107 |    29539319 |    29539319 |     29539319 | 2023-11-06 21:58:38.082764       |
+-----------+---------+----------+----------+-------+----------------------------+----------------------------+-------------+---------------------+-------------+-------------+---------------------+--------+---------+-------------+-------------+--------------+----------------------------------+
2 rows in set (0.03 sec)

确认租户日志流状态。

MySQL [oceanbase]> select * from oceanbase.__all_virtual_ls_status where tenant_id=1004;

输出结果如下：

+-----------+-------+----------------------------+----------------------------+------------------+--------------------+--------+-------------+---------------+--------------+
| tenant_id | ls_id | gmt_create                 | gmt_modified               | init_member_list | b_init_member_list | status | ls_group_id | unit_group_id | primary_zone |
+-----------+-------+----------------------------+----------------------------+------------------+--------------------+--------+-------------+---------------+--------------+
|      1004 |     1 | 2023-08-25 21:05:55.646509 | 2023-08-25 21:05:55.688068 |                  |                    | NORMAL |           0 |             0 | zone3        |
|      1004 |  1001 | 2023-08-25 21:06:04.771557 | 2023-08-25 21:06:04.808467 |                  |                    | NORMAL |        1001 |          1003 | zone3        |
|      1004 |  1002 | 2023-08-25 21:06:04.812671 | 2023-08-25 21:06:05.843835 |                  |                    | NORMAL |        1002 |          1004 | zone3        |
+-----------+-------+----------------------------+----------------------------+------------------+--------------------+--------+-------------+---------------+--------------+
3 rows in set (0.02 sec)

确认日志流 leader 节点。

对比第 1 和第 2 步，可以看出 1004 租户的少了一个 1002 号日志流归档的记录，归档日志由日志流 leader 负责的。

MySQL [oceanbase]> select * from oceanbase.gv$ob_log_stat where tenant_id=1004 and ls_id=1002;

输出结果如下：

+-----------+-------+--------------+----------+----------+-------------+---------------------------------+-------------+-------------------------------------------------------------+-------------------+---------+------------+-----------+---------------------+------------+---------------------+------------+---------------------+--------------------+---------------+
| TENANT_ID | LS_ID | SVR_IP       | SVR_PORT | ROLE     | PROPOSAL_ID | CONFIG_VERSION                  | ACCESS_MODE | PAXOS_MEMBER_LIST                                           | PAXOS_REPLICA_NUM | IN_SYNC | BASE_LSN   | BEGIN_LSN | BEGIN_SCN           | END_LSN    | END_SCN             | MAX_LSN    | MAX_SCN             | ARBITRATION_MEMBER | DEGRADED_LIST |
+-----------+-------+--------------+----------+----------+-------------+---------------------------------+-------------+-------------------------------------------------------------+-------------------+---------+------------+-----------+---------------------+------------+---------------------+------------+---------------------+--------------------+---------------+
|      1004 |  1002 | 10.xxx.xx.15 |     2882 | LEADER   |          10 | {proposal_id:10, config_seq:11} | APPEND      | 10.xxx.xx.11:2882:1,10.xxx.xx.13:2882:1,10.xxx.xx.15:2882:1 |                 3 | YES     | 7918362624 |         0 | 1692968764811129415 | 8000469638 | 1699320789901413260 | 8000469638 | 1699320789901413260 |                    |               |
|      1004 |  1002 | 10.xxx.xx.11 |     2882 | FOLLOWER |          10 | {proposal_id:10, config_seq:11} | APPEND      | 10.xxx.xx.11:2882:1,10.xxx.xx.13:2882:1,10.xxx.xx.15:2882:1 |                 3 | YES     | 7918362624 |         0 | 1692968764811129415 | 8000469638 | 1699320789901413260 | 8000469638 | 1699320789901413260 |                    |               |
|      1004 |  1002 | 10.xxx.xx.13 |     2882 | FOLLOWER |          10 | {proposal_id:10, config_seq:11} | APPEND      | 10.xxx.xx.11:2882:1,10.xxx.xx.13:2882:1,10.xxx.xx.15:2882:1 |                 3 | YES     | 7918362624 |         0 | 1692968764811129415 | 8000469638 | 1699320789901413260 | 8000469638 | 1699320789901413260 |                    |               |
+-----------+-------+--------------+----------+----------+-------------+---------------------------------+-------------+-------------------------------------------------------------+-------------------+---------+------------+-----------+---------------------+------------+---------------------+------------+---------------------+--------------------+---------------+
3 rows in set (0.05 sec

确认日志。

在 10.xxx.xx.15 节点最新的 observer.log 日志里可以看到目录权限有问题，进一步 df -h 命令确认发现没有挂载的 NFS（网络文件系统）。

挂载 NFS（网络文件系统）后，1004 租户的日志归档进度恢复正常。

日志归档延迟或日志归档慢的原因和解决方法

OBServer 日志归档一直处于 BEGINING 状态的原因及解决方法

OceanBase 数据库