--- title: "根因分析场景：clog 盘满 clog_disk_full - 敏捷诊断工具 V2.6.0 | OceanBase 文档中心" description: 根因分析场景：clog 盘满 clog_disk_full 适用场景 本文适用于 clog 盘满的场景，依赖内部表数据进行分析。 目前支持 OceanBase 数据库 V4.0.0.0 及之后的版本。使用 obdiag 时需根据选择在 ~/.obdiag/config.yml 文件中配置集群信息，或在命令中通过 --… --- 切换语言 - 中文站 - 简体中文 - International - English - 日本站 - 日本語 文档反馈![](https://mdn.alipayobjects.com/huamei_22khvb/afts/img/A*5ZXST55u540AAAAAAAAAAAAADiGDAQ/original) 敏捷诊断工具V 2.6.0 # 根因分析场景：clog 盘满 clog_disk_full 更新时间：2026-04-13 17:11:13 [编辑](https://github.com/oceanbase/odt-doc/edit/V2.6.0/zh-CN/460.rca/466.rca-clog_disk_full.md) ## 适用场景 本文适用于 clog 盘满的场景，依赖内部表数据进行分析。 目前支持 OceanBase 数据库 V4.0.0.0 及之后的版本。使用 obdiag 时需根据选择在 `~/.obdiag/config.yml` 文件中配置集群信息，或在命令中通过 `--config` 选项配置集群信息。 #### 注意 由于内部数据存在及时性，所以仅支持对当前时的数据进行分析。 ## 可导入环境变量 | 变量名 | 是否必选 | 数据类型 | 默认值 | 说明 | | --- | --- | --- | --- | --- | | since | 否 | string | "30m" | 所需日志的分析时间 | ## 使用示例 配置分析场景，默认执行示例如下： ```shell obdiag rca run --scene=clog_disk_full ``` ## 结果示例 根据如下输出可发现节点中的 1005 日志流存在回放问题，需要进一步追踪。 ```shell +-------------------------------------------------------------------------------------------------------------+ | record | +------+------------------------------------------------------------------------------------------------------+ | step | info | +------+------------------------------------------------------------------------------------------------------+ | 1 | check error tenant_ls_data is {'tenant_id': 1xxx, 'ls_id': 1005, 'ip': 'xxx.xxx.xxx.xxx', 'port': | | | 2882} | | 2 | __check_checkpoint | | 3 | find log_disk_full about checkpoint in | | | ./rca//obdiag_clog_disk_full_20240719104714/tenant_id_1xxx/ls_id_1005/checkpoint/ | | 4 | is_clog_checkpoint_stuck is True | | 5 | the log is [2024-07-19 10:45:08.043682] INFO [STORAGE] update_clog_checkpoint | | | (ob_checkpoint_executor.cpp:158) [52704][T1xxx_CKClogDis][T1xxx][Y0-0000000000000000-0-0] [lt=0] | | | [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1720543850356918166, v:0}, | | | checkpoint_scn_in_ls_meta={val:1720543850356918166, v:0}, ls_id={id:1005}, | | | service_type="TRANS_SERVICE") | | 6 | stuck_service_type is TRANS_SERVICE | | 7 | __check_checkpoint end | | 8 | __get_min_ckpt_type start | | 9 | get min ckpt type is on [2024-07-19 10:45:59.748107] INFO [STORAGE.TRANS] get_rec_scn | | | (ob_ls_tx_service.cpp:507) [52704][T1xxx_CKClogDis][T1xxx][Y0-0000000000000000-0-0] [lt=0] | | | [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", | | | common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7f4a69ac8510, is_inited:true, | | | freezing_scn:{val:1720543850382118208, v:0}, ls:{ls_meta:{tenant_id:1xxx, ls_id:{id:1005}, | | | ls_create_status:1, clog_checkpoint_scn:{val:1720543850356918166, v:0}, | | | clog_base_lsn:{lsn:2367456215040}, rebuild_seq:1, migration_status:0, gc_state_:1, | | | offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:8}, | | | replayable_point:{val:1720662756177146368, v:0}, | | | tablet_change_checkpoint_scn:{val:1720543765175444272, v:0}, all_id_meta:{id_meta:[{limited_id:1, | | | latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, | | | latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, | | | latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:1720536600653472428, v:0}, | | | rebuild_info:{status:{status:0}, type:{type:0}}}, switch_epoch:1, log_handler:{role:2, | | | proposal_id:17, palf_env_:0x7f1de657c030, is_in_stop_state_:false, is_inited_:true, id_:1005}, | | | restore_handler:{is_inited:true, is_in_stop_state:false, id:1005, proposal_id:9223372036854775807, | | | role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, | | | max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, | | | max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, | | | trace_id:Y0-0000000000000000-0-0, error_type:1048580, err_lsn:{lsn:18446744073709551615}}, | | | task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, | | | tablet_gc_handler:{tablet_persist_trigger:2, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, | | | transfer_start_scn:{val:18446744073709551615, v:3}}}}, min_rec_scn={val:1720543850356918166, v:0}, | | | ls_id_={id:1005}) | | 10 | min_checkpoint_tx_log_type is MDS_TABLE_TYPE | | 11 | min_checkpoint_scn is 1720543850356918166 | | 12 | check_min_ckpt_type is True | | 13 | __get_min_ckpt_type end | | 14 | __check_replay_stuck start | | 15 | check_replay_stuck is True. the line: [2024-07-19 10:47:33.367691] INFO [CLOG] | | | get_min_unreplayed_log_info (ob_replay_status.cpp:984) | | | [4600][T1xxx_TenantWea][T1xxx][Y0-0000000000000000-0-0] [lt=14] | | | get_min_unreplayed_log_info(lsn={lsn:2367477760943}, scn={val:1720543850382118209, v:0}, | | | this={ls_id_:{id:1005}, is_enabled_:true, is_submit_blocked_:false, role_:2, | | | err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, | | | err_ts_:0, err_ret_:0}, ref_cnt_:4, post_barrier_lsn_:{lsn:2367477760943}, pending_task_count_:1, | | | submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1721357253367659, | | | err_info_:{has_fatal_error_:false, fail_ts_:1721213879495661, fail_cost_:176528190, | | | ret_code_:-4023}}, next_to_submit_lsn_:{lsn:2367477763353}, | | | next_to_submit_scn_:{val:1720543850382118210, v:0}, base_lsn_:{lsn:2332293316608}, | | | base_scn_:{val:1720468881546872775, v:0}, iterator_:{iterator_impl:{buf_:0x7f00f0805000, | | | next_round_pread_size:2121728, curr_read_pos:387999, curr_read_buf_start_pos:0, | | | curr_read_buf_end_pos:2121728, log_storage_:{IteratorStorage:{start_lsn:{lsn:2367477375410}, | | | end_lsn:{lsn:2367479497138}, read_buf:{buf_len_:2125824, buf_:0x7f00f0805000}, block_size:67104768, | | | log_storage_:0x7f2b1a5bf2f0}, IteratorStorageType::"DiskIteratorStorage"}, | | | curr_entry_is_raw_write:true, curr_entry_size:99, prev_entry_scn:{val:1720543850382118210, v:0}, | | | curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:67, scn_:{val:1720543850382118210, | | | v:0}, data_checksum:927461962, flag:1}}, init_mode_version:14, accumulate_checksum:1210730574, | | | curr_entry_is_padding:0, padding_entry_size:99, padding_entry_scn:{val:1720543850382118210, | | | v:0}}}}}) | | 16 | get min unreplayed log info is [2024-07-19 10:47:33.367691] INFO [CLOG] get_min_unreplayed_log_info | | | (ob_replay_status.cpp:984) [4600][T1xxx_TenantWea][T1xxx][Y0-0000000000000000-0-0] [lt=14] | | | get_min_unreplayed_log_info(lsn={lsn:2367477760943}, scn={val:1720543850382118209, v:0}, | | | this={ls_id_:{id:1005}, is_enabled_:true, is_submit_blocked_:false, role_:2, | | | err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, | | | err_ts_:0, err_ret_:0}, ref_cnt_:4, post_barrier_lsn_:{lsn:2367477760943}, pending_task_count_:1, | | | submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1721357253367659, | | | err_info_:{has_fatal_error_:false, fail_ts_:1721213879495661, fail_cost_:176528190, | | | ret_code_:-4023}}, next_to_submit_lsn_:{lsn:2367477763353}, | | | next_to_submit_scn_:{val:1720543850382118210, v:0}, base_lsn_:{lsn:2332293316608}, | | | base_scn_:{val:1720468881546872775, v:0}, iterator_:{iterator_impl:{buf_:0x7f00f0805000, | | | next_round_pread_size:2121728, curr_read_pos:387999, curr_read_buf_start_pos:0, | | | curr_read_buf_end_pos:2121728, log_storage_:{IteratorStorage:{start_lsn:{lsn:2367477375410}, | | | end_lsn:{lsn:2367479497138}, read_buf:{buf_len_:2125824, buf_:0x7f00f0805000}, block_size:67104768, | | | log_storage_:0x7f2b1a5bf2f0}, IteratorStorageType::"DiskIteratorStorage"}, | | | curr_entry_is_raw_write:true, curr_entry_size:99, prev_entry_scn:{val:1720543850382118210, v:0}, | | | curr_entry:{LogEntryHeader:{magic:19528, version:1, log_size:67, scn_:{val:1720543850382118210, | | | v:0}, data_checksum:927461962, flag:1}}, init_mode_version:14, accumulate_checksum:1210730574, | | | curr_entry_is_padding:0, padding_entry_size:99, padding_entry_scn:{val:1720543850382118210, | | | v:0}}}}}) | | 17 | log_time - replay_scn_time : 2024-07-19 10:47:33.367691 - 2024-07-10 00:50:50.382118 | | 18 | datetime.timedelta(minutes=0.5): 0:00:30 | | 19 | log_time - replay_scn_time > datetime.timedelta(minutes=0.5) is True | | 20 | log_time:{0}, replay_scn_time:{1} | | 21 | check_replay_stuck is True | | 22 | check_replay_stuck is True. Please check replay status | | 23 | __check_replay_stuck end | | 24 | __check_dump_stuck start | | 25 | check_dump_stuck is False | | 26 | __check_dump_stuck end | | 27 | __check_data_disk_full start | | 28 | check_data_disk_full is False | | 29 | __check_data_disk_full end | | 30 | __check_too_many_sstable start | | 31 | check_too_many_sstable is False | | 32 | __check_too_many_sstable end | | 33 | check end | +------+------------------------------------------------------------------------------------------------------+ The suggest: min_checkpoint_tx_log_type is MDS_TABLE_TYPE. please check it. ``` ### 参考博客 [OceanBase诊断调优 (十二) ——clog盘满问题排查](https://open.oceanbase.com/blog/12986354194) 上一篇 下一篇 ![有帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*y6ocSqN8cqsAAAAAAAAAAAAAARQnAQ)![无帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*BG9IQJyLHF8AAAAAAAAAAAAAARQnAQ)![反馈](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*eTWdQKCRKHwAAAAAAAAAAAAAARQnAQ)[AI](https://www.oceanbase.com/obi) 咨询热线