--- title: OceanBase 数据库 V3.2.x 数据备份 force_cancel 会导致备份流程卡住，退不出来-OceanBase数据库使用指南 description: 了解OceanBase数据库在实际应用中关于 OceanBase 数据库 V3.2.x 数据备份 force_cancel 会导致备份流程卡住，退不出来相关的常见问题和使用技巧，帮助您快速解决 OceanBase 数据库 V3.2.x 数据备份 force_cancel 会导致备份流程卡住，退不出来的难题。 --- 切换语言 - 简体中文 - English 划线反馈 # OceanBase 数据库 V3.2.x 数据备份 force_cancel 会导致备份流程卡住，退不出来 更新时间：2026-05-11 09:51 适用版本： V3.1.x、V3.2.x 内容类型：Troubleshoot ## 问题现象 系统租户处于 prepare 状态，普通租户处于 schedule 状态，当发起 do_cancel 时，备份的状态卡在 cancel，无法退出，无法发起下一个备份任务。 1. 备份过程中用户执行 `alter system cancel all backup force` 命令，使用 `select value from oceanbase.__all_tenant_backup_info where name = 'backup_status'` 可以看到集群备份状态一直卡在 cancel 状态，长时间不能推进。 2. 通过 trace_id 查找相关日志看到 cancel 的报错信息 ```shell grep "YBxxxxx-xxxxxx-xxxxxx-xxxxxx" observer.log.2* observer.log | vi - ``` 若看到报错文件、错误码以及报错信息完全符合，即可确认该错误。 ```shell [2023-11-07 16:46:16.519279] WDIAG [SHARE] ob_backup_path.cpp:1922 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=10] [dc=0][errcode=-4002] invalid argument!(ret=-4002, param={tenant_id:1, incarnation:1, c opy_id:0, full_set_id:397, inc_set_id:397, backup_date:20231030, compatible:0, backup_dest_ptr:0x7fa0a53636f0}, buffer_length=2048, buffer=0x7fa0a53621b0) [2023-11-07 16:46:16.519310] WDIAG [SERVER] ob_backup_operator.cpp:2784 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=14] [dc=0][errcode=-4002] fail to get backup set relative path(ret=-4002, param={tenan t_id:1, incarnation:1, copy_id:0, full_set_id:397, inc_set_id:397, backup_date:20231030, compatible:0, backup_dest_ptr:0x7fa0a53636f0}, item={tenant_id:1, backup_set_id:397, incarnation:1, copy_id:0, snapshot_v ersion:1698644686857754, prev_full_backup_set_id:0, prev_inc_backup_set_id:0, prev_backup_data_version:0, pg_count:0, macro_block_count:0, finish_pg_count:0, finish_macro_block_count:0, input_bytes:0, output_by tes:0, start_time:1698644686857754, end_time:0, compatible:0, cluster_version:0, backup_type:{type:1}, status:0, result:-4072, cluster_id:1641437681, backup_dest:"oss://xxx", backup_data_version:0, backup_schema_version:1697688796461568, partition_count:0, finish_partition_co unt:0, encryption_mode:0, passwd:"", file_status:1, start_replay_log_ts:0, date:20231030, backup_level:2}) [2023-11-07 16:46:16.519334] WDIAG [SERVER] ob_backup_operator.cpp:3264 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=12] [dc=0][errcode=-4002] failed to fill on item(ret=-4002, backup_set_file_info={tena nt_id:1, backup_set_id:397, incarnation:1, copy_id:0, snapshot_version:1698644686857754, prev_full_backup_set_id:0, prev_inc_backup_set_id:0, prev_backup_data_version:0, pg_count:0, macro_block_count:0, finish_ pg_count:0, finish_macro_block_count:0, input_bytes:0, output_bytes:0, start_time:1698644686857754, end_time:0, compatible:0, cluster_version:0, backup_type:{type:1}, status:0, result:-4072, cluster_id:16414376 81, backup_dest:"oss://xxx", backup_data_version:0, backup_schema_ version:1697688796461568, partition_count:0, finish_partition_count:0, encryption_mode:0, passwd:"", file_status:1, start_replay_log_ts:0, date:20231030, backup_level:2}) [2023-11-07 16:46:16.519395] WDIAG [RS] ob_root_backup.cpp:3177 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=49] [dc=0][errcode=-4002] failed to insert tenant backup set file info(ret=-4002, backup_set_f ile_info={tenant_id:1, backup_set_id:397, incarnation:1, copy_id:0, snapshot_version:1698644686857754, prev_full_backup_set_id:0, prev_inc_backup_set_id:0, prev_backup_data_version:0, pg_count:0, macro_block_co unt:0, finish_pg_count:0, finish_macro_block_count:0, input_bytes:0, output_bytes:0, start_time:1698644686857754, end_time:0, compatible:0, cluster_version:0, backup_type:{type:1}, status:0, result:-4072, clust er_id:1641437681, backup_dest:"oss://xxx", backup_data_version:0, backup_schema_version:1697688796461568, partition_count:0, finish_partition_count:0, encryption_mode:0, passwd:"", file_status:1, start_replay_log_ts:0, date:20231030, backup_level:2}) [2023-11-07 16:46:16.519420] WDIAG [RS] ob_root_backup.cpp:2142 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=11] [dc=0][errcode=-4002] failed to insert tenant backup set file failed(ret=-4002, sys_backup _task={tenant_id:1, backup_set_id:397, incarnation:1, snapshot_version:1698644686857754, prev_full_backup_set_id:0, prev_inc_backup_set_id:0, prev_backup_data_version:0, pg_count:0, macro_block_count:0, finish_ pg_count:0, finish_macro_block_count:0, input_bytes:0, output_bytes:0, start_time:1698644686857754, end_time:0, compatible:0, cluster_version:0, backup_type:{type:1}, status:1, device_type:0, result:-4072, clus ter_id:1641437681, backup_dest:{device_type:0, root_path:"oss://xxx", type:"OSS"}, backup_data_version:0, backup_schema_version:1697688796461568, partition_count:0, finish_partition_count:0, encrypt ion_mode:0, passwd:"", is_mark_deleted:false, start_replay_log_ts:0, date:20231030, copy_id:0, backup_level:2}) [2023-11-07 16:46:16.519665] WDIAG [RS] ob_root_backup.cpp:1056 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=11] [dc=0][errcode=-4002] failed to do sys tenant cancel(ret=-4002, info={tenant_id:1, backup_ set_id:397, incarnation:1, backup_dest:"oss://xxx", backup_backup_ dest:"", backup_snapshot_version:1698644686857754, backup_schema_version:1697688796461568, backup_data_version:1169, detected_backup_region:"", backup_type:{type:1}, backup_status:{status:4, status_str:"CANCEL" }, backup_task_id:0, copy_id:0, encryption_mode:0}) [2023-11-07 16:46:16.519686] WDIAG [RS] ob_root_backup.cpp:554 [2697475][0][YBxxxxx-xxxxxx-xxxxxx-xxxxxx] [lt=13] [dc=0][errcode=-4002] failed to do cancel(ret=-4002, info={tenant_id:1, backup_set_id:397, incarnation:1, backup_dest:"oss://xxx", backup_backup_dest:"", bac kup_snapshot_version:1698644686857754, backup_schema_version:1697688796461568, backup_data_version:1169, detected_backup_region:"", backup_type:{type:1}, backup_status:{status:4, status_str:"CANCEL"}, backup_ta sk_id:0, copy_id:0, encryption_mode:0}) ``` ## 关键诊断信息 ### 触发条件 系统租户处于 prepare 状态，普通租户处于 schedule 状态，发起 do_cancel，备份的状态卡在 cancel。 ### 事后诊断 1. 备份过程中用户执行 `alter system cancel all backup force` 命令，使用 `select value from oceanbase.__all_tenant_backup_info where name = 'backup_status'` 可以看到集群备份状态一直卡在 cancel 状态，长时间不能推进。 2. 通过 trace_id 查找相关日志看到 cancel 的报错信息 ``` ``` ## 问题原因 发起备份时，系统租户处于 prepare 状态，普通租户处于 schedule 状态，当 do_scheduler 出错时，导致构造的备份 task 没有插入内部表。当发起 force_cancel 时，因没有在内部表内查到任务，会对任务进行构造，构造新任务 task struct 时，没有将 compatible 赋值，在后续的流程中进行了对 task 的有效性进行了校检，compatible 需要大于 0。导致出现了 4002 错误。从而使得备份卡住。 ## 问题的风险及影响 会导致 force_cancel 备份任务一直无法退出，下一轮新的备份不能发起。 ## 影响租户 影响 OceanBase 数据库中的 SYS 租户和 Oracle 租户以及 MySQL 租户。 ## 影响版本 OceanBase 数据库企业版 V3.1.2 GA（oceanbase-3.1.2-20210618150922）及之后版本、V3.2.3 GA（oceanbase-3.2.3.0-20220418212020）及之后版本、V3.2.4 GA（oceanbase-3.2.4.0-100000072022102819）及之后版本。 ## 解决方法 升级至问题已修复版本。目前已修复的版本包括 OceanBase 数据库企业版 V3.2.3 BP11（oceanbase-3.2.3.3-111000032024070822）、V3.2.4 BP7（oceanbase-3.2.4.7-107000012023113010）。 ## 规避方式 调整发起备份语句级别事务超时时间。 线上应急手段（人工介入恢复）。 #### 警告 更新内部表 `__all_tennat_backup_info` 存在风险，请勿自行操作。如果需要修改，请咨询 OceanBase 技术支持。 1. 用 root 权限链接到 1002 租户，执行 SQL。 ```shell update __all_tenant_backup_info set value = 'STOP' where name = 'backup_status'; ``` 2. 用 root 权限链接到 SYS 租户，执行 SQL。 ```shell update __all_tenant_backup_info set value = 'CLEANUP' where name = 'backup_status'; ``` 3. 等待 SYS 租户的 `__all_tennat_backup_info` 中的 `backup_status` 变成 STOP 就恢复了。 ```shell update __all_tenant_backup_info set value = '' where name = 'backup_data_version'; update __all_tenant_backup_info set value = '' where name = 'backup_schema_version'; update __all_tenant_backup_info set value = '' where name = 'backup_schema_snapshot_version'; update __all_tenant_backup_info set value = 'STOP' where name = 'backup_status'; update __all_tenant_backup_info set value = '' where name = 'backup_type'; ``` Previous [OceanBase 数据库集群机器因 clog 被回收导致日志备份中断（日志备份状态变为 INTERRUPTED）](https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000003252549) Next [OceanBase 数据库 V4.x 版本中备份任务导致 RS 调度线程高 CPU 占用的问题分析与规避建议](https://www.oceanbase.com/knowledge-base/oceanbase-database-1000000003472590) ![有帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*y6ocSqN8cqsAAAAAAAAAAAAAARQnAQ)![无帮助](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*BG9IQJyLHF8AAAAAAAAAAAAAARQnAQ)![反馈](https://gw.alipayobjects.com/mdn/ob_asset/afts/img/A*eTWdQKCRKHwAAAAAAAAAAAAAARQnAQ)[AI](https://www.oceanbase.com/obi) 咨询热线