OCP takeOverPreCheck timeout-OceanBase数据库使用指南

适用版本

OAT 各个版本
OCP 各个版本

问题现象

部署 OCP 时遇到如下报错。

[2023-02-20 12:24:11,099] {subprocess.py:78} INFO - [2023-02-20 12:24:11.098842] INFO [start takeOverPreCheck, already wait 0s][2023-02-20 12:24:11,504] {subprocess.py:78} INFO - {}[2023-02-20 12:24:41,511] {subprocess.py:78} INFO - [2023-02-20 12:24:41.509901] INFO [start takeOverPreCheck, already wait 30s][2023-02-20 12:24:42,726] {subprocess.py:78} INFO - {}[2023-02-20 12:25:12,733] {subprocess.py:78} INFO - [2023-02-20 12:25:12.732371] INFO [start takeOverPreCheck, already wait 60s][2023-02-20 12:25:12,951] {subprocess.py:78} INFO - {}[2023-02-20 12:25:42,957] {subprocess.py:78} INFO - [2023-02-20 12:25:42.956420] INFO [start takeOverPreCheck, already wait 90s][2023-02-20 12:25:43,174] {subprocess.py:78} INFO - {}[2023-02-20 12:26:13,180] {subprocess.py:78} INFO - [2023-02-20 12:26:13.179513] INFO [start takeOverPreCheck, already wait 120s][2023-02-20 12:26:13,398] {subprocess.py:78} INFO - {}[2023-02-20 12:26:43,404] {subprocess.py:78} INFO - [2023-02-20 12:26:43.403090] INFO [start takeOverPreCheck, already wait 150s][2023-02-20 12:26:43,635] {subprocess.py:78} INFO - {}[2023-02-20 12:27:13,641] {subprocess.py:78} INFO - [2023-02-20 12:27:13.640091] ERROR [takeOverPreCheck timeout, failed]

问题原因

安装 OCP 时一般会去接管 MetaDB 集群，这一步如果添加 MetaDB 的主机、接管预检查或者接管失败，都会导致 takeOverPreCheck timeout。

解决方法

当执行到这一步遇到上述报错时，OCP 的白屏界面已经可以登录了，打开 OCP 的白屏界面，检查一下任务列表里面是否有失败的任务。

比如，添加 MetaDB 的主机失败报错。

2023-02-20 13:00:46.136 ERROR 48 --- [pool-manual-subtask-executor8,8bfa724ff0b345ff,c888e4296f07] com.alipay.ocp.core.util.ExceptionUtils  : Checked Exception: com.alipay.ocp.core.exception.UnexpectedException occurred with code error.common.unexpected, and args [port is occupied, 62888]2023-02-20 13:00:46.137  INFO 48 --- [pool-manual-subtask-executor8,8bfa724ff0b345ff,c888e4296f07] c.a.o.c.m.t.model.SubtaskInstanceEntity  : Set state for subtask: 3, current state: RUNNING, new state: FAILED2023-02-20 13:00:46.140  WARN 48 --- [pool-manual-subtask-executor8,8bfa724ff0b345ff,c888e4296f07] c.a.o.c.t.engine.runner.RunnerFactory    : Execute task failed, subtask=SubtaskInstanceEntity{id=3, name=Install ocp agent, state=FAILED, operation=RETRY, className=com.alipay.ocp.service.task.business.host.InstallOcpAgentTask, seriesId=2, startTime=2023-02-20T13:00:44.268+08:00, endTime=2023-02-20T13:00:46.139+08:00}, failedMessage=An unknown error has occurred. Cause: port is occupied. Error message: 62,888. Contact the administrator.com.alipay.ocp.core.exception.UnexpectedException: [OCP UnexpectedException]: status=500 INTERNAL_SERVER_ERROR, errorCode=COMMON_UNEXPECTED, args=port is occupied,62888

出现该问题是因为之前这台服务器上安装过 OCP Agent，这次重新部署 MetaDB 前旧的 OCP Agent 没有被卸载/OCP Agent进程仍在运行，导致 62888 端口被占用。

解决方法是手工停止旧的 OCP Agent 进程。

[root@localhost /home/admin]# netstat -anp | grep :62888
tcp        0      0 0.0.0.0:62888           0.0.0.0:*               LISTEN      41805/ocp_mgragent

[root@localhost /home/admin]# /home/admin/ocp_agent/bin/ocp_agentctl status
{"successful":true,"message":{"state":"running","ready":true,"version":"","pid":41799,"socket":"/home/admin/ocp_agent/run/ocp_agentd.41799.sock","services":{"ocp_mgragent":{"state":"running","version":"4.0.2-20230207","pid":41805,"startAt":1676101458397222041,"socket":"/home/admin/ocp_agent/run/ocp_mgragent.41805.sock","endAt":-6795364578871345152},"ocp_monagent":{"state":"running","version":"4.0.2-20230207","pid":41806,"startAt":1676101458398219753,"socket":"/home/admin/ocp_agent/run/ocp_monagent.41806.sock","endAt":-6795364578871345152}},"dangling":null},"error":""}

[root@localhost /home/admin]# /home/admin/ocp_agent/bin/ocp_agentctl stop
{"successful":true,"message":"ok","error":""}

[root@localhost /home/admin]# ps -ef | grep ocp | grep agent
[root@localhost /home/admin]#

然后重试失败的任务就可以了。

注意：这边如果修改了 OCP 的登录密码，记得需要将 OCP 的密码重置为默认的初始密码，然后再重试失败的任务（OAT V4.1.1及之后版本不需要再重置密码为初始密码，直接重试即可）。