划线反馈

机器可用内存不足导致 SQL 一直卡住重试 retry_info 显示 -4121

更新时间：2026-05-09 01:51

适用版本：V3.1.x、V3.2.x内容类型：Troubleshoot

问题现象

在小规格机器上执行大数据量 SQL（形如 insert into select）时预期只需要 10s 但是执行 500s 还不结束，通过查询会话详情 show processlist; 发现此 SQL 一直在重试，并且重试信息 retry_info 显示 -4121 报错，如下图。

关键信息

关键日志： Failed to do easy_buf_check_read_space。

[2024-06-06 15:02:22.689345] EDIAG [LIB] achunk_mgr.cpp:127 [9588][0][Y0-0000000000000000-0-0] [lt=20] [dc=0][errcode=-4013] low alloc fail(size=1178599424, orig_errno=0, errno=12) BACKTRACE:0x11270680 0x11257784 0x5f4a51e 0x5f4a068 0x5f49c95 0x5f572ca 0x5f35590 0x5f353c9 0x5f35afa 0x5f33978 0x5f2b70d 0x5ce2b68 0x5ce24ee 0x5ce1b47 0x5d92e69 0x5f521fc 0x5d92979 0x5d9249d 0x5d922e5 0x5cb59eb 0x5db7655 0x5dbb810 0x5d9bf15 0x5dcd714 0x5ddcf52 0x12de9f31 0x7f3e65cd1fed 0x7f3e65b5b18f
[2024-06-06 15:02:22.689568] EDIAG easy_connection.c:1593 [9588][0][Y0-0000000000000000-0-0] [lt=14] [dc=0][errcode=0] Failed to do easy_buf_check_read_space, m=0x7f3c305d5510, len=1177668844

诊断过程： 通过 processlist 内记录的 trace_id 过滤日志，可以找到远程 RPC 接收端的机器 IP 地址 send packet fail(dst="xx.xx.xx.25:2882"。

observer.log:[2024-06-06 14:45:29.685999] WDIAG [SERVER] ob_req_transport.cpp:366 [18057][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=9] [dc=0][errcode=0] send packet fail(dst="xx.xx.xx.25:2882", s=0x7f15cd415110)
[2024-06-06 14:45:29.686264] WDIAG [RPC.FRAME] ob_req_transport.h:321 [18057][0][xxxxx-xxxxx-xxxxx-xxxxx] [lt=20] [dc=0][errcode=-4121] send packet fail(easy_error=-55, req={pkt:{hdr_:{checksum_:57089278, pcode_:1292, hlen_:0, priority_:5, flags_:6151, tenant_id_:1003, priv_tenant_id_:1, session_id_:0, trace_id_:12378266687768, timeout_:9935838183, timestamp:1717656329265245, dst_cluster_id:1666258333, cost_time:{len:0, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:3562039345406, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1666258333, seq_no_:1717656329684724}, chid_:54352407, clen_:1177635686, assemble:false, msg_count:0, payload:0}}, ret=-4121)

随后到对应节点上使用 free -h 查看机器剩余可用内存，会发现剩余内存很少。