基于湖库一体架构,统一管理结构化、半结构化与非结构化等多模态数据,一个系统承载事务处理、实时分析与 AI 工作负载。
OMS Store 异常 Data lost while parsing record
更新时间:2026-06-23 06:11
本文为您介绍 Store 组件出现 Data lost while parsing record 异常的原因和解决方法。
适用版本
适用于 OceanBase 迁移服务(OceanBase Migration Service,OMS)V4.1.0 及之后版本。
问题描述
迁移 Oracle 数据库的数据至 OceanBase 数据库 Oracle 租户时,OMS 的 Store 组件遇到 "Data lost while parsing record" 报错而异常终止。详细报错如下:
[2023-09-06 14:00:19.922] [ERROR] [Oracle LogRecord Serializer Instance#1] [Serializer failed, caused by](LogRecordSerializer.java:53)
java.lang.RuntimeException: [Unexpected behavior] Data lost while parsing record: 755
at com.oceanbase.xlog.util.storage.core.FileStore.parseCacheRecord(FileStore.java:320)
at com.oceanbase.xlog.util.storage.core.FileStore.get(FileStore.java:230)
at com.oceanbase.xlog.util.BigList.get(BigList.java:49)
at com.taobao.drc.logminer.aggregator.OracleLogKVStoreAggregator.dealCommittedTransaction(OracleLogKVStoreAggregator.java:375)
at com.taobao.drc.logminer.aggregator.OracleLogKVStoreAggregator.dealStartCommitRollbackDDLTypeRecord(OracleLogKVStoreAggregator.java:286)
at com.taobao.drc.logminer.aggregator.OracleLogKVStoreAggregator.put(OracleLogKVStoreAggregator.java:198)
at com.taobao.drc.logminer.LogRecordSerializer.doSerialize(LogRecordSerializer.java:67)
at com.taobao.drc.logminer.LogRecordSerializer.run(LogRecordSerializer.java:51)
[2023-09-06 14:00:19.922] [INFO] [Oracle LogRecord Serializer Instance#1] [Stop to serialize Oracle LogRecord!!!](LogRecordSerializer.java:63)
[2023-09-06 14:00:19.922] [INFO] [Record Dispatcher] [Stop to dispatch records!!!](OracleBackRecordQueue.java:303)
[2023-09-06 14:00:19.922] [INFO] [Record Selector-10] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.923] [INFO] [Record Selector-8] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.923] [INFO] [Oracle LogRecord Converter Core] [Stop to convert Oracle LogRecord!!!](LogRecordConverterCore.java:109)
[2023-09-06 14:00:19.925] [INFO] [Record Selector-4] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.924] [INFO] [Record Selector-0] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.924] [INFO] [Record Selector-1] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.925] [INFO] [Record Selector-7] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:19.971] [INFO] [Record Selector-11] [select for update record back query, not find result. select sql:select "REGION", "ORDERID", "SN", "DETAILDATA", "DATALEN", "RECDATE" from "NGCRM_XX"."CS_REC_DETAIL" where ROWID = ?, params:AAAWpTAEnAAMXK4AAC](OracleBackRecordQueue.java:955)
[2023-09-06 14:00:19.972] [INFO] [Record Selector-12] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.016] [INFO] [Record Selector-13] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.045] [ERROR] [pool-2-thread-1] [runtime exception](LogMinerConnectorTask.java:197)
java.lang.RuntimeException: Serializer failed
at com.taobao.drc.logminer.LogRecordSerializer.run(LogRecordSerializer.java:59)
[2023-09-06 14:00:20.045] [ERROR] [pool-2-thread-1] [
at com.taobao.drc.logminer.LogRecordSerializer.run(LogRecordSerializer.java:59)
](LogMinerConnectorTask.java:198)
[2023-09-06 14:00:20.046] [INFO] [Thread-1] [Kafka Connect stopping](Connect.java:68)
[2023-09-06 14:00:20.046] [INFO] [Thread-1] [Stopping REST server](RestServer.java:154)
[2023-09-06 14:00:20.050] [INFO] [Thread-2] [Stopped ServerConnector@1ce93c18{HTTP/1.1}{0.0.0.0:0}](AbstractConnector.java:306)
[2023-09-06 14:00:20.060] [INFO] [Thread-2] [Stopped o.e.j.s.ServletContextHandler@3c4bc9fc{/,null,UNAVAILABLE}](ContextHandler.java:865)
[2023-09-06 14:00:20.061] [INFO] [Thread-1] [REST server stopped](RestServer.java:165)
[2023-09-06 14:00:20.061] [INFO] [Thread-1] [Kafka Connect stopped](Connect.java:73)
[2023-09-06 14:00:20.071] [INFO] [Record Selector-14] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.116] [INFO] [Record Selector-15] [select for update record back query, not find result. select sql:select "REGION", "ORDERID", "SN", "DETAILDATA", "DATALEN", "RECDATE" from "NGCRM_XX"."CS_REC_DETAIL" where ROWID = ?, params:AAAWpTAEnAAMXK4AAF](OracleBackRecordQueue.java:955)
[2023-09-06 14:00:20.118] [INFO] [Record Selector-16] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.171] [INFO] [Record Selector-17] [select for update record back query, not find result. select sql:select "REGION", "ORDERID", "SN", "DETAILDATA", "DATALEN", "RECDATE" from "NGCRM_XX"."CS_REC_DETAIL" where ROWID = ?, params:AAAWpTAEnAAMXK4AAG](OracleBackRecordQueue.java:955)
[2023-09-06 14:00:20.173] [INFO] [Record Selector-18] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.224] [INFO] [Record Selector-19] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.258] [INFO] [sysdate-synchronizer] [current synchronized sysdate value: 2023-09-06 14:00:20 ,processId: 12***9@stp1****asyn-3](CurrentValueSynchronizer.java:69)
[2023-09-06 14:00:20.259] [INFO] [scn-synchronizer] [current synchronized scn value: 16871455536722 ,processId: 112***9@stp1****asyn](CurrentValueSynchronizer.java:69)
[2023-09-06 14:00:20.273] [INFO] [Record Selector-20] [select for update record back query, not find result. select sql:select "REGION", "ORDERID", "SN", "DETAILDATA", "DATALEN", "RECDATE" from "NGCRM_XX"."CS_REC_DETAIL" where ROWID = ?, params:AAAWpTAEnAAMXK4AAH](OracleBackRecordQueue.java:955)
[2023-09-06 14:00:20.275] [INFO] [Record Selector-21] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.327] [INFO] [Record Selector-22] [Stop to select records!!!](OracleBackRecordQueue.java:454)
[2023-09-06 14:00:20.375] [INFO] [Record Selector-23] [Stop to select records!!!](OracleBackRecordQueue.java:454)
通过核查,报错中涉及的 rowid 已经无法从 Oracle 数据库中查询。尝试通过设置参数 deliver2store.logminer.enable_skip_revise_valid_rowid_exception=true 解决问题,但没成功,仍然报错。
deliver2store.logminer.enable_skip_revise_valid_rowid_exception 参数的默认值为 false,用于设置是否跳过修正 rowid 引起的异常。
问题原因
OMS 的 Store 组件处理大事务时序列化出现问题(原因未知)。
解决方法
您可以通过设置参数 deliver2store.logminer.enable_level_store=false,关闭 Store 组件的大事务处理机制。
请注意,设置该参数会导致内存使用量上涨。必要时请修改 Store 组件的内存配置,以免遇到 GC 问题。