使用示例

分析在线日志

本节假设 OBServer 节点的 IP 为 10.10.10.1，端口为 2881，以分析用户 root@mysql 在 2024-01-09 16:23:00 到当前时间之间日志，并将分析结果写入 /data/logminer_output/ 文件为例。具体的操作命令如下：

[admin@test001 ~]$ oblogminer -c "10.10.10.1:2881" -u "root@mysql" -p "***" -s "2024-01-09 16:23:00" -o "file:///data/logminer_output/"

分析归档日志

本节以分析归档目录 /data/log_archive/ 下 2024-01-09 16:23:00 到 2024-01-10 10:00:00 之间的日志，并将分析结果写入 /data/logminer_output/ 为例。具体的操作命令如下：

[admin@test001 ~]$ oblogminer -a "file:///data/log_archive/" -s "2024-01-09 16:23:00" -e "2024-01-10 10:00:00" -o "file:///data/logminer_output/"

分析结果示例

执行命令后会在设置的输出文件目录下生成如下文件：

./logminer_output/
├── 0.csv
├── CHECKPOINT
├── COMMIT_INDEX
├── CONFIG
└── META
    └── 0.meta

目录介绍

本节简单介绍目录中不同文件的作用。

• 0.csv：输出的数据文件，默认存储为 CSV 格式，可在命令中通过配置 -f/--record_format 指定数据文件的格式。文件名为从 0 开始连续递增的整数。输出完一个数据文件后会在 META 目录写入对应当前数据文件的 meta 文件，文件名与数据文件名称相同。

• CHECKPOINT：记录分析进度。

• COMMIT_INDEX：索引文件，记录了数据文件对应的时间戳。

• CONFIG：记录本次日志分析的配置信息。

• META：输出完一个数据文件后会在 META 目录写入对应数据文件的 meta 文件，文件名与数据文件名称相同，后缀是 .meta。记录相应数据文件对应的时间戳和数据大小。内容示例如下：

  MIN_COMMIT_TS=1704788579999999
  MAX_COMMIT_TS=1704788633200501
  DATA_LEN=1066
  

数据文件介绍

可在执行命令中配置 -f/--record_format 指定数据文件的格式，可配置为：CSV、JSON、REDO_ONLY 或 UNDO_ONLY。不同格式的数据文件介绍如下。

• CSV 格式

  数据文件后缀为 .csv，内容示例如下，具体字段的含义可参见下文 字段介绍。

  TENANT_ID,TRANS_ID,PRIMARY_KEY,TENANT_NAME,DATABASE_NAME,TABLE_NAME,OPERATION,OPERATION_CODE,COMMIT_SCN,COMMIT_TIMESTAMP,SQL_REDO,SQL_UNDO,ORG_CLUSTER_ID
  1004,0,"","mysql","test","","DDL",4,1704788606147445567,"2024-01-09 16:23:26.147445","create table t(a int, b varchar(100))","",1
  1004,42590,"","","","","BEGIN",5,1704788614963917100,"2024-01-09 16:23:34.963917","","",1
  1004,42590,"","mysql","test","t","INSERT",1,1704788614963917100,"2024-01-09 16:23:34.963917","INSERT INTO `test`.`t` (`a`, `b`) VALUES (1, 'abc');","DELETE FROM `test`.`t` WHERE `a`=1 AND `b`='abc' LIMIT 1;",1
  1004,42590,"","","","","COMMIT",6,1704788614963917100,"2024-01-09 16:23:34.963917","","",1
  1004,42604,"","","","","BEGIN",5,1704788619155575003,"2024-01-09 16:23:39.155575","","",1
  1004,42604,"","mysql","test","t","DELETE",3,1704788619155575003,"2024-01-09 16:23:39.155575","DELETE FROM `test`.`t` WHERE `a`=1 AND `b`='abc' LIMIT 1;","INSERT INTO `test`.`t` (`a`, `b`) VALUES (1, 'abc');",1
  1004,42604,"","","","","COMMIT",6,1704788619155575003,"2024-01-09 16:23:39.155575","","",1
  

• JSON 格式

  数据文件后缀为 .json，每条记录输出为一行，通过换行符（\n）分隔。JSON 格式数据文件内容示例如下，具体字段的含义可参见下文 字段介绍。

  {"TENANT_ID":1002,"TRANS_ID":4415602,"PRIMARY_KEY":"","TENANT_NAME":"","DATABASE_NAME":"","TABLE_NAME":"","OPERATION":"BEGIN","OPERATION_CODE":5,"COMMIT_SCN":1708391913195415000,"COMMIT_TIMESTAMP":"2024-02-20 01:18:33.195415","SQL_REDO":"","SQL_UNDO":"","ORG_CLUSTER_ID":1}
  {"TENANT_ID":1002,"TRANS_ID":4415602,"PRIMARY_KEY":"","TENANT_NAME":"mysql","DATABASE_NAME":"test","TABLE_NAME":"t1","OPERATION":"INSERT","OPERATION_CODE":1,"COMMIT_SCN":1708391913195415000,"COMMIT_TIMESTAMP":"2024-02-20 01:18:33.195415","SQL_REDO":"INSERT INTO `test`.`t1` (`id`, `name`) VALUES (1, 'aaaa');","SQL_UNDO":"DELETE FROM `test`.`t1` WHERE `id`=1 AND `name`='aaaa' LIMIT 1;","ORG_CLUSTER_ID":1}
  {"TENANT_ID":1002,"TRANS_ID":4415602,"PRIMARY_KEY":"","TENANT_NAME":"","DATABASE_NAME":"","TABLE_NAME":"","OPERATION":"COMMIT","OPERATION_CODE":6,"COMMIT_SCN":1708391913195415000,"COMMIT_TIMESTAMP":"2024-02-20 01:18:33.195415","SQL_REDO":"","SQL_UNDO":"","ORG_CLUSTER_ID":1}
  

• REDO_ONLY 格式

  仅输出 SQL_REDO 结果，数据文件后缀为 .sql。

• UNDO_ONLY 格式

  仅输出 SQL_UNDO 结果，数据文件后缀为 .sql。

字段详细介绍如下：