首批通过分布式安全可靠测评,为关键业务系统打造
索引混合搜索(SQL 接口)
更新时间:2026-08-05 14:51:35
本文档介绍 OceanBase 的混合搜索 SQL 接口。该接口通过 HYBRID_SEARCH 关键字直接在一条 SELECT 查询语句中组合全文、向量搜索与过滤条件,并返回按相关性融合后的结果。
混合搜索(Hybrid Search)结合了基于向量的语义搜索和基于全文索引的关键词搜索,通过综合排序提供更准确、全面的搜索结果。向量搜索擅长语义近似匹配,但对精确的关键字、数字和专有名词等匹配能力较弱,而全文搜索能有效弥补这一不足。因此,混合搜索已成为向量数据库的关键特性之一,广泛应用于各类产品中。
功能限制
- 当前仅支持在堆表上使用
HYBRID_SEARCHSQL 子句。 - 向量搜索要求目标向量列已创建向量索引。多路向量搜索当前仅支持稠密向量列。
- 全文搜索要求目标文本列已创建全文索引。如果全文列上是多个列的联合索引,则对于混合搜索无效。
- 标量过滤、JSON/ARRAY 过滤在无索引时也可执行,但建议创建相应索引以获得更好性能。
- 当前版本向量索引类型仅支持 HNSW 系列。
- 仅支持行存表。
- 不支持在生成列上使用。
本节只列出功能限制,需结合文末相关文档 HYBRID_SEARCH 中的语法限制内容一并理解。
语法
SELECT select_list
FROM HYBRID_SEARCH(TABLE table_name, dsl_string);
参数说明如下:
table_name:目标表名,仅支持堆表(ORGANIZATION = HEAP);支持分区表和非分区表。dsl_string:JSON 字符串,用于描述全文、向量搜索、过滤、排序融合等查询语义。
完整语法、参数和限制说明请参见文末相关文档 HYBRID_SEARCH。
创建示例表并插入数据
本文与 索引混合搜索(PL 接口) 使用相同的示例表。
doc_table:用于全文、向量、标量过滤混合搜索、精排的示例表
CREATE TABLE doc_table(
c1 INT,
vector VECTOR(3),
query VARCHAR(255),
content VARCHAR(255),
VECTOR INDEX idx_vec(vector) WITH (distance=l2, type=hnsw_sq, lib=vsag),
FULLTEXT INDEX idx_query(query),
FULLTEXT INDEX idx_content(content)
) ORGANIZATION HEAP;
-- 体验全文、向量、标量过滤混合搜索能力请仅插入以下数据,不要插入精排数据
INSERT INTO doc_table VALUES
(1, '[1,2,3]', 'hello world', 'oceanbase Elasticsearch database'),
(2, '[1,2,1]', 'hello world, what is your name', 'oceanbase mysql database'),
(3, '[1,1,1]', 'hello world, how are you', 'oceanbase oracle database'),
(4, '[1,3,1]', 'real world, where are you from', 'postgres oracle database'),
(5, '[1,3,2]', 'real world, how old are you', 'redis oracle database'),
(6, '[2,1,1]', 'hello world, where are you from', 'starrocks oceanbase database');
-- 体验精排能力请只插入以下数据
INSERT INTO doc_table VALUES
(1, '[1,2,3]', 'hello world', 'OceanBase is a distributed relational database with strong consistency and high availability across multiple zones'),
(2, '[1,2,1]', 'hello world, what is your name', 'MySQL is an open-source database popular for web applications and content management systems'),
(3, '[1,1,1]', 'hello world, how are you', 'Redis is an in-memory key-value store used as database cache and message broker for fast data access'),
(4, '[1,3,1]', 'real world, where are you from', 'MongoDB is a document-oriented NoSQL database designed for flexible schema and unstructured data storage'),
(5, '[1,3,2]', 'real world, how old are you', 'TiDB is a distributed NewSQL database that supports horizontal scaling and online transaction processing'),
(6, '[2,1,1]', 'hello world, where are you from', 'PostgreSQL is a single-node relational database known for extensibility and advanced SQL compliance');
products_multi_vector:用于多路向量搜索的示例表
CREATE TABLE products_multi_vector (
product_id VARCHAR(50),
product_name VARCHAR(255),
description TEXT,
vec1 VECTOR(4),
vec2 VECTOR(4),
vec3 VECTOR(4),
VECTOR INDEX idx_vec1(vec1) WITH (distance=l2, type=hnsw_sq, lib=vsag),
VECTOR INDEX idx_vec2(vec2) WITH (distance=l2, type=hnsw_sq, lib=vsag),
VECTOR INDEX idx_vec3(vec3) WITH (distance=l2, type=hnsw_sq, lib=vsag)
) ORGANIZATION HEAP;
INSERT INTO products_multi_vector VALUES
('prod-001', 'Gamer-Pro Mechanical Keyboard', 'A responsive mechanical keyboard', '[0.5,0.1,0.6,0.9]', '[0.2,0.3,0.4,0.5]', '[0.1,0.2,0.3,0.4]'),
('prod-002', 'Gamer-Pro Headset', 'High-fidelity gaming headset', '[0.1,0.9,0.2,0]', '[0.3,0.4,0.5,0.6]', '[0.2,0.3,0.4,0.5]'),
('prod-003', 'Eco-Friendly Yoga Mat', 'A non-slip yoga mat', '[0.1,0.9,0.3,0]', '[0.4,0.5,0.6,0.7]', '[0.3,0.4,0.5,0.6]');
doc_json_array:用于带有 JSON、ARRAY 过滤的混合搜索的示例表
CREATE TABLE doc_json_array (
id INT,
created_date date,
title varchar(255),
doc_json JSON,
tags_array1 ARRAY(VARCHAR(255)),
tags_array2 ARRAY(VARCHAR(255)),
INDEX idx_multivalue_tags((CAST(doc_json->'$.tags' AS CHAR(255) ARRAY))),
INDEX idx1(id),
INDEX idx2(title),
INDEX idx_created_date(created_date)
) ORGANIZATION HEAP;
INSERT INTO doc_json_array VALUES
(1, '2023-01-01', 'doc1', '{"name":"doc1","tags":["database","oceanbase"],"metadata":{"type":"test","score":40}}', ['database','oceanbase'], ['database','mysql']),
(2, '2023-01-02', 'doc2', '{"name":"doc2","tags":["database","mysql"],"metadata":{"type":"production","score":57}}', ['database','mysql'], ['database','mysql']),
(3, '2023-01-03', 'doc3', '{"name":"doc3","tags":["database","oracle"],"metadata":{"type":"test","score":19}}', ['database','oracle'], ['database','oracle']),
(4, '2023-01-04', 'doc4', '{"name":"doc4","tags":["database","postgres"],"metadata":{"type":"production","score":14}}', ['database','postgres'], ['database','postgres']),
(6, '2023-01-06', 'doc6', '{"name":"doc6","tags":["database","starrocks"],"metadata":{"type":"production","score":25}}', ['database','starrocks'], ['database','starrocks']),
(10, '2023-01-10', 'doc10', '{"name":"doc10","tags":["mobile","ios"],"metadata":{"type":"app","score":90}}', ['mobile','ios'], ['mobile','ios']);
-- 推荐插入数据后建搜索索引(Search Index),以获得最佳搜索性能
CREATE SEARCH INDEX idx_json ON doc_json_array(doc_json);
CREATE SEARCH INDEX idx_tags_array1 ON doc_json_array(tags_array1);
CREATE SEARCH INDEX idx_tags_array2 ON doc_json_array(tags_array2);
以下给出不同使用场景的示例,包含快速入门、拓展示例:
快速入门示例
这部分提供 6 个简单的核心示例,覆盖混合搜索的常用场景,包含向量搜索、全文搜索、向量搜索 + 全文搜索(RRF 融合)、多路向量搜索、过滤条件合并、分数阈值过滤。
向量搜索
本示例搜索 doc_table 表中与向量 [1,2,3] 最相似的 3 条记录,并返回 c1 列。
SELECT c1 FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"knn": {
"field": "vector",
"k": 3,
"query_vector": "[1,2,3]"
}
}'
);
预期返回结果如下:
+------+
| c1 |
+------+
| 1 |
| 5 |
| 2 |
+------+
3 rows in set
全文搜索
本示例搜索 doc_table 表中 content 列包含 oceanbase mysql 的 4 条记录,并返回所有列。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {
"match": {"content": "oceanbase mysql"}
}
}'
);
预期返回结果如下:
+------+---------+---------------------------------+----------------------------------+--------------------+
| c1 | vector | query | content | __score |
+------+---------+---------------------------------+----------------------------------+--------------------+
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 2.170969786679347 |
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.3503184713375797 |
| 3 | [1,1,1] | hello world, how are you | oceanbase oracle database | 0.3503184713375797 |
| 6 | [2,1,1] | hello world, where are you from | starrocks oceanbase database | 0.3503184713375797 |
+------+---------+---------------------------------+----------------------------------+--------------------+
4 rows in set
全文与向量 RRF 混合搜索
本示例语句同时执行全文搜索(匹配关键词 "oceanbase mysql")和向量搜索(搜索与向量 [1,2,3] 最相似的 5 条记录),然后通过 RRF 融合算法将两路结果合并,默认返回与查询最相关的 10 条文档,最终符合条件的共 6 条。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {
"match": {"content": "oceanbase mysql"}
},
"knn": {
"field": "vector",
"k": 5,
"query_vector": "[1,2,3]"
},
"rank": {
"rrf": {
"rank_constant": 60,
"rank_window_size": 10
}
}
}'
);
预期返回结果如下:
+------+---------+---------------------------------+----------------------------------+----------------------+
| c1 | vector | query | content | __score |
+------+---------+---------------------------------+----------------------------------+----------------------+
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.03252247488101534 |
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 0.032266458495966696 |
| 3 | [1,1,1] | hello world, how are you | oceanbase oracle database | 0.031754032258064516 |
| 5 | [1,3,2] | real world, how old are you | redis oracle database | 0.016129032258064516 |
| 6 | [2,1,1] | hello world, where are you from | starrocks oceanbase database | 0.016129032258064516 |
| 4 | [1,3,1] | real world, where are you from | postgres oracle database | 0.015625 |
+------+---------+---------------------------------+----------------------------------+----------------------+
6 rows in set
多路向量搜索
本示例语句同时在三个向量字段(vec1、vec2、vec3)上进行独立的向量搜索,各返回 5 个最相似的结果,然后通过默认的加权融合算法合并,返回综合相关性最高的文档。
SELECT * FROM HYBRID_SEARCH(
TABLE products_multi_vector,
'{
"knn": [
{"field":"vec1","k":5,"query_vector":"[0.5,0.1,0.6,0.9]"},
{"field":"vec2","k":5,"query_vector":"[0.2,0.3,0.4,0.5]"},
{"field":"vec3","k":5,"query_vector":"[0.1,0.2,0.3,0.4]"}
]
}'
);
预期返回结果如下:
+------------+-------------------------------+----------------------------------+-------------------+-------------------+-------------------+--------------------+
| product_id | product_name | description | vec1 | vec2 | vec3 | __score |
+------------+-------------------------------+----------------------------------+-------------------+-------------------+-------------------+--------------------+
| prod-002 | Gamer-Pro Headset | High-fidelity gaming headset | [0.1,0.9,0.2,0] | [0.3,0.4,0.5,0.6] | [0.2,0.3,0.4,0.5] | 2.7134181710480463 |
| prod-003 | Eco-Friendly Yoga Mat | A non-slip yoga mat | [0.1,0.9,0.3,0] | [0.4,0.5,0.6,0.7] | [0.3,0.4,0.5,0.6] | 2.6366155630441215 |
| prod-001 | Gamer-Pro Mechanical Keyboard | A responsive mechanical keyboard | [0.5,0.1,0.6,0.9] | [0.2,0.3,0.4,0.5] | [0.1,0.2,0.3,0.4] | 2.6237901221768167 |
+------------+-------------------------------+----------------------------------+-------------------+-------------------+-------------------+--------------------+
3 rows in set
拓展示例
这部分提供一些更复杂的拓展示例,涵盖混合搜索的复杂场景,包含过滤条件合并、分数阈值过滤、复杂类型过滤(JSON/ARRAY 等)、向量和全文搜索的加权混合搜索、WRRF 混合、归一化、混合搜索分数查询、Rerank 精排。
过滤条件合并
对于同一列在同一个 bool.must 或 bool.filter 路径下的多个标量过滤条件,这些条件仍以 AND 逻辑取交集,筛选同时满足所有条件的记录。查询优化器会在逻辑计划阶段将这些条件合并,避免执行阶段重复迭代。下例设置 c1 >= 3 与 c1 <= 5(相当于 c1 >= 3 AND c1 <= 5),因此只返回 c1 为 3、4、5 的行。
SELECT c1 FROM HYBRID_SEARCH(TABLE doc_table, '{
"query": { "bool": {
"filter": [
{"range": {"c1": {"gte" : 3}}},
{"range": {"c1": {"lte" : 5}}}
]
}}
}');
预期返回结果如下:
+------+
| c1 |
+------+
| 3 |
| 4 |
| 5 |
+------+
3 rows in set
分数阈值过滤(min_score)
注意
仅 SQL 接口支持 min_score 参数,PL 接口不支持。
本示例搜索 doc_table 表中 content 列包含 oceanbase mysql 的 2 条记录,分数阈值为 0.5,并返回所有列。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {"match": {"content":{"query": "oceanbase mysql", "boost": 0.3}}},
"knn": {"field":"vector","k":5,"query_vector":"[1,2,3]", "boost": 0.7},
"min_score": 0.5
}'
);
预期返回结果如下:
+------+---------+--------------------------------+----------------------------------+--------------------+
| c1 | vector | query | content | __score |
+------+---------+--------------------------------+----------------------------------+--------------------+
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.8050955414012738 |
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 0.7912909360038041 |
+------+---------+--------------------------------+----------------------------------+--------------------+
2 rows in set
拓展示例
这部分提供一些更复杂的拓展示例,涵盖混合搜索的复杂场景,包含复杂类型过滤(JSON/ARRAY 等)、num_candidates 指定向量候选集数量、wildcard 通配符过滤、向量和全文搜索的加权混合、WRRF 混合、归一化、混合搜索分数查询等。
标量+JSON/ARRAY 过滤
这部分示例需结合搜索索引使用。对于混合搜索场景,全文分词及关键词检索由全文索引负责,搜索索引则用于复杂类型内部路径的结构化、半结构化或简单标量条件,二者适用列类型不同,可在同一表中同时存在,并与向量索引配合使用,以提升查询性能。
除此之外,在分析型宽表场景下,搜索索引还可以叠加优化器的多索引联合查询(Index Merge)能力,自动识别并组合可用的全文和标量索引,选择最优扫描路径,以进一步提升查询性能。更多关于搜索索引的用法、语法说明和完整示例,请参考搜索索引(Search Index)。
JSON 示例
JSON_CONTAINS
JSON_CONTAINS() 函数用于判断目标 JSON 文档中是否包含指定的
candidate子文档,或在给定路径path(可选)下是否存在该元素。SELECT id, doc_json FROM HYBRID_SEARCH( TABLE doc_json_array, '{ "query": { "bool" : { "filter" : [{ "json_contains":{ "doc_json": { "candidate": "doc2", "path": "$.name" } } }] } } }');预期返回结果如下:
+------+--------------------------------------------------------------------------------------------------+ | id | doc_json | +------+--------------------------------------------------------------------------------------------------+ | 2 | {"name": "doc2", "tags": ["database", "mysql"], "metadata": {"type": "production", "score": 57}} | +------+--------------------------------------------------------------------------------------------------+ 1 row in setJSON_OVERLAPS
JSON_OVERLAPS() 函数用于判断两个 JSON 文档是否存在共同的键值对(key-value)或数组元素,
path为可选参数。SELECT id, doc_json FROM HYBRID_SEARCH( TABLE doc_json_array, '{ "query": { "bool" : { "filter" : [{ "json_overlaps":{ "doc_json": { "candidate": "[\\"database\\", \\"mysql\\"]", "path": "$.tags" } } }] } } }');预期返回结果如下:
+------+------------------------------------------------------------------------------------------------------+ | id | doc_json | +------+------------------------------------------------------------------------------------------------------+ | 1 | {"name": "doc1", "tags": ["database", "oceanbase"], "metadata": {"type": "test", "score": 40}} | | 2 | {"name": "doc2", "tags": ["database", "mysql"], "metadata": {"type": "production", "score": 57}} | | 3 | {"name": "doc3", "tags": ["database", "oracle"], "metadata": {"type": "test", "score": 19}} | | 4 | {"name": "doc4", "tags": ["database", "postgres"], "metadata": {"type": "production", "score": 14}} | | 6 | {"name": "doc6", "tags": ["database", "starrocks"], "metadata": {"type": "production", "score": 25}} | +------+------------------------------------------------------------------------------------------------------+ 5 rows in setJSON_MEMBER_OF
JSON_MEMBER_OF() 函数用于判断被检索的元素是否和 JSON 数组中的任意一个元素相同,
path为可选参数。SELECT id, doc_json FROM HYBRID_SEARCH( TABLE doc_json_array, '{ "query": { "bool" : { "filter" : [{ "json_member_of":{ "doc_json": { "candidate": "\\"database\\"" } } }] } } }');
预期返回空。
JSON_EXTRACT
JSON_EXTRACT() 函数用于从 JSON 文档中指定的路径返回数据。
-- 提取 JSON 字段 doc_json 中 key 为 "name" 的节点,判断 value 是否等于 "doc2" SELECT id, doc_json FROM HYBRID_SEARCH( TABLE doc_json_array, '{ "query": { "bool" : { "filter" : [{ "term":{"doc_json.name": "doc2"} }] } } }');预期返回结果如下:
+------+--------------------------------------------------------------------------------------------------+ | id | doc_json | +------+--------------------------------------------------------------------------------------------------+ | 2 | {"name": "doc2", "tags": ["database", "mysql"], "metadata": {"type": "production", "score": 57}} | +------+--------------------------------------------------------------------------------------------------+ 1 row in set
ARRAY 示例
语法定义:
-- ARRAY 表达式语法
{
"array_func" : {
"field_name" : "value"
}
}
- ARRAY_CONTAINS
ARRAY_CONTAINS() 函数用于判断数组中是否含有某个元素。
SELECT id, tags_array1 FROM HYBRID_SEARCH(
TABLE doc_json_array,
'{
"query": {
"bool" : {
"filter" : [
{ "array_contains": { "tags_array1" : "ios" }}
]
}
}
}');
预期返回结果如下:
+------+------------------+
| id | tags_array1 |
+------+------------------+
| 10 | ["mobile","ios"] |
+------+------------------+
1 row in set
- ARRAY_CONTAINS_ALL
ARRAY_CONTAINS_ALL() 函数用于判断一个数组是否包含另一个数组中的所有元素。
SELECT id, tags_array1 FROM HYBRID_SEARCH(
TABLE doc_json_array,
'{
"query": {
"bool" : {
"filter" : [
{ "array_contains_all": { "tags_array1": ["database", "postgres"]} }
]
}
}
}');
预期返回结果如下:
+------+-------------------------+
| id | tags_array1 |
+------+-------------------------+
| 4 | ["database","postgres"] |
+------+-------------------------+
1 row in set
- ARRAY_OVERLAPS
ARRAY_OVERLAPS() 函数用于判断两个数组是否存在交集。
SELECT id, tags_array1 FROM HYBRID_SEARCH(
TABLE doc_json_array,
'{
"query": {
"bool" : {
"filter" : [
{ "array_overlaps": { "tags_array1": ["database", "postgres", "oceanbase"]} }
]
}
}
}');
预期返回结果如下:
+------+--------------------------+
| id | tags_array1 |
+------+--------------------------+
| 1 | ["database","oceanbase"] |
| 2 | ["database","mysql"] |
| 3 | ["database","oracle"] |
| 4 | ["database","postgres"] |
| 6 | ["database","starrocks"] |
+------+--------------------------+
5 rows in set
向量候选集大小(num_candidates)
说明
本功能从 V4.6.0 BP1 版本开始支持。
num_candidates 用于指定向量搜索时的候选集规模,执行层映射为 ef_search,便于从其他搜索系统迁移。取值范围为 [k, 10000] 的整数。与 search_options.ef_search 同时设置时,以 num_candidates 为准;search_options 中的 refine_k、filter_mode 等子项仍可生效。优先级与 filter_mode 取值等完整说明及示例见文末 HYBRID_SEARCH 语法文档中相关小节。
wildcard 通配符过滤
说明
本功能从 V4.6.0 BP1 版本开始支持。
wildcard 用于标量列的通配符模糊匹配,语义等同于 SQL LIKE ... ESCAPE '\\':* 匹配任意多个字符,? 匹配任意单个字符。wildcard 可在 bool.must / bool.should 中参与算分,也可用于 bool.filter 或 knn.filter 做硬过滤。VARCHAR 列上的普通索引可走索引扫描路径加速;前缀索引、全文索引及 JSON 相关索引不支持。
更多说明与示例见文末 HYBRID_SEARCH 语法文档中的模糊匹配相关小节。
其他混合搜索场景
向量与全文搜索加权混合
加权混合是 OceanBase AI 默认的混合方式,其中全文搜索的权重为 0.3,向量搜索的权重为 0.7。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {
"match": {"content": {"query": "oceanbase mysql", "boost": 0.3}}
},
"knn": {
"field": "vector",
"k": 5,
"query_vector": "[1,2,3]",
"boost": 0.7
}
}'
);
预期返回结果如下:
+------+---------+---------------------------------+----------------------------------+---------------------+
| c1 | vector | query | content | __score |
+------+---------+---------------------------------+----------------------------------+---------------------+
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.8050955414012738 |
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 0.7912909360038041 |
| 5 | [1,3,2] | real world, how old are you | redis oracle database | 0.2333333333333333 |
| 3 | [1,1,1] | hello world, how are you | oceanbase oracle database | 0.22176220806794056 |
| 4 | [1,3,1] | real world, where are you from | postgres oracle database | 0.11666666666666665 |
| 6 | [2,1,1] | hello world, where are you from | starrocks oceanbase database | 0.1050955414012739 |
+------+---------+---------------------------------+----------------------------------+---------------------+
6 rows in set
向量与全文搜索 WRRF 混合
在 RRF 基础上支持设置 boost 权重,本示例查询 query 和 knn 中的 boost 就是 WRRF 融合的权重。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {
"match": {"content": {"query": "oceanbase mysql", "boost": 0.3}}
},
"knn": {
"field": "vector",
"k": 5,
"query_vector": "[1,2,3]",
"boost": 0.7
},
"rank": {
"rrf": {
"rank_constant": 60,
"rank_window_size": 10
}
}
}'
);
预期返回结果如下:
+------+---------+---------------------------------+----------------------------------+----------------------+
| c1 | vector | query | content | __score |
+------+---------+---------------------------------+----------------------------------+----------------------+
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.03252247488101534 |
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 0.032266458495966696 |
| 3 | [1,1,1] | hello world, how are you | oceanbase oracle database | 0.031754032258064516 |
| 5 | [1,3,2] | real world, how old are you | redis oracle database | 0.016129032258064516 |
| 6 | [2,1,1] | hello world, where are you from | starrocks oceanbase database | 0.016129032258064516 |
| 4 | [1,3,1] | real world, where are you from | postgres oracle database | 0.015625 |
+------+---------+---------------------------------+----------------------------------+----------------------+
6 rows in set
向量与全文搜索归一化
支持设置 normalizer 参数,用于对查询结果进行归一化处理,本示例设置 normalizer 为 "minmax",表示使用 Min-Max 归一化。
SELECT * FROM HYBRID_SEARCH(
TABLE doc_table,
'{
"query": {
"match": {"content": {"query": "oceanbase mysql", "boost": 0.3}}
},
"knn": {
"field": "vector",
"k": 5,
"query_vector": "[1,2,3]",
"boost": 0.7
},
"rank": {
"weighted_sum": {
"normalizer": "minmax",
"rank_window_size": 10
}
}
}'
);
预期返回结果如下:
+------+---------+---------------------------------+----------------------------------+---------------------+
| c1 | vector | query | content | __score |
+------+---------+---------------------------------+----------------------------------+---------------------+
| 1 | [1,2,3] | hello world | oceanbase Elasticsearch database | 0.7 |
| 2 | [1,2,1] | hello world, what is your name | oceanbase mysql database | 0.328 |
| 5 | [1,3,2] | real world, how old are you | redis oracle database | 0.13999999999999999 |
| 3 | [1,1,1] | hello world, how are you | oceanbase oracle database | 0 |
| 4 | [1,3,1] | real world, where are you from | postgres oracle database | 0 |
| 6 | [2,1,1] | hello world, where are you from | starrocks oceanbase database | 0 |
+------+---------+---------------------------------+----------------------------------+---------------------+
6 rows in set
Rerank 精排
注意
Rerank 精排从 V4.6.0 BP1 版本开始支持。
混搜先通过 query / knn 召回,再经 rank(或未指定时的默认融合)完成粗排;rerank 在粗排候选集上调用 AI Rerank 模型做精排,按语义相关性重排后返回 size 条结果。
使用前须先注册模型供应商,再通过 provider/model 格式指定 Rerank 模型。以下为内置供应商 aliyun-dashscope 示例(请将 access_key 替换为实际 API Key):
CALL DBMS_AI_SERVICE.REGISTER_PROVIDER('aliyun-dashscope', '{
"access_key": "sk-xxxx"
}');
注意
rerank 不能单独使用,必须与 query 或 knn 至少其一同时出现;精排在粗排之后执行。
本例先通过向量搜索召回与向量 [1,2,3] 最相似的 6 条记录,再进行精排。
SELECT c1, content FROM hybrid_search(TABLE doc_table, '{
"knn": {
"field": "vector",
"k": 6,
"query_vector": [1, 2, 3]
},
"rerank": {
"model": "aliyun-dashscope/gte-rerank-v2",
"field": "content",
"query": "distributed database for transaction processing with high availability",
"rank_window_size": 6
}
}');
预期返回结果如下:
+------+--------------------------------------------------------------------------------------------------------------------+
| c1 | content |
+------+--------------------------------------------------------------------------------------------------------------------+
| 1 | OceanBase is a distributed relational database with strong consistency and high availability across multiple zones |
| 5 | TiDB is a distributed NewSQL database that supports horizontal scaling and online transaction processing |
| 2 | MySQL is an open-source database popular for web applications and content management systems |
| 4 | MongoDB is a document-oriented NoSQL database designed for flexible schema and unstructured data storage |
| 3 | Redis is an in-memory key-value store used as database cache and message broker for fast data access |
| 6 | PostgreSQL is a single-node relational database known for extensibility and advanced SQL compliance |
+------+--------------------------------------------------------------------------------------------------------------------+
6 rows in set
tab 单路全文搜索 + Rerank 精排
本例先通过全文搜索召回包含 "database" 的文档,再进行 rerank 精排。
SELECT c1, content FROM hybrid_search(TABLE doc_table, '{
"query": {
"match" : {"content" : "database"}
},
"rerank": {
"model": "aliyun-dashscope/gte-rerank-v2",
"field": "content",
"query": "distributed database for transaction processing",
"rank_window_size": 6
}
}');
预期返回结果如下:
+------+--------------------------------------------------------------------------------------------------------------------+
| c1 | content |
+------+--------------------------------------------------------------------------------------------------------------------+
| 5 | TiDB is a distributed NewSQL database that supports horizontal scaling and online transaction processing |
| 1 | OceanBase is a distributed relational database with strong consistency and high availability across multiple zones |
| 2 | MySQL is an open-source database popular for web applications and content management systems |
| 4 | MongoDB is a document-oriented NoSQL database designed for flexible schema and unstructured data storage |
| 6 | PostgreSQL is a single-node relational database known for extensibility and advanced SQL compliance |
| 3 | Redis is an in-memory key-value store used as database cache and message broker for fast data access |
+------+--------------------------------------------------------------------------------------------------------------------+
6 rows in set
tab 混合搜索 + Rerank 精排
本示例先通过向量检索召回与向量 [1,2,3] 最相似的 4 条记录,再通过全文搜索召回包含 "database" 的文档,最后精排。
SELECT c1, content FROM hybrid_search(TABLE doc_table, '{
"knn": {
"field": "vector",
"k": 4,
"query_vector": [1, 2, 3]
},
"query": {
"match" : {"content" : "database"}
},
"rerank": {
"model": "aliyun-dashscope/gte-rerank-v2",
"field": "content",
"query": "high availability distributed relational database",
"rank_window_size": 6
}
}');
预期返回结果如下:
+------+--------------------------------------------------------------------------------------------------------------------+
| c1 | content |
+------+--------------------------------------------------------------------------------------------------------------------+
| 1 | OceanBase is a distributed relational database with strong consistency and high availability across multiple zones |
| 5 | TiDB is a distributed NewSQL database that supports horizontal scaling and online transaction processing |
| 6 | PostgreSQL is a single-node relational database known for extensibility and advanced SQL compliance |
| 2 | MySQL is an open-source database popular for web applications and content management systems |
| 4 | MongoDB is a document-oriented NoSQL database designed for flexible schema and unstructured data storage |
| 3 | Redis is an in-memory key-value store used as database cache and message broker for fast data access |
+------+--------------------------------------------------------------------------------------------------------------------+
6 rows in set
模型注册详见文末 AI 模型注册相关文档。
通过分区内并行优化性能
混合搜索默认采用单线程串行执行,因此可能会导致在数据量较大、CPU 资源充足的场景下单条查询语句的响应时间(RT)偏高。为解决此问题,引入了分区内并行执行功能,可在单个分区内用并行执行的策略来降低 RT。本功能默认关闭,需要手动开启。
说明
本功能从 V4.6.0 BP1 版本开始支持。
并行执行的具体策略说明如下:
| 并行策略 | 说明 |
|---|---|
| 多路并行 | 同一查询语句中的多条召回路径(多条 knn 路 + 全文 query 路)由不同工作线程并行执行,整体 RT 接近耗时最长的那条路径。 |
query 路径内并行 |
单条 query 查询路径内并行,包含全文、标量、JSON 类型过滤等召回路径,按数据范围切分后由多个工作线程并行扫描不同区间,进一步降低单路 RT。 |
不满足并行条件、或并行开关关闭时,执行路径会自动退化为串行。
具体使用说明及示例如下:
开启并行
混合搜索并行默认关闭。使用前需在租户内开启配置项:
ALTER SYSTEM SET _enable_hybrid_search_parallel_execution = true;
开启后效果:
- 若查询包含 ≥ 2 条召回路径(例如一条
query全文路 + 一条或多条knn路),多路并行会自动生效,无需在dsl_string字符串中额外配置。 - 在
dsl_string字符串中query.search_options对象下通过query_dop子参数控制全文/标量召回路径内部的并行度。
多路并行(开启后自动生效)
以下示例在 doc_table 上组合全文与向量搜索。开启并行开关后,query 路与 knn 路将并行执行:
SET @q = '{
"query": {
"match": {"text": "machine learning"}
},
"knn": [
{"field": "content_vector", "k": 10, "query_vector": [0.1, 0.2, 0.3, 0.4]}
],
"rank": {"weighted_sum": {"normalizer": "minmax"}},
"size": 10
}';
SELECT tid, __score
FROM HYBRID_SEARCH(table passages_test, @q);
query 路径内并行
在 dsl_string 字符串中 query.search_options 对象下通过 query_dop 控制单条全文/标量召回路径内部的并行度。
| 字段 | 类型 | 默认值 | 取值范围 | 说明 |
|---|---|---|---|---|
query_dop |
int | 1 |
[1, 128] |
1 表示路径内不并行;大于 1 时按数据范围切分并行扫描。超出范围会在 dsl_string 字符串解析阶段报错。 |
注意
query_dop 仅在并行开关开启时或通过 Hint 强制并行时生效:租户级开关关闭,或通过 Hint 强制串行时,dsl_string 字符串中的 query_dop 会被忽略,不会报错。
在 query 路上设置 query_dop = 4,并与 knn 路组合(多路并行 + 路径内并行同时生效),示例如下:
-- query_dop = 4:全文召回路径按数据范围切成 4 段并行扫描
SET @q = '{
"query": {
"search_options": {"query_dop": 4},
"match": {"text": "distributed database"}
},
"knn": [
{"field": "content_vector", "k": 50, "query_vector": [0.1, 0.2, 0.3, 0.4]}
],
"rank": {"rrf": {"rank_constant": 60}},
"size": 10
}';
SELECT tid, __score
FROM HYBRID_SEARCH(table passages_test, @q);
query_dop 参数详细信息请参见文末相关文档中 “HYBRID_SEARCH" 语法文档中的查询选项部分。
查询级开关
可在单条查询上通过 Hint 临时覆盖租户级配置,便于灰度测试:
| Hint | 取值 | 说明 |
|---|---|---|
OPT_PARAM('enable_hybrid_search_parallel_execution', 'TRUE'/'FALSE') |
'TRUE' / 'FALSE' |
单条查询级开关,优先级高于租户级配置 enable_hybrid_search_parallel_execution。 |
注意
Hint 中参数不需要带下划线前缀,直接使用 enable_hybrid_search_parallel_execution 即可。
强制串行与强制并行对比示例(dsl_string 字符串相同,仅 Hint 不同):
-- 强制并行(即使系统配置关闭,Hint 也会覆盖)
SELECT /*+ OPT_PARAM('enable_hybrid_search_parallel_execution', 'TRUE') */
tid, __score
FROM HYBRID_SEARCH(table passages_test, @q);
-- 强制串行(即使系统配置开启,Hint 也会覆盖)
SELECT /*+ OPT_PARAM('enable_hybrid_search_parallel_execution', 'FALSE') */
tid, __score
FROM HYBRID_SEARCH(table passages_test, @q);
Hint 详细信息请参见文末相关文档。
并行开关与生效条件
混合搜索并行的开关有两种,优先级如下(由高到低):
- 查询级 Hint:
OPT_PARAMHint 优先级最高,会覆盖系统配置项设置。 - 租户配置项:
_enable_hybrid_search_parallel_execution(未写 Hint 时生效)。
当并行开关最终取值为 TRUE 时,是否真正启用并行还要由执行计划最终决定,满足如下条件则启用多路或路径内并行:
- 自动多路并行:召回路径数 ≥ 2 时,自动启用多路并行。
query路径内并行:query_dop > 1时,启用路径内并行(由 DSL 字段独立控制)。
注意
Hint 可覆盖系统配置。因此,仅将租户配置项 _enable_hybrid_search_parallel_execution 设置为 FALSE 并不能保证所有并行查询被关闭。如果有 SQL 明确使用 Hint 强制开启并行,仍需移除这些 Hint 后才能完全回退为串行。
其他并行问题解答请参见文末相关文档中的 AI 常见问题。
查看执行计划
可使用 EXPLAIN 查看混合搜索计划中各子查询节点、混合融合方式和是否使用索引:
EXPLAIN SELECT c1 FROM HYBRID_SEARCH(table doc_table, '{"query": { "bool": {
"must" : [{"match" : {"content": "oceanbase mysql"}}],
"filter": [
{"range": {"c1": {"gte" : 3}}},
{"range": {"c1": {"lte" : 10}}}
]
}},
"knn":
{
"field": "vector",
"k": 10,
"query_vector": "[1, 2, 3]"
}}');
预期返回结果如下,结果中展示了混合搜索计划中各子查询节点、子查询混合方式为 WRRF 融合,且使用了索引 idx_vector 和全文索引 idx_content:
+------------------------------------------------------------------------------+
| Query Plan |
+------------------------------------------------------------------------------+
| ================================================================== |
| |ID|OPERATOR |NAME |EST.ROWS|EST.TIME(us)| |
| ------------------------------------------------------------------ |
| |0 |INDEX MERGE SCAN|doc_table(idx_vector)|1 |3 | |
| ================================================================== |
| Outputs & filters: |
| ------------------------------------- |
| 0 - output([doc_table.c1]), filter(nil), rowset=16 |
| access([doc_table.__pk_increment], [doc_table.c1]), partitions(p0) |
| is_index_back=true, is_global_index=false, use_index_merge=true, |
| fusion node: method=WEIGHT_SUM, limit(10), window_size(10) |
| vector node: index name=idx_vector |
| boolean node: |
| must: |
| match node: index name=idx_content |
| filter: |
| scalar node: filter([doc_table.c1 >= 3], [doc_table.c1 <= 10]) |
+------------------------------------------------------------------------------+
17 rows in set
相关文档
- 语法、参数和限制说明请参见 HYBRID_SEARCH
- 搜索索引(Search Index)
- 全文索引
- 向量索引
- AI 模型注册
- 全文/标量查询并行执行 Hint 说明请参见 OPT_PARAM Hint 中
_enable_hybrid_search_parallel_execution参数说明。 - AI 常见问题