arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.02537cs.DB
收藏

大语言模型增强的关系运算符:分类、基准测试与分析

Large Language Model-Enhanced Relational Operators: Taxonomy, Benchmark, and Analysis

Alibaba Group(阿里巴巴集团);Tsinghua University(清华大学);Renmin University of China(中国人民大学)

Yunxiang Su, Tianjing Zeng, Zhongjun Ding, Yin Lin, Rong Zhu, Zhewei Wei, Bolin Ding, Jingren Zhou

AI总结:

本文提出统一的关系运算符分类,设计全面的基准测试,并评估不同实现下的运算符性能,以指导复杂语义查询系统的设计。

摘要:

随着大语言模型(LLMs)的发展,许多研究通过运算符-like组件将LLMs整合到关系数据处理任务中,例如具有语义谓词的过滤器、知识增强的表格补全、推理驱动的实体匹配以及更具挑战性的语义查询处理。这些组件在保持关系输入/输出接口的同时调用LLMs,我们称之为LLM增强的关系运算符(LROs)。从运算符的角度来看,现有的LROs存在定义碎片化、各种实现策略和评估基准不足的问题。为此,本文首先建立了统一的LRO分类,将现有LROs归类为选择、匹配、补全、聚类和排序,并涵盖其操作数和实现变体。其次,我们设计了LROBench,一个涵盖290个单LRO查询和60个多LRO查询的全面基准测试,覆盖27个数据库,超过10个领域。LROBench在单LRO工作负载中涵盖了所有操作逻辑和操作数粒度,并提供了按查询复杂度分层的具有挑战性的多LRO查询。基于这些,我们评估了不同实现下的单个LROs,得出了关于LRO设计选择的实用见解,并总结了我们的经验最佳实践。我们进一步比较了现有多LRO系统与使用这些最佳实践的LRO套件的端到端性能,以研究如何设计有效的LRO集以支持复杂语义查询的多LRO系统。最后,为了促进未来工作,我们概述了有前景的未来方向,并开源了所有基准测试数据和评估代码,可在https://github.com/LROBench/LROBench/上获取。

英文摘要:

With the development of large language models (LLMs), numerous studies integrate LLMs through operator-like components to enhance relational data processing tasks, e.g., filters with semantic predicates, knowledge-augmented table imputation, reasoning-driven entity matching and more challenging semantic query processing. These components invoke LLMs while preserving a relational input/output interface, which we refer to as LLM-Enhanced Relational Operators (LROs). From an operator perspective, unfortunately, these existing LROs suffer from fragmented definition, various implementation strategies and inadequate evaluation benchmarks. To this end, in this paper, we first establish a unified LRO taxonomy to align existing LROs, and categorize them into: Select, Match, Impute, Cluster and Order, along with their operands and implementation variants. Second, we design LROBench, a comprehensive benchmark featuring 290 single-LRO queries and 60 multi-LRO queries, spanning 27 databases across more than 10 domains. LROBench covers all operating logics and operand granularities in its single-LRO workload, and provides challenging multi-LRO queries stratified by query complexity. Based on these, we evaluate individual LROs under various implementations, deriving practical insights into LRO design choices and summarizing our empirical best practices. We further compare the end-to-end performance of existing multi-LRO systems against an LRO suite instantiated with these best practices, in order to investigate how to design an effective LRO set for multi-LRO systems targeting complex semantic queries. Last, to facilitate future work, we outline promising future directions and open-source all benchmark data and evaluation code, available at https://github.com/LROBench/LROBench/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑