arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.08489cs.DBcs.LG
收藏

高效实现语义连接运算符

Implementing Semantic Join Operators Efficiently

Cornell University(康奈尔大学)

Immanuel Trummer

AI总结:

本文提出一种受块嵌套循环连接启发的语义连接算法,通过批量整合行对并优化批次大小,显著降低LLM调用成本,优于现有实现。

摘要:

语义查询处理引擎通常支持语义连接,使用户能够匹配满足自然语言指定条件的行。此类连接条件可以使用大型语言模型(LLM)来评估,这些模型无需针对特定任务的训练即可解决新任务。目前,许多语义查询处理引擎通过嵌套循环实现语义连接,即调用LLM来评估行对上的连接条件。相反,本文提出了一种新颖的算法,其灵感来自传统数据库系统中的块嵌套循环连接运算符实现。所提出的算法将两个输入表中的行批次整合到单个提示中。LLM调用的目标是识别当前输入中的所有匹配行对。本文介绍了可用于优化行批次大小的公式,同时考虑了LLM上下文窗口大小的约束(限制输入和输出大小)。所提出算法的一种自适应变体适用于输出大小难以估计的情况。对渐近处理成本的正式分析以及实证结果表明,与近期语义查询处理引擎使用的连接实现相比,所提出的方法显著降低了成本,并且表现良好。

英文摘要:

Semantic query processing engines often support semantic joins, enabling users to match rows that satisfy conditions specified in natural language. Such join conditions can be evaluated using large language models (LLMs) that solve novel tasks without task-specific training. Currently, many semantic query processing engines implement semantic joins via nested loops, invoking the LLM to evaluate the join condition on row pairs. Instead, this paper proposes a novel algorithm, inspired by the block nested loops join operator implementation in traditional database systems. The proposed algorithm integrates batches of rows from both input tables into a single prompt. The goal of the LLM invocation is to identify all matching row pairs in the current input. The paper introduces formulas that can be used to optimize the size of the row batches, taking into account constraints on the size of the LLM context window (limiting both input and output size). An adaptive variant of the proposed algorithm refers to cases in which the size of the output is difficult to estimate. A formal analysis of asymptotic processing costs, as well as empirical results, demonstrates that the proposed approach reduces costs significantly and performs well compared to join implementations used by recent semantic query processing engines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑