arXiv 2502.170572025-05-29cs.IRcs.AI

ExpandR：在 LLM 指导下超越原始查询的稠密检索器训练

ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance

Department of Computer Science and Technology, Northeastern University, China（东北大学计算机科学与技术系）；Department of Computer Science and Technology, Institute for AI, Tsinghua University, China（清华大学人工智能研究院计算机科学与技术系）；Beijing National Research Center for Information Science and Technology, China（北京信息科学与技术国家研究中心）

Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu, Ge Yu

备注：: 16 pages, 10 tables, 5 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 ExpandR 框架，通过联合优化 LLM 和稠密检索器，利用 LLM 生成查询扩展并采用 DPO 训练 LLM，实现检索性能提升超 5%。

摘要：

大型语言模型（LLM）通过查询增强提升稠密检索方面展现出巨大潜力。然而，现有方法多将 LLM 和检索器作为独立模块，忽略了生成与排序目标间的对齐。在本研究中，我们提出了 ExpandR，一个统一的 LLM 增强稠密检索框架，联合优化 LLM 和检索器。ExpandR 利用 LLM 生成语义丰富的查询扩展，用于增强检索器训练。同时，使用直接偏好优化（DPO）训练 LLM，由精心设计的奖励函数引导，平衡检索有效性与生成一致性。此联合优化范式实现了 LLM 与检索器的相互适应，生成既有信息量又适合检索的查询扩展。在多个基准上的实验表明，ExpandR 持续优于强基线，检索性能提升超过 5%。代码见 https://github.com/NEUIR/ExpandR。

英文摘要：

Large language models (LLMs) have demonstrated significant potential in enhancing dense retrieval through query augmentation. However, most existing methods treat the LLM and the retriever as separate modules, overlooking the alignment between generation and ranking objectives. In this work, we propose ExpandR, a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever. ExpandR employs the LLM to generate semantically rich query expansions, which are leveraged to enhance the retriever's training. Simultaneously, the LLM is trained using Direct Preference Optimization (DPO), guided by a carefully designed reward function that balances retrieval effectiveness and generation consistency. This joint optimization paradigm enables mutual adaptation between the LLM and the retriever, resulting in query expansions that are both informative and well-suited for retrieval. Experimental results on multiple benchmarks show that ExpandR consistently outperforms strong baselines, achieving more than a 5% improvement in retrieval performance. All codes are available at https://github.com/NEUIR/ExpandR.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。