arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.12974cs.IR
收藏

通过审慎思考学习用于密集检索的精炼文档表示

Learning Refined Document Representations for Dense Retrieval via Deliberate Thinking

Northeastern University(东北大学);Tsinghua University(清华大学);ModelBest Inc.(魔搭智能)

Yifan Ji, Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shi Yu, Yishan Li, Zhiyuan Liu, Yu Gu, Ge Yu, Maosong Sun

AI总结:

针对单嵌入难以表达文档多面语义的问题,提出Debater,通过审慎思考链迭代精炼文档嵌入,并用自蒸馏融合高信息量步骤,在多个检索基准上显著提升准确率与鲁棒性。

摘要:

近期的密集检索器越来越多地利用大型语言模型(LLM)强大的文本理解能力,将查询和文档编码到共享嵌入空间中,以实现有效检索。然而,大多数现有方法只用单个嵌入表示每篇文档,这在捕捉文档多方面语义方面效果较差,从而限制了匹配准确率。本文提出基于审慎思考的密集检索器(Debater),这是一种通过引入逐步思考过程来增强文档表示的新方法。Debater引入了审慎思考链(Chain-of-Deliberation)机制,通过连续的思维链迭代精炼文档嵌入。为整合来自不同思考步骤的信息,Debater进一步采用自蒸馏(Self Distillation)机制,识别并融合信息量最大的步骤,形成统一嵌入。实验结果表明,Debater在多个检索基准上显著优于现有方法,展现出更优的准确率和鲁棒性。所有代码和数据集均可在https://github.com/OpenBMB/DEBATER获取。

英文摘要:

Recent dense retrievers increasingly leverage the robust text understanding capabilities of Large Language Models (LLMs), encoding queries and documents into a shared embedding space for effective retrieval. However, most existing methods represent each document with a single embedding, which is less effective at capturing its multifaceted semantics and thereby limits matching accuracy. In this paper, we propose Deliberate Thinking based Dense Retriever (Debater), a novel approach that enhances document representations by incorporating a step-by-step thinking process. Debater introduces a Chain-of-Deliberation mechanism, which iteratively refines document embeddings through a continuous chain-of-thought. To integrate information from various thinking steps, Debater further employs a Self Distillation mechanism that identifies and fuses the most informative steps into a unified embedding. Experimental results show that Debater significantly outperforms existing methods across several retrieval benchmarks, demonstrating superior accuracy and robustness. All codes and datasets are available at https://github.com/OpenBMB/DEBATER.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑