AI 中文总结
研究针对基于LLM的检索器存在的问题，提出SHIFT框架。通过残差投影等转换LLM为高效检索器，利用细粒度重建缓解不匹配，经实验验证其性能优于其他检索器，为信息检索提供新方法。
AI 中文摘要
基于大语言模型（LLM）的检索器已成为现代信息检索系统的基本组成部分。“重写然后检索”范式在检索前引入显式推理，而诸如GIRCSE和LaSER等隐式推理检索器通过用软令牌替换显式推理来提高效率。尽管这些方法在推理密集型检索基准上表现出有竞争力的性能，但它们难以解决检索和生成目标之间的不匹配问题。在这项工作中，我们提出了SHIFT（自我重建利用隐式细粒度思维进行检索），这是一个基于LLM的检索训练框架。首先，我们通过在潜在空间中的残差投影和面向任务的双向注意力聚合，将LLM转换为推理高效的检索器。其次，我们使用基于细粒度下一个令牌预测的重建来缓解对比学习和隐式推理之间的不匹配。在推理密集型检索基准上的大量实验表明，SHIFT始终优于其他广泛使用的检索器。我们还进行了详细分析来说明我们的方法是如何工作工作的。
英文摘要
LLM-based retrievers have become a fundamental component of modern information retrieval systems. The paradigm of "rewrite-then-retriev" introduces explicit reasoning before retrieval. In addition, implicit-reasoning retrievers such as GIRCSE and LaSER improve efficiency by replacing explicit reasoning with soft tokens. Although these methods demonstrated competitive performance on reasoning-intensive retrieval benchmarks, they struggle to address the mismatch between the objectives of retrieval and generation. In this work, we propose SHIFT ($\textbf{S}$elf-reconstruction $\textbf{H}$arnesses $\textbf{I}$mplicit $\textbf{F}$ine-grained $\textbf{T}$hinking for Retrieval), a retrieval training framework based on LLMs. Firstly, we transfer LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space. Secondly, we alleviate the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction. Extensive experiments on reasoning-intensive retrieval benchmarks show that SHIFT consistently outperforms other widely used retrievers. We also carried out a detailed analysis to illustrate how our method works.
