arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

反对用于检索的生成式模型:判别式语言模型作为有效的检索器

The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers

Zhe Xu, Prachi Agrawal, Kavosh Asadi, Tianyi Chen, Carl Hu, Justin Johnson, Wuwei Lan, Mingfu Liang, Xi Liu, Tik On Lui, Oladipo Ositelu, Sandeep Pandey, Ankit Peshin, Feng Qi, Anil Ramakrishna, Kaushik Rangadurai, Frank Shyu, Luke Simon, Yang Yang, Chiyu Zhang

arXiv 2607.25346首次发表:更新:

AI 中文总结

研究探讨大语言模型用于检索的问题,通过将其作为语义表示主干重振双塔检索架构,引入创新的双塔框架,经实验评估,该架构在公共基准和生产系统中表现出色,证明传统双塔范式结合现代学习后在工业检索中仍具竞争力。

AI 中文摘要

大语言模型已成为推荐系统的强大工具。然而,在网络规模上将它们部署为生成式推荐器或零样本排序器,仍受计算开销和基础挑战的限制。本文通过将大语言模型用作语义表示主干而非生成引擎,重振经典的高效双塔检索架构。我们引入了为高通量、大规模检索设计的大语言模型原生双塔框架。该架构有多项关键创新,如共享大语言模型编码器、句尾令牌池化、跨数据集迁移学习等。在三个公共基准上的广泛评估表明,跨编码器架构优于当前最先进模型,高效双塔模型也达到了可比性能。此外,在内部大规模生产系统上的实验取得了显著改进。研究结果表明,传统双塔范式在结合现代表示学习后,对工业检索系统仍是极具竞争力和实用性的解决方案。

英文摘要

Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-tower retrieval architecture by adapting LLMs as semantic representation backbones rather than generative engines. We introduce an LLM-native two-tower framework engineered for high-throughput, large-scale retrieval. Our architecture introduces several key innovations: a shared LLM encoder for joint user-item modeling, End-Of-Sentence (EOS) token pooling for compact sequence embedding, cross-dataset transfer learning, knowledge distillation from powerful cross-encoder teachers, and latent reasoning within the user tower. Extensive evaluation across three public benchmarks demonstrates that cross-encoder architecture outperforms current state-of-the-art (SoTA) models, while the efficient two-tower student achieves SoTA-comparable retrieval performance. Furthermore, experiments on internal large-scale production systems yield substantial topline retrieval improvements along with high resilience to model staleness and superior data scaling. Our findings demonstrate that when augmented with modern representation learning, the traditional two-tower paradigm remains an exceptionally competitive and practical solution for industrial retrieval systems.

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑