arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.14280cs.IR
收藏

ASI++:迈向分布均衡的端到端生成式检索

ASI++: Towards Distributionally Balanced End-to-End Generative Retrieval

Yuxuan Liu, Tianchi Yang, Zihan Zhang, Minghui Song, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang

AI总结:

针对生成式检索中ID分配不均衡和端到端训练困难的问题,提出ASI++方法,通过分布均衡、表示瓶颈和信息一致性准则,实现均衡ID分配并提升检索性能。

摘要:

生成式检索是信息检索中一种有前景的新范式,它采用序列到序列(seq2seq)模型将文档特征编码为参数,并根据搜索查询解码出相关的文档标识符(ID)。现有的生成式检索解决方案通常依赖预处理阶段来预先定义文档ID,这可能导致这些ID与检索任务之间存在语义鸿沟。然而,由于真实世界数据的长尾分布特性,对ID分配和检索任务进行端到端训练颇具挑战性,导致ID空间利用效率低下且不均衡。为解决这些问题,我们提出了ASI++,一种新颖的全端到端生成式检索方法,旨在同时学习均衡的ID分配并提升检索性能。ASI++基于原始ASI的全端到端训练框架,并引入了若干关键创新。首先,一个分布均衡准则解决了ID分配中的不均衡问题,促进了ID空间更高效的利用。其次,一个表示瓶颈准则增强了稠密表示,以缓解学习ID分配时的瓶颈。最后,一个信息一致性准则将这些过程整合到一个基于信息论的联合优化框架中。我们进一步探索了用于学习ID分配的各种模块结构,包括神经量化、可微乘积量化和残差量化。在公共和工业数据集上的大量实验证明了ASI++在提升检索性能和实现均衡ID分配方面的有效性。

英文摘要:

Generative retrieval, a promising new paradigm in information retrieval, employs a seq2seq model to encode document features into parameters and decode relevant document identifiers (IDs) based on search queries. Existing generative retrieval solutions typically rely on a preprocessing stage to pre-define document IDs, which can suffer from a semantic gap between these IDs and the retrieval task. However, end-to-end training for both ID assignments and retrieval tasks is challenging due to the long-tailed distribution characteristics of real-world data, resulting in inefficient and unbalanced ID space utilization. To address these issues, we propose ASI++, a novel fully end-to-end generative retrieval method that aims to simultaneously learn balanced ID assignments and improve retrieval performance. ASI++ builds on the fully end-to-end training framework of vanilla ASI and introduces several key innovations. First, a distributionally balanced criterion addresses the imbalance in ID assignments, promoting more efficient utilization of the ID space. Next, a representation bottleneck criterion enhances dense representations to alleviate bottlenecks in learning ID assignments. Finally, an information consistency criterion integrates these processes into a joint optimization framework grounded in information theory. We further explore various module structures for learning ID assignments, including neural quantization, differentiable product quantization, and residual quantization. Extensive experiments on both public and industrial datasets demonstrate the effectiveness of ASI++ in improving retrieval performance and achieving balanced ID assignments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑