arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.08150cs.IR
收藏

DiffuGR:基于扩散语言模型的生成性文档检索

DiffuGR: Generative Document Retrieval with Diffusion Language Models

Shandong University(山东大学);Leiden University(莱顿大学);Baidu Inc.(百度公司);Chinese Academy of Sciences(中国科学院);Peking University(北京大学)

Xinpeng Zhao, Zhaochun Ren, Yukun Zhao, Zhenyang Li, Mengqi Zhang, Jun Feng, Ran Chen, Ying Zhou, Zhumin Chen, Shuaiqiang Wang, Dawei Yin, Xin Xin

AI总结:

DiffuGR通过扩散语言模型实现生成性文档检索,解决DocID生成与自然语言生成不匹配及效率与准确性权衡问题,提升检索性能与灵活性。

摘要:

生成性检索(GR)将文档检索重新定义为一个端到端的任务,即生成序列文档标识符(DocIDs)。现有GR方法主要依赖于从左到右的自回归解码,这有两个根本性的局限性:(i)DocID生成与自然语言生成之间的不匹配,其中早期生成的错误DocID标记可能导致完全错误的检索;(ii)无法动态平衡检索效率和准确性的权衡,这对实际应用至关重要。为了解决这些挑战,我们提出了基于扩散语言模型的生成性文档检索,称为DiffuGR。DiffuGR将DocID生成建模为一个离散的扩散过程。在训练过程中,DocIDs通过随机掩码过程被破坏,然后扩散语言模型在检索感知的目标下训练以恢复它们。在推理过程中,DiffuGR并行生成DocID标记,并通过可控数量的去噪步骤进行细化。与自回归解码不同,DiffuGR引入了一种新的机制,首先生成大量自信的DocID标记,然后通过基于扩散的去噪进行细化。此外,DiffuGR还提供了对质量-延迟权衡的显式运行时控制。在广泛使用的检索基准测试中,DiffuGR在强大的自回归生成检索器上表现更优。此外,我们验证了DiffuGR通过可变去噪预算实现了对质量-延迟权衡的灵活控制。

英文摘要:

Generative retrieval (GR) reframes document retrieval as an end-to-end task of generating sequential document identifiers (DocIDs). Existing GR methods predominantly rely on left-to-right auto-regressive decoding, which suffers from two fundamental limitations: (i) a \emph{mismatch between DocID generation and natural language generation}, whereby an incorrect DocID token generated at an early step can lead to entirely erroneous retrieval; and (ii) an \emph{inability to dynamically balance the trade-off between retrieval efficiency and accuracy}, which is crucial for practical applications. To tackle these challenges, we propose generative document retrieval with diffusion language models, termed \emph{DiffuGR}. DiffuGR formulates DocID generation as a discrete diffusion process. During training, DocIDs are corrupted through a stochastic masking process, and a diffusion language model is trained to recover them under a retrieval-aware objective. For inference, DiffuGR generates DocID tokens in parallel and refines them through a controllable number of denoising steps. Unlike auto-regressive decoding, DiffuGR introduce \emph{a novel mechanism to first generate plenty of confident DocID tokens and then refine the generation through diffusion-based denoising}. Moreover, DiffuGR also offers \emph{explicit runtime control over the quality-latency tradeoff}. Extensive experiments on widely-applied retrieval benchmarks show that DiffuGR outperforms strong auto-regressive generative retrievers. Additionally, we verify that DiffuGR achieves flexible control over the quality-latency trade-off via variable denoising budgets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑