arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.02773cs.IR
收藏

生成信息检索中的新文档整合模型编辑

Model Editing for New Document Integration in Generative Information Retrieval

Shandong University(山东大学);University of Amsterdam(阿姆斯特丹大学);Baidu Inc.(百度公司);Leiden University(莱顿大学)

Zhen Zhang, Zihan Wang, Xinyu Ma, Shuaiqiang Wang, Dawei Yin, Xin Xin, Pengjie Ren, Maarten de Rijke, Zhaochun Ren

AI总结:

本文提出DOME方法,通过模型编辑提升生成检索模型对新文档的适应能力,有效解决文档ID映射问题,减少训练时间,提高检索性能。

摘要:

生成检索(GR)将信息检索任务重新定义为生成文档标识符(docIDs)。尽管有前景,现有GR模型在处理新增文档时泛化能力差,常无法生成正确的docIDs。尽管增量训练提供了一种简单的方法,但计算成本高、资源消耗大且易产生灾难性遗忘,限制了GR的可扩展性和实用性。本文将核心瓶颈确定为解码器将隐藏状态映射到新文档正确docIDs的能力。模型编辑,通过针对docID映射进行参数修改,代表了有前景的解决方案。然而,将模型编辑应用于现有GR模型并不简单,这受到查询之间不可区分的编辑向量的严重阻碍,由于检索结果中共享docIDs的高重叠。为此,我们提出DOME(docID导向的模型编辑),一种新的方法,有效且高效地适应GR模型到未见过的文档。DOME包含三个阶段:(1)关键层的识别,(2)编辑向量的优化,(3)构建和应用更新。其核心是采用混合标签自适应训练策略,通过结合软标签(保留查询特定语义以区分更新)和硬标签(强制精确映射修改)来学习判别性编辑向量。在广泛使用的基准测试中,包括NQ和MS MARCO,实验表明,我们的方法在新文档上显著提高了检索性能,同时保持对原始集合的有效性。此外,DOME仅需增量训练约60%的训练时间,大大减少了计算成本,实现了高效、频繁的模型更新。

英文摘要:

Generative retrieval (GR) reformulates the Information Retrieval (IR) task as the generation of document identifiers (docIDs). Despite its promise, existing GR models exhibit poor generalization to newly added documents, often failing to generate the correct docIDs. While incremental training offers a straightforward remedy, it is computationally expensive, resource-intensive, and prone to catastrophic forgetting, thereby limiting the scalability and practicality of GR. In this paper, we identify the core bottleneck as the decoder's ability to map hidden states to the correct docIDs of newly added documents. Model editing, which enables targeted parameter modifications for docID mapping, represents a promising solution. However, applying model editing to current GR models is not trivial, which is severely hindered by indistinguishable edit vectors across queries, due to the high overlap of shared docIDs in retrieval results. To address this, we propose DOME (docID-oriented model editing), a novel method that effectively and efficiently adapts GR models to unseen documents. DOME comprises three stages: (1) identification of critical layers, (2) optimization of edit vectors, and (3) construction and application of updates. At its core, DOME employs a hybrid-label adaptive training strategy that learns discriminative edit vectors by combining soft labels, which preserve query-specific semantics for distinguishable updates, with hard labels that enforce precise mapping modifications. Experiments on widely used benchmarks, including NQ and MS MARCO, show that our method significantly improves retrieval performance on new documents while maintaining effectiveness on the original collection. Moreover, DOME achieves this with only about 60% of the training time required by incremental training, considerably reducing computational cost and enabling efficient, frequent model updates.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑