一个参数化的内存头用于连续生成检索
A Parametric Memory Head for Continual Generative Retrieval
AI总结：
本文提出PAMT方法，通过参数化内存头解决连续生成检索中稳定性和可塑性之间的权衡问题，提升早期文档检索性能且对新增文档影响小。
摘要：
生成信息检索（GenIR）将检索整合为一个神经模型，直接从查询解码文档标识符（docids）。尽管模型-索引架构简单，但难以适应动态文档集合。与模块化系统不同，GenIR的知识参数化编码在权重中；因此，标准适应方法如全量和参数高效微调会导致灾难性遗忘。我们证明顺序适应在新添加文档上提高检索，但显著降低早期切片性能，暴露显著的稳定性-可塑性权衡。为此，我们提出后适应内存调谐（PAMT），一个仅依赖内存的稳定阶段，通过附加模块化参数化内存头（PMH）增强适应模型。PAMT冻结主干并附加固定寻址的产品密钥内存。在前缀-trie约束解码中，解码器隐藏状态稀疏查询PMH以产生隐藏空间的残差修正；这些修正通过冻结的输出嵌入矩阵映射到分数调整，仅在trie有效令牌上计算。这引导docid生成，同时保持路由和主干参数固定。为限制跨切片干扰，PAMT仅更新使用解码时访问统计选择的固定预算内存值，优先选择当前切片频繁激活但先前会话很少使用的条目。在MS MARCO和自然问题上的实验表明，PAMT显著提高早期切片的保留，对新增文档检索性能影响小，同时仅修改每个会话稀疏的内存值子集。
英文摘要：
Generative information retrieval (GenIR) consolidates retrieval into a single neural model that decodes document identifiers (docids) directly from queries. While this model-as-index paradigm offers architectural simplicity, it is poorly suited to dynamic document collections. Unlike modular systems, where indexes are easily updated, GenIR's knowledge is parametrically encoded in its weights; consequently, standard adaptation methods such as full and parameter-efficient fine-tuning can induce catastrophic forgetting. We show that sequential adaptation improves retrieval on newly added documents but substantially degrades performance on earlier slices, exposing a pronounced stability-plasticity trade-off. To address this, we propose post-adaptation memory tuning (PAMT), a memory-only stabilization stage that augments an adapted model with a modular parametric memory head (PMH). PAMT freezes the backbone and attaches a product-key memory with fixed addressing. During prefix-trie constrained decoding, decoder hidden states sparsely query PMH to produce residual corrections in hidden space; these corrections are mapped to score adjustments via the frozen output embedding matrix, computed only over trie-valid tokens. This guides docid generation while keeping routing and backbone parameters fixed. To limit cross-slice interference, PAMT updates only a fixed budget of memory values selected using decoding-time access statistics, prioritizing entries frequently activated by the current slice and rarely used in prior sessions. Experiments on MS MARCO and Natural Questions under sequential, disjoint corpus increments show that PAMT substantially improves retention on earlier slices with minimal impact on retrieval performance for newly added documents, while modifying only a sparse subset of memory values per session.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
