arXiv 2406.125932025-06-28cs.IRcs.AIcs.CLcs.LG

PromptDSI：基于提示的无排练持续学习文档检索方法

PromptDSI: Prompt-based Rehearsal-free Continual Learning for Document Retrieval

Department of Data Science \& AI, Monash University, Australia Department of Human-Centred Computing, Monash University, Australia

Tuan-Luc Huynh, Thuy-Trang Vu, Weiqing Wang, Yinwei Wei, Trung Le, Dragan Gasevic, Yuan-Fang Li, Thanh-Toan Do

备注：: ECML PKDD 2025 Research track. Camera-ready version. Code is available at https://github.com/LouisDo2108/PromptDSI

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

PromptDSI 提出基于提示的无排练持续学习文档检索方法，通过可学习提示和主题感知提示池高效索引新文档，移除前向传递降低延迟，显著提升新语料库检索性能并有效缓解遗忘。

摘要：

可微搜索索引（DSI）利用预训练语言模型通过端到端学习执行索引和文档检索，而无需依赖外部索引。然而，DSI 需要全量重新训练来索引新文档，导致显著的计算低效。持续学习（CL）通过使模型无需全量重新训练即可增量更新，提供了一种解决方案。文档检索中现有的 CL 解决方案依赖内存缓冲区或生成模型进行排练，这在因隐私问题而限制访问先前训练数据时是不可行的。为此，我们引入了 PromptDSI，一种基于提示的无排练持续学习文档检索方法。PromptDSI 遵循基于提示的持续学习（PCL）框架，使用可学习的提示高效索引新文档，而无需访问先前的文档或查询。为改善检索延迟，我们移除了 PCL 的初始前向传递，否则它会大幅增加训练和推理时间，而性能仅有可忽略的折衷。此外，我们引入了一种新颖的主题感知提示池，采用神经主题嵌入作为固定键，消除了提示键优化的不稳定性，同时保持与现有 PCL 提示池相当的性能。在具有挑战性的无排练持续学习设置中，我们证明 PromptDSI 变体优于基于排练的基线，在缓解遗忘方面匹配强大的基于缓存的基线，并显著提升新语料库的检索性能。

英文摘要：

Differentiable Search Index (DSI) utilizes pre-trained language models to perform indexing and document retrieval via end-to-end learning without relying on external indexes. However, DSI requires full re-training to index new documents, causing significant computational inefficiencies. Continual learning (CL) offers a solution by enabling the model to incrementally update without full re-training. Existing CL solutions in document retrieval rely on memory buffers or generative models for rehearsal, which is infeasible when accessing previous training data is restricted due to privacy concerns. To this end, we introduce PromptDSI, a prompt-based, rehearsal-free continual learning approach for document retrieval. PromptDSI follows the Prompt-based Continual Learning (PCL) framework, using learnable prompts to efficiently index new documents without accessing previous documents or queries. To improve retrieval latency, we remove the initial forward pass of PCL, which otherwise greatly increases training and inference time, with a negligible trade-off in performance. Additionally, we introduce a novel topic-aware prompt pool that employs neural topic embeddings as fixed keys, eliminating the instability of prompt key optimization while maintaining competitive performance with existing PCL prompt pools. In a challenging rehearsal-free continual learning setup, we demonstrate that PromptDSI variants outperform rehearsal-based baselines, match the strong cache-based baseline in mitigating forgetting, and significantly improving retrieval performance on new corpora.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。