arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.02708cs.IRcs.AI
收藏

CREAM: 动态数据流中基于自适应软记忆的持续检索

CREAM: Continual Retrieval on Dynamic Streaming Corpora with Adaptive Soft Memory

Korea University(韩国大学);Yonsei University(延世大学)

HuiJeong Son, Hyeongu Kang, Sunho Kim, Subeen Ho, SeongKu Kang, Dongha Lee, Susik Yoon

AI总结:

CREAM通过自监督框架实现动态数据流中的持续检索,利用自适应软记忆提升对未知主题的适应性和检索准确性。

摘要:

信息检索(IR)在动态数据流中的应用是一项关键任务,因为数据分布的变化会降低人工智能驱动的IR系统的表现。为缓解这一问题,基于内存的持续学习已被广泛应用于IR。然而,现有方法依赖于一组固定查询和真实文档,这限制了对未见数据的泛化能力,使其在实际应用中不切实际。为了在没有真实标签的情况下更有效地学习新语料库中的未知主题,我们提出了CREAM,一种基于内存的自监督框架。CREAM将流式查询和文档的演变语义捕捉到动态结构的软记忆中,并利用它在无监督设置下适应已见和未见的主题。我们通过三种关键技术实现这一目标:细粒度相似性估计、正则化聚类原型化和分层核心集采样。在两个基准数据集上的实验表明,CREAM表现出优越的适应性和检索准确性,在无标签设置中,其在Success@5上比最强方法高出27.79%,在Recall@10上平均高出44.5%,并且在性能上可与或超过监督方法。

英文摘要:

Information retrieval (IR) in dynamic data streams is a crucial task, as shifts in data distribution degrade the performance of AI-powered IR systems. To mitigate this issue, memory-based continual learning has been widely adopted for IR. However, existing methods rely on a fixed set of queries with ground-truth documents, which limits generalization to unseen data, making them impractical for real-world applications. To enable more effective learning with unseen topics of a new corpus without ground-truth labels, we propose CREAM, a self-supervised framework for memory-based continual retrieval. CREAM captures the evolving semantics of streaming queries and documents into dynamically structured soft memory and leverages it to adapt to both seen and unseen topics in an unsupervised setting. We realize this through three key techniques: fine-grained similarity estimation, regularized cluster prototyping, and stratified coreset sampling. Experiments on two benchmark datasets demonstrate that CREAM exhibits superior adaptability and retrieval accuracy, outperforming the strongest method in a label-free setting by 27.79% in Success@5 and 44.5% in Recall@10 on average, and achieving performance comparable to or even exceeding that of supervised methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑