arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.05096cs.CL
收藏

RAG还是学习?在现实世界中连续知识漂移下理解LLM适应的极限

RAG or Learning? Understanding the Limits of LLM Adaptation under Continuous Knowledge Drift in the Real World

Tsinghua University(清华大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);School of Artificial Intelligence, Chinese University of Hong Kong (Shenzhen)(香港中文大学(深圳)人工智能学院);Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University(深圳 ubiquitous 数据赋能重点实验室,清华大学深圳国际研究生院,清华大学)

Hanbing Liu, Lang Cao, Yang Li

AI总结:

本文提出一个现实世界动态事件基准,评估模型在连续知识漂移下的适应能力,揭示现有方法如RAG和学习方法的局限性,并提出时间感知检索基线Chronos以提升时序一致性。

摘要:

大型语言模型(LLMs)在预训练过程中获取大部分知识,使其与固定世界快照绑定,难以适应持续变化的知识。随着事实、实体和事件随时间变化,模型可能经历连续知识漂移,导致不仅预测过时,还有时间不一致的推理。现有方法如持续微调、知识编辑和检索增强生成(RAG)旨在更新或补充模型知识,但很少在反映时间演化的现实世界知识演变设置中被评估。本文引入了一个新的现实世界动态事件基准,由时间戳证据构建,捕捉知识随时间演变的方式,从而系统评估模型在连续知识漂移下的适应能力。该基准揭示大多数现有方法,包括vanilla RAG和几种学习方法,在此设置下表现不佳,暴露了关键限制,如灾难性遗忘和时间不一致性。为缓解这些限制,我们提出时间感知检索基线Chronos,通过逐步将检索到的证据组织成事件演变图,使LLMs在不额外训练的情况下实现更一致的时间理解。整体而言,本文为分析和推进LLM在现实设置中连续知识漂移的适应提供了基础。

英文摘要:

Large language models (LLMs) acquire most of their knowledge during pretraining, which ties them to a fixed snapshot of the world and makes adaptation to continuously evolving knowledge challenging. As facts, entities, and events change over time, models may experience continuous knowledge drift, resulting not only in outdated predictions but also in temporally inconsistent reasoning. Although existing approaches, such as continual finetuning, knowledge editing, and retrieval-augmented generation (RAG), aim to update or supplement model knowledge, they are rarely evaluated in settings that reflect chronological, evolving, and real-world knowledge evolution. In this work, we introduce a new benchmark of real-world dynamic events, constructed from time-stamped evidence that captures how knowledge evolves over time, which enables systematic evaluation of model adaptation under continuous knowledge drift. The benchmark reveals that most existing methods, including vanilla RAG and several learning-based approaches, struggle under this setting, exposing critical limitations such as catastrophic forgetting and temporal inconsistency. To mitigate these limitations, we propose a time-aware retrieval baseline, Chronos, which progressively organizes retrieved evidence into an Event Evolution Graph to enable more temporally consistent understanding in LLMs without additional training. Overall, this work provides a foundation for analyzing and advancing LLM adaptation to continuous knowledge drift in realistic settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑