arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.14802cs.CLcs.AI
收藏

从RAG到记忆:大型语言模型的非参数持续学习

From RAG to Memory: Non-Parametric Continual Learning for Large Language Models

The Ohio State University, Columbus, OH, USA(俄亥俄州立大学);University of Illinois Urbana-Champaign, IL, USA(伊利诺伊大学厄巴纳-香槟分校)

Bernal Jiménez Gutiérrez, Yiheng Shu, Weijian Qi, Sizhe Zhou, Yu Su

AI总结:

针对RAG在持续学习中无法模拟人类长期记忆的问题,提出HippoRAG 2框架,通过增强个性化PageRank算法和LLM在线使用,在事实性、意义构建和关联性记忆任务上全面超越标准RAG,推动非参数持续学习。

摘要:

我们持续获取、组织和利用知识的能力是人类智能的一个关键特征,人工智能系统必须近似这一能力才能发挥其全部潜力。鉴于大型语言模型(LLMs)在持续学习方面的挑战,检索增强生成(RAG)已成为引入新信息的主要方式。然而,其对向量检索的依赖阻碍了其模拟人类长期记忆的动态性和互联性的能力。最近的RAG方法通过引入知识图谱等各种结构来增强向量嵌入,以解决其中的一些不足,即意义构建和关联性。然而,它们在更基础的事实性记忆任务上的性能却大幅低于标准RAG。我们针对这一非预期的性能下降,提出了HippoRAG 2,这是一个在事实性、意义构建和关联性记忆任务上全面优于标准RAG的框架。HippoRAG 2基于HippoRAG中使用的个性化PageRank算法,并通过更深入的段落整合和更有效的LLM在线使用对其进行了增强。这种结合使该RAG系统更接近人类长期记忆的有效性,在关联性记忆任务上比最先进的嵌入模型提升了7%,同时还展现出更优的事实性知识和意义构建记忆能力。这项工作为LLMs的非参数持续学习铺平了道路。代码和数据可在https://github.com/OSU-NLP-Group/HippoRAG获取。

英文摘要:

Our ability to continuously acquire, organize, and leverage knowledge is a key feature of human intelligence that AI systems must approximate to unlock their full potential. Given the challenges in continual learning with large language models (LLMs), retrieval-augmented generation (RAG) has become the dominant way to introduce new information. However, its reliance on vector retrieval hinders its ability to mimic the dynamic and interconnected nature of human long-term memory. Recent RAG approaches augment vector embeddings with various structures like knowledge graphs to address some of these gaps, namely sense-making and associativity. However, their performance on more basic factual memory tasks drops considerably below standard RAG. We address this unintended deterioration and propose HippoRAG 2, a framework that outperforms standard RAG comprehensively on factual, sense-making, and associative memory tasks. HippoRAG 2 builds upon the Personalized PageRank algorithm used in HippoRAG and enhances it with deeper passage integration and more effective online use of an LLM. This combination pushes this RAG system closer to the effectiveness of human long-term memory, achieving a 7% improvement in associative memory tasks over the state-of-the-art embedding model while also exhibiting superior factual knowledge and sense-making memory capabilities. This work paves the way for non-parametric continual learning for LLMs. Code and data are available at https://github.com/OSU-NLP-Group/HippoRAG.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑