arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18312cs.CLcs.AI
收藏

AI原生记忆:从大语言模型通向通用人工智能的路径

AI-native Memory: A Pathway from LLMs Towards AGI

Mindverse AI(星幻人工智能)

Jingbo Shang, Zai Zheng, Jiale Wei, Xiang Ying, Felix Tao, Mindverse Team

AI总结:

本文指出现有LLMs长上下文能力有限,提出通过整合AI原生记忆构建以LLM为核心的AGI系统,存储推理结论以简化复杂查询,并探讨其潜力与安全挑战。

摘要:

大语言模型(LLMs)已经向世界展示了通用人工智能(AGI)的火花。一种观点(尤其是来自一些致力于LLMs的初创公司)认为,具有几乎无限上下文长度的LLM能够实现AGI。然而,他们对(现有)LLMs的长上下文能力可能过于乐观——(1)近期文献表明,它们的有效上下文长度明显小于其声称的上下文长度;并且(2)我们的“草垛推理”实验进一步证明,同时从长上下文中寻找相关信息并进行(简单)推理几乎是不可能的。在本文中,我们设想了一条通过整合记忆从LLMs通向AGI的路径。我们认为AGI应该是一个以LLMs作为核心处理器的系统。除了原始数据外,该系统中的记忆将存储大量从推理过程中得出的重要结论。与仅处理原始数据的检索增强生成(RAG)相比,这种方法不仅将语义相关的信息连接得更紧密,而且简化了查询时的复杂推理。作为中间阶段,记忆很可能以自然语言描述的形式存在,用户也能直接使用它们。最终,每个智能体/个人都应拥有自己的大型个人模型,这是一个深度神经网络模型(即AI-native),能够参数化并压缩所有类型的记忆,甚至是那些无法用自然语言描述的记忆。最后,我们讨论了AI原生记忆作为AGI时代(主动)参与、个性化、分布式和社交的变革性基础设施的巨大潜力,以及由此引发的隐私和安全挑战和初步解决方案。

英文摘要:

Large language models (LLMs) have demonstrated the world with the sparks of artificial general intelligence (AGI). One opinion, especially from some startups working on LLMs, argues that an LLM with nearly unlimited context length can realize AGI. However, they might be too optimistic about the long-context capability of (existing) LLMs -- (1) Recent literature has shown that their effective context length is significantly smaller than their claimed context length; and (2) Our reasoning-in-a-haystack experiments further demonstrate that simultaneously finding the relevant information from a long context and conducting (simple) reasoning is nearly impossible. In this paper, we envision a pathway from LLMs to AGI through the integration of \emph{memory}. We believe that AGI should be a system where LLMs serve as core processors. In addition to raw data, the memory in this system would store a large number of important conclusions derived from reasoning processes. Compared with retrieval-augmented generation (RAG) that merely processing raw data, this approach not only connects semantically related information closer, but also simplifies complex inferences at the time of querying. As an intermediate stage, the memory will likely be in the form of natural language descriptions, which can be directly consumed by users too. Ultimately, every agent/person should have its own large personal model, a deep neural network model (thus \emph{AI-native}) that parameterizes and compresses all types of memory, even the ones cannot be described by natural languages. Finally, we discuss the significant potential of AI-native memory as the transformative infrastructure for (proactive) engagement, personalization, distribution, and social in the AGI era, as well as the incurred privacy and security challenges with preliminary solutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑