大语言模型的记忆
Memory for Large Language Models
浏览论文内容
中文总结 AI 辅助
综述对大语言模型中记忆这一基础架构维度进行系统分类，沿表示、更新动态、持久性三个正交轴表征记忆，形式化相关机制，阐明不同内存概念界限，分析混合架构等，为以记忆为中心的LLM设计及未来创新提供基础。
中文摘要 AI 辅助
记忆已成为大语言模型（LLMs）的一个基础架构维度，从计算的隐式副产品转变为一系列显式、可控机制。虽然近期进展引入了多种策略，包括瞬态注意力、循环状态动态、参数高效适配和可扩展查找存储等，但快速发展导致研究领域高度碎片化。在本综述中，我们提出了一个以架构为中心的LLMs记忆系统分类法。我们的框架沿三个正交轴对记忆进行表征：表示（隐式与显式）、更新动态（离线与在线）和持久性（短期与长期）。我们进一步形式化了决定记忆写入、路由、状态转换和整合的细粒度机制。这种统一视角阐明了计算耦合内存和独立可寻址内存之间的概念界限，有效弥合了不同架构范式。此外，我们批判性地分析了混合内存架构、系统级效率权衡和多维度评估方法。通过将这些分散进展整合为一个连贯框架，本综述描绘了以记忆为中心的LLM设计轨迹，并为可扩展和自适应语言建模的未来创新提供了原则基础。
英文摘要
Memory has evolved into a foundational architectural dimension in large language models (LLMs), shifting from an implicit byproduct of computation to a spectrum of explicit, controllable mechanisms. While recent advances introduce diverse strategies---spanning transient attention, recurrent state dynamics, parameter-efficient adaptations, and scalable lookup storage---this rapid evolution has led to a highly fragmented research landscape. In this survey, we present a systematic, architecture-centric taxonomy of memory in LLMs. Our framework characterizes memory along three orthogonal axes: representation (implicit versus explicit), update dynamics (offline versus online), and persistence (short-term versus long-term). We further formalize the granular mechanisms dictating memory writing, routing, state transitions, and consolidation. This unified perspective elucidates the conceptual boundaries between computation-coupled and independently addressable memory, effectively bridging disparate architectural paradigms. Additionally, we critically analyze hybrid memory architectures, system-level efficiency trade-offs, and multi-dimensional evaluation methodologies. By consolidating these scattered advancements into a cohesive framework, this survey charts the trajectory of memory-centric LLM design and provides a principled foundation for future innovations in scalable and adaptive language modeling.
发表机构
- Tsinghua University（清华大学）
- National University of Singapore（新加坡国立大学）
- Bosch AI（博世人工智能）
机构由 AI 辅助整理，请以论文原文为准。
