发表机构
Kyoto University; LIGHTSPEED; Independent Researcher（京都大学; LIGHTSPEED; 独立研究者）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对大语言模型潜在内存更新时因编码问题致性能降，提出MemDefrag框架，利用中间层追踪信号整理内存碎片、超容时用信息引导遗忘机制，实验显示其在知识保留等方面优于其他模型且泛化性好。
AI 中文摘要
潜在内存作为大语言模型中长期记忆的一种有前景的范式出现，它将过去的知识片段存储为每层的隐藏状态。然而，由于位置编码未对齐以及缺乏区分目标内存片段和无关片段的追踪机制，该范式在内存更新期间性能显著下降。为发现追踪机制，我们探测存储内存片段上的逐层注意力密度，发现一小部分中间变压器层始终将最高密度集中在目标片段上，从而揭示了一个内在的追踪信号。基于此，我们提出MemDefrag，一个无需训练且与模型无关的框架，它利用中间层追踪信号进行内存碎片整理，并在容量超限时应用信息引导的比例遗忘机制。实验表明，MemDefrag在知识保留和长上下文基准测试中大幅优于MemoryLLM和M+，并能很好地推广到各种大语言模型和潜在内存变体。
英文摘要
Latent memory, which stores past knowledge fragments as per-layer hidden states, has emerged as a promising paradigm (e.g., MemoryLLM and M+) for long-term memory in large language models (LLMs). However, the paradigm suffers from significant performance degradation during memory updates, due to positional encoding misalignment and the absence of any tracing mechanism to distinguish target memory fragments from irrelevant ones. To discover such a tracing mechanism, we probe the layer-wise attention density over stored memory fragments, and find that a small set of middle transformer layers consistently concentrates the highest density on the target fragment - exposing an inherent tracing signal. In light of this, we propose MemDefrag, a training-free and model-agnostic framework that (1) uses a middle-layer tracing signal to conduct memory defragmentation (rank, reorder, and filter memories), and (2) applies an informativeness-guided proportional forgetting mechanism once capacity is exceeded. Experiments show that MemDefrag substantially outperforms MemoryLLM and M+ on knowledge retention (e.g., 43.0% vs. 17.4%/17.6% after 50 memory updates) and long-context benchmarks, and generalizes well across various LLMs and latent-memory variants. The code is available at github.com/ryehr/MemDefrag.
CommentsEMNLP 2026
