MetaMem: 通过自反思符号优化实现元记忆的进化以促进知识利用
MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization
AI总结:
本文提出MetaMem框架,通过自反思符号优化增强记忆系统,帮助LLM更有效地利用记忆知识。实验表明,MetaMem在多个任务中显著优于基线模型,提升超过3.6%。
摘要:
现有记忆系统使大型语言模型(LLMs)能够通过持久化历史交互支持长周期的人机交互。然而,尽管近期方法在构建有效记忆方面取得成功,但往往破坏交互会话中的内在逻辑和时间关系,导致记忆单元碎片化和推理性能下降。本文提出MetaMem,一种新型框架,通过自进化元记忆增强记忆系统,旨在教LLM如何有效利用记忆知识。在元记忆优化过程中,MetaMem通过自反思推理过程并执行操作来更新当前元记忆状态,迭代提炼跨任务的可转移知识利用经验。积累的元记忆单元作为显式知识利用经验,引导LLM系统性地识别和整合来自散落记忆碎片中的关键证据。大量实验表明,MetaMem的有效性显著优于强基线模型,提升超过3.6%。所有代码和数据集均可在https://github.com/OpenBMB/MetaMem获取。
英文摘要:
Existing memory systems enable Large Language Models (LLMs) to support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows. However, while recent approaches have succeeded in constructing effective memories, they often disrupt the inherent logical and temporal relationships within interaction sessions, resulting in fragmented memory units and degraded reasoning performance. In this paper, we propose MetaMem, a novel framework that augments memory systems with a self-evolving meta-memory, aiming to teach LLMs how to effectively utilize memorized knowledge. During meta-memory optimization, MetaMem iteratively distills transferable knowledge utilization experiences across different tasks by self-reflecting on reasoning processes and performing actions to update the current meta-memory state. The accumulated meta-memory units serve as explicit knowledge utilization experiences, guiding the LLM to systematically identify and integrate critical evidence from scattered memory fragments. Extensive experiments demonstrate the effectiveness of MetaMem, which significantly outperforms strong baselines by over 3.6%. All codes and datasets are available at https://github.com/OpenBMB/MetaMem.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。