arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.01560cs.CL
收藏

DeltaMem:通过强化学习实现面向代理的记忆管理

DeltaMem: Towards Agentic Memory Management via Reinforcement Learning

Alibaba Tongyi Lab, Hangzhou, China(阿里巴巴通义实验室,杭州,中国);Zhejiang University, Hangzhou, China(浙江大学,杭州,中国)

Qi Zhang, Shen Huang, Chu Liu, Shouqing Yang, Junbo Zhao, Haobo Wang, Pengjun Xie

AI总结:

本文提出DeltaMem,通过强化学习实现单代理环境下的人格中心记忆管理,结合用户-助手对话数据集和操作级记忆更新标签,引入基于记忆的Levenshtein距离以提升记忆管理能力。

摘要:

近期关于以人物为中心的记忆的研究揭示了多代理系统在管理人物记忆方面的强大能力,尤其是在对话场景中。然而,这些复杂框架常常面临信息丢失和在不同场景下脆弱的问题,导致性能不佳。本文提出DeltaMem,一种面向代理的记忆管理系统,将人物中心的记忆管理作为单代理设置中的端到端任务。为进一步提升我们的人格记忆管理器的性能,我们借鉴人类记忆的进化过程,合成一个用户-助手对话数据集及其相应的操作级记忆更新标签。在此基础上,我们引入一种新的基于记忆的Levenshtein距离来正式化记忆更新奖励,并提出一种定制化的强化学习框架以进一步增强DeltaMem的管理能力。大量实验表明,无论是无需训练的DeltaMem还是经过强化学习训练的DeltaMem,在多样化的长期记忆基准测试中,如LoCoMo、HaluMem和PersonaMem,均优于所有产品级基线。

英文摘要:

Recent advances in persona-centric memory have revealed the powerful capability of multi-agent systems in managing persona memory, especially in conversational scenarios. However, these complex frameworks often suffer from information loss and are fragile across varying scenarios, resulting in suboptimal performance. In this paper, we propose DeltaMem, an agentic memory management system that formulates persona-centric memory management as an end-to-end task within a single-agent setting. To further improve the performance of our agentic memory manager, we draw inspiration from the evolution of human memory and synthesize a user-assistant dialogue dataset along with corresponding operation-level memory updating labels. Building on this, we introduce a novel Memory-based Levenshtein Distance to formalize the memory updating reward, and propose a tailored reinforcement learning framework to further enhance the management capabilities of DeltaMem. Extensive experiments show that both training-free and RL-trained DeltaMem outperform all product-level baselines across diverse long-term memory benchmarks, including LoCoMo, HaluMem, and PersonaMem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑