arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.03192cs.CL
收藏

MemRL: 通过运行时强化学习在事件记忆上实现自我进化代理

MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory

Shanghai Jiao Tong University, Shanghai, China(上海交通大学);Xidian University, Xi'an, China(西安电子科技大学);National University of Singapore, Singapore(新加坡国立大学);Shanghai Innovation Institute, Shanghai, China(上海创新研究院);MemTensor (Shanghai) Technology Co., Ltd., Shanghai, China(MemTensor(上海)科技有限公司);University of Science and Technology of China, Hefei, China(中国科学技术大学)

Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Zhuo Li, Yujie Zheng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, Muning Wen

AI总结:

MemRL通过运行时强化学习在事件记忆上实现自我进化,解决稳定性与可塑性矛盾,实现持续改进。

摘要:

人类智能的显著特征是具备自我进化的能力,能够通过学习过去的经历来掌握新技能。然而,当前的AI代理在模拟这种自我进化方面存在困难:微调计算成本高且容易导致灾难性遗忘,而现有的基于记忆的方法依赖于被动的语义匹配,往往检索出噪声。为了解决这些挑战,我们提出了MemRL,一种非参数方法,通过在事件记忆上的强化学习进行进化。通过将稳定推理与可塑性记忆解耦,MemRL采用双阶段检索机制,通过环境反馈过滤噪声并识别高效益的策略。在HLE、BigCodeBench、ALFWorld和终身代理基准上的广泛实验表明,MemRL显著优于最先进的基线,证实了MemRL有效解决了稳定性与可塑性之间的矛盾,实现了无需权重更新的持续运行时改进。代码可在https://github.com/MemTensor/MemRL上获得。

英文摘要:

The hallmark of human intelligence is the self-evolving ability to master new skills by learning from past experiences. However, current AI agents struggle to emulate this self-evolution: fine-tuning is computationally expensive and prone to catastrophic forgetting, while existing memory-based methods rely on passive semantic matching that often retrieves noise. To address these challenges, we propose MemRL, a non-parametric approach that evolves via reinforcement learning on episodic memory. By decoupling stable reasoning from plastic memory, MemRL employs a Two-Phase Retrieval mechanism to filter noise and identify high-utility strategies through environmental feedback. Extensive experiments on HLE, BigCodeBench, ALFWorld, and Lifelong Agent Bench demonstrate that MemRL significantly outperforms state-of-the-art baselines, confirming that MemRL effectively reconciles the stability-plasticity dilemma, enabling continuous runtime improvement without weight updates. Code is available at https://github.com/MemTensor/MemRL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑