arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.21091cs.AIcs.CL
收藏

MemComposer:通过强化学习学习为大语言模型智能体组装记忆

MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning

University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Amazon(亚马逊公司)

Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han

AI总结:

该研究提出MemComposer,一种基于ExPO框架训练的记忆组装器,通过检查查询与候选记忆优化记忆选择排序,在τ²-bench实验中提升性能并降低记忆令牌使用,策略可迁移且受益于强模型记忆。

摘要:

智能体记忆使大语言模型(LLM)智能体能够复用过往经验而无需更新其参数。然而,仅语义相似性无法保证记忆的实用性:下游性能取决于如何为智能体选择和呈现记忆。我们提出MemComposer,这是一种记忆组装器,它同时检查当前查询和检索到的候选记忆,以决定包含哪些经验、选择多少个以及如何排序。我们使用Experience Policy Optimization(ExPO)训练该组装器,这是一种基于GRPO的框架,通过完整的智能体交互评估组装后的记忆上下文,并利用任务成功反馈来优化组装策略。在三个τ²-bench域中使用两个冻结执行器时,MemComposer在每个执行器上均优于最强基线,同时分别将平均记忆令牌使用量降低16.9%和4.9%。所学策略还可迁移到未见过的域、执行器和记忆库,且无需额外训练即可受益于更强模型生成的记忆。这些结果证明了利用下游任务反馈学习查询特定记忆组装的价值。

英文摘要:

Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory usefulness: downstream performance depends on how memories are selected and presented to the agent. We propose MemComposer, a memory composer that jointly examines the current query and retrieved candidates to decide which experiences to include, how many to select, and how to order them. We train the composer using Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through complete agent interactions and uses task-success feedback to optimize the assembly policy. Across three $τ^2$-bench domains with two frozen executors, MemComposer outperforms the strongest baseline for each executor while reducing average memory-token usage by 16.9% and 4.9%, respectively. The learned policy also transfers to unseen domains, executors, and memory banks, and benefits from memories generated by stronger models without further training. These results demonstrate the value of learning query-specific memory assembly using downstream task feedback.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑