Evo-Memory：通过自演化记忆基准测试LLM代理的测试时间学习
Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory
AI总结：
本文提出Evo-Memory，一个用于评估LLM代理自演化记忆能力的综合流基准和框架，通过构建序列任务流数据集，要求LLM在每次交互后搜索、适应和演化记忆，并在10个多样化的多轮目标导向和单轮推理与问答数据集上评估了超过十种代表性的记忆模块。
摘要：
状态性对于大型语言模型（LLM）代理进行长期规划和问题解决至关重要。这使得记忆成为关键组件，但其管理和进化仍 largely underexplored。现有的评估主要集中在静态对话设置上，其中记忆被动地从对话中检索以回答查询，忽略了在不断变化的任务流中积累和重用经验的能力。在现实世界环境中，如交互问题助手或具身代理中，LLM需要处理连续的任务流，但通常无法从积累的交互中学习，失去有价值的上下文见解，这限制了测试时间的进化，即LLM在部署期间持续检索、整合和更新记忆。为了弥合这一差距，我们引入了Evo-Memory，一个综合的流基准和框架，用于评估LLM代理的自演化记忆能力。Evo-Memory将数据集结构化为连续的任务流，要求LLM在每次交互后搜索、适应和演化记忆。我们统一并实现了超过十种代表性的记忆模块，并在10个多样化的多轮目标导向和单轮推理与问答数据集上评估了它们。为了更好地基准测试经验重用，我们提供了一个基线方法ExpRAG，用于检索和利用先前经验，并进一步提出ReMem，一个将推理、任务动作和记忆更新紧密集成的行动-思考-记忆精炼流程，以实现持续改进。
英文摘要：
Statefulness is essential for large language model (LLM) agents to perform long-term planning and problem-solving. This makes memory a critical component, yet its management and evolution remain largely underexplored. Existing evaluations mostly focus on static conversational settings, where memory is passively retrieved from dialogue to answer queries, overlooking the dynamic ability to accumulate and reuse experience across evolving task streams. In real-world environments such as interactive problem assistants or embodied agents, LLMs are required to handle continuous task streams, yet often fail to learn from accumulated interactions, losing valuable contextual insights, a limitation that calls for test-time evolution, where LLMs retrieve, integrate, and update memory continuously during deployment. To bridge this gap, we introduce Evo-Memory, a comprehensive streaming benchmark and framework for evaluating self-evolving memory in LLM agents. Evo-Memory structures datasets into sequential task streams, requiring LLMs to search, adapt, and evolve memory after each interaction. We unify and implement over ten representative memory modules and evaluate them across 10 diverse multi-turn goal-oriented and single-turn reasoning and QA datasets. To better benchmark experience reuse, we provide a baseline method, ExpRAG, for retrieving and utilizing prior experience, and further propose ReMem, an action-think-memory refine pipeline that tightly integrates reasoning, task actions, and memory updates to achieve continual improvement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
