arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2609.37923cs.CV

EpiCon:通过共同演化多模态记忆实现智能体集体学习

EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory

  • MIT-IBM Computing Research Lab(MIT-IBM计算研究实验室)
  • University of Rochester(罗切斯特大学)
  • Massachusetts Institute of Technology(麻省理工学院)

机构由 AI 辅助整理,请以论文原文为准。

Ziyun Zeng, Hang Hua, Shaden Alshammari, Rogerio Feris, William T. Freeman, Jiebo Luo

AI总结:

EpiCon提出共享多模态记忆框架,通过记忆控制器和树状自组织器实现智能体集体学习,在十一个基准上将宏平均分数提升1.7至4.9分,并显著降低记忆操作时间。

AI中文摘要:

智能体可以从过去的执行中学习,但让不同智能体复用并基于彼此的经验进行构建仍然具有挑战性。我们提出了EpiCon,一个共享的多模态记忆框架,用于智能体集体学习,无需更新宿主模型参数。EpiCon通过两个独立训练的2B模型(一个记忆控制器和一个树状自组织器)将问题级别的记忆演化与持久经验库连接起来。控制器在多次尝试中联合优化文本指导和视觉证据,并选择性纳入视觉记忆。自组织器分层整合经验教训,并为新问题检索经验和规则。我们在涵盖四个多模态任务域的十一个基准上评估了EpiCon,使用了两个测试框架和多个主干模型。一个冻结的经验库即使在单次求解尝试中也能提升其他系统的性能。第二个测试框架将原始系统的宏平均分数在十一个基准上提高了2.6分。在四种宿主配置中,与无记忆相比,EpiCon将宏平均分数提高了1.7至4.9分,并将记忆操作时间相对于主干规模记忆模型减少了67%至74%。

英文摘要:

Agents can learn from past executions, but enabling different agents to reuse and build on one another's experience remains challenging. We introduce EpiCon, a shared multimodal memory framework for agent collective learning without updating host model parameters. EpiCon links question-level memory evolution to a persistent experience bank through two independently trained 2B models: a memory controller and a tree self-organizer. The controller jointly refines textual guidance and visual evidence across attempts and selectively includes visual memory. The self-organizer consolidates lessons hierarchically and retrieves experience and rules for new problems. We evaluate EpiCon on eleven benchmarks spanning four multimodal task domains, using two harnesses and multiple backbones. A frozen bank improves other systems even with a single solving attempt. A second harness raises the original system's macro-average score by 2.6 points across eleven benchmarks. Across four host configurations, EpiCon improves macro-average scores by 1.7 to 4.9 points over No Memory and reduces memory-operation time by 67\% to 74\% relative to backbone-sized memory models.

补充信息

↑