arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.30296cs.AI
收藏

ManimAgent: 用于视觉教育的自进化多模态智能体

ManimAgent: Self-Evolving Multimodal Agents for Visual Education

University of Alberta(阿尔伯塔大学);Southeast University(东南大学);Virginia Tech(弗吉尼亚理工学院);Xidian University(西安电子科技大学);Vivavia Inc(Vivavia公司)

Wenjia Jiang, Zongyuan Cai, Yuanhang Shao, Chenru Wang, Boyan Han, Zhixue Song, Keyu Chen, Shengwei An, Xu Yang, Zhou Yang

AI总结:

提出ManimAgent,通过双通道情节记忆库跨任务传递反思经验,无需权重更新或人工种子,在代码生成任务中提升通过率并减少反思轮次。

摘要:

多轮反思使基于大语言模型的智能体能够在单个任务中从失败中恢复,但每个任务仍然是孤立的片段:在一个任务的多轮反思中学到的经验在下一个任务开始前被丢弃。我们在一个代码生成任务上研究这一差距:从科学论文的章节中,智能体使用开源Manim库编写Python代码来渲染数学动画。我们提出了ManimAgent,一种自进化的多模态智能体,通过完全由其自身任务流生成的双通道情节记忆库(无需权重更新或人工种子)跨任务传递反思经验。每次动画收敛后,视觉语言模型对渲染的关键帧进行评分;得到的信号填充正通道M+(存储成功理由作为软参考示例)和负通道M-(存储验证过的失败模式作为硬已知陷阱)。在固定探针评估中,与无记忆、匹配预算的检索增强生成和打乱记忆基线相比,随着记忆规模增长,盲人人工Pass@1上升,反思轮次下降。我们将发布代码、冻结的记忆快照和任务流。

英文摘要:

Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode: lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code-generation task: from a scientific paper section, the agent writes Python in the open-source Manim library to render a mathematical animation. We present ManimAgent, a self-evolving multimodal agent that carries reflection experience across tasks through a dual-channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision-language model scores the rendered keyframes; the resulting signals populate a positive channel M+ that stores success rationales as soft Reference Examples, and a negative channel M- that stores validated failure patterns as hard Known Pitfalls. On a fixed-probe evaluation against no-memory, matched-budget retrieval-augmented generation, and shuffled-memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑