arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.14596cs.CVcs.AIcs.LG

VLM智能体生成自身记忆：将经验提炼为具身思维程序

VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought

Carnegie Mellon University（卡内基梅隆大学）；Google DeepMind（谷歌DeepMind）

Gabriel Sarch, Lawrence Jang, Michael J. Tarr, William W. Cohen, Kenneth Marino, Katerina Fragkiadaki

备注：
Project website: https://ical-learning.github.io/
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出上下文抽象学习（ICAL），让VLM智能体通过自我反思和人类反馈将次优轨迹提炼为高质量示例，在多个基准上显著超越原始示范并减少人工干预。

摘要：

大规模生成式语言模型和视觉-语言模型（LLMs和VLMs）在少样本学习中表现优异，但需要高质量的示范。我们提出上下文抽象学习（In-Context Abstraction Learning，ICAL），使VLM智能体能够通过自我反思和人类反馈将次优轨迹转化为高质量训练数据。给定不完美的任务示范，VLM通过纠正低效之处并标注认知抽象，将轨迹抽象为通用策略和动作标注，认知抽象包括：因果关系、物体状态变化、时间子目标和与任务相关的视觉元素。这些标注在相似环境中执行期间通过人类反馈迭代优化。所得示例在用于检索增强生成或微调时显著提升决策能力。随着智能体示例库的增长，其在抽象新示例时变得更加高效，所需人类反馈和环境交互更少。ICAL在多个基准上取得了最先进的结果。在TEACh基于对话的指令跟随任务中，在ICAL示例上结合微调和检索，目标条件成功率比原始人类示范和专家示例高出17.5%。在VisualWebArena中，使用ICAL的检索增强GPT-4V将任务成功率提升1.6倍，而微调后的Qwen2-VL相比基础模型实现了2.8倍的提升。在Ego4D动作预测中，我们超越了少样本GPT-4V，并与监督模型保持竞争力。我们的方法相比原始示范具有2倍更好的扩展性，并显著减少了人工提示工程的需求。

英文摘要：

Large-scale generative language and vision-language models (LLMs and VLMs) excel in few-shot learning but require high-quality demonstrations. We propose In-Context Abstraction Learning (ICAL), enabling VLM agents to transform suboptimal trajectories into high-quality training data through self-reflection and human feedback. Given imperfect task demonstrations, a VLM abstracts trajectories into generalized strategies and action annotations by correcting inefficiencies and annotating cognitive abstractions: causal relationships, object state changes, temporal subgoals, and task-relevant visual elements. These annotations are iteratively refined through human feedback during execution in similar environments. The resulting examples significantly improve decision-making when used for retrieval-augmented generation or fine-tuning. As the agent's example library grows, it becomes more efficient at abstracting new examples, requiring less human feedback and fewer environment interactions. ICAL achieves state-of-the-art results across multiple benchmarks. In TEACh dialogue-based instruction following, combining fine-tuning and retrieval on ICAL examples outperforms raw human demonstrations and expert examples by 17.5% in goal-condition success. In VisualWebArena, retrieval-augmented GPT-4V with ICAL improves task success 1.6x, while fine-tuned Qwen2-VL achieves 2.8x improvement over the base model. In Ego4D action forecasting, we surpass few-shot GPT-4V and remain competitive with supervised models. Our approach scales 2x better than raw demonstrations and significantly reduces manual prompt engineering requirements.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑