想象、验证、执行：基于记忆引导的具身智能体探索与视觉语言模型
Imagine, Verify, Execute: Memory-guided Agentic Exploration with Vision-Language Models
AI总结：
提出IVE框架，利用视觉语言模型将观测抽象为场景图，通过想象、验证、执行实现具身智能体的多样化探索，在仿真和真实环境中显著提升状态熵，并支持下游策略学习。
摘要：
探索对于通用机器人学习至关重要，尤其是在奖励稀疏、目标不明确或任务特定监督匮乏的开放式环境中。视觉语言模型（VLMs）凭借其对物体、空间关系和潜在结果的语义推理能力，为生成高层探索行为提供了坚实的基础。然而，其输出往往缺乏物理基础，难以判断想象出的状态转换是否物理可行或具有信息量。为了弥合想象与执行之间的鸿沟，我们提出了IVE（想象、验证、执行），一个受人类好奇心启发的智能体探索框架。人类探索通常由发现新场景配置和加深对环境理解的欲望驱动。类似地，IVE利用视觉语言模型将RGB-D观测抽象为语义场景图，想象新场景，预测其物理合理性，并通过动作工具生成可执行的技能序列。我们在仿真和真实世界的桌面环境中评估了IVE。结果表明，与强化学习基线相比，IVE实现了更多样化和更有意义的探索，访问状态的熵增加了4.1至7.8倍。此外，收集的经验支持下游学习，产生的策略在性能上接近或超过基于人类收集的演示训练的策略。
英文摘要：
Exploration is essential for general-purpose robotic learning, especially in open-ended environments where dense rewards, explicit goals, or task-specific supervision are scarce. Vision-language models (VLMs), with their semantic reasoning over objects, spatial relations, and potential outcomes, present a compelling foundation for generating high-level exploratory behaviors. However, their outputs are often ungrounded, making it difficult to determine whether imagined transitions are physically feasible or informative. To bridge the gap between imagination and execution, we present IVE (Imagine, Verify, Execute), an agentic exploration framework inspired by human curiosity. Human exploration is often driven by the desire to discover novel scene configurations and to deepen understanding of the environment. Similarly, IVE leverages VLMs to abstract RGB-D observations into semantic scene graphs, imagine novel scenes, predict their physical plausibility, and generate executable skill sequences through action tools. We evaluate IVE in both simulated and real-world tabletop environments. The results show that IVE enables more diverse and meaningful exploration than RL baselines, as evidenced by a 4.1 to 7.8x increase in the entropy of visited states. Moreover, the collected experience supports downstream learning, producing policies that closely match or exceed the performance of those trained on human-collected demonstrations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
