arXiv 2506.239192026-03-30cs.RO

Goal-VLA: 图像生成视觉语言模型作为对象中心世界模型，赋能零样本机器人操作

Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation

School of Computing, National University of Singapore（新加坡国立大学计算机学院）；The HKU Musketeers Foundation Institute of Data Science, The University of Hong Kong（香港大学数据科学研究院）；Yuanpei College, Peking University（北京大学元培学院）；Department of Automation, Tsinghua University（清华大学自动化系）

Haonan Chen, Jingxiang Guo, Bangjun Wang, Tianrui Zhang, Xuchuan Huang, Boren Zheng, Yiwen Hou, Chenrui Tie, Jiajun Deng, Lin Shao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Goal-VLA框架，利用图像生成视觉语言模型生成目标状态，通过对象状态表示分离高阶与低阶策略，提升机器人操作的泛化能力。

摘要：

泛化仍是机器人操作中的基本挑战。为解决此问题，最近的视觉语言-动作（VLA）模型在视觉语言模型（VLMs）上构建策略，试图转移其开放世界语义知识。然而，其零样本能力显著落后于基础VLMs，因为指令-视觉-动作数据过于有限，无法覆盖多样化的场景、任务和机器人形态。在本文中，我们提出Goal-VLA，一种零样本框架，利用图像生成VLMs作为世界模型生成所需的目标状态，从而推导出目标物体姿态以实现可泛化的操作。关键见解是对象状态表示是黄金接口，自然地将操作系统分为高层和低层策略。这种表示抽象了显式动作注释，允许使用高度可泛化的VLMs，同时提供空间线索以免训练的低层控制。为进一步提高鲁棒性，我们引入了反射-合成过程，该过程迭代验证并优化生成的目标图像后再执行。模拟和现实世界实验表明，我们的方法在操作任务中实现了强大的性能和令人鼓舞的泛化能力。补充材料可在https://nus-lins-lab.github.io/goalvlaweb/上获取。

英文摘要：

Generalization remains a fundamental challenge in robotic manipulation. To tackle this challenge, recent Vision-Language-Action (VLA) models build policies on top of Vision-Language Models (VLMs), seeking to transfer their open-world semantic knowledge. However, their zero-shot capability lags significantly behind the base VLMs, as the instruction-vision-action data is too limited to cover diverse scenarios, tasks, and robot embodiments. In this work, we present Goal-VLA, a zero-shot framework that leverages Image-Generative VLMs as world models to generate desired goal states, from which the target object pose is derived to enable generalizable manipulation. The key insight is that object state representation is the golden interface, naturally separating a manipulation system into high-level and low-level policies. This representation abstracts away explicit action annotations, allowing the use of highly generalizable VLMs while simultaneously providing spatial cues for training-free low-level control. To further improve robustness, we introduce a Reflection-through-Synthesis process that iteratively validates and refines the generated goal image before execution. Both simulated and real-world experiments demonstrate that our \name achieves strong performance and inspiring generalizability in manipulation tasks. Supplementary materials are available at https://nus-lins-lab.github.io/goalvlaweb/.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。