arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.10983cs.RO
收藏

为分层操作策略扩展世界模型

Scaling World Model for Hierarchical Manipulation Policies

Xi’an Jiao Tong University(西安交通大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Qian Long, Yueze Wang, Jiaxi Song, Junbo Zhang, Peiyan Li, Wenxuan Wang, Yuqi Wang, Haoyang Li, Shaoxuan Xie, Guocai Yao, Hanbo Zhang, Xinlong Wang, Zhongyuan W… 展开作者

Qian Long, Yueze Wang, Jiaxi Song, Junbo Zhang, Peiyan Li, Wenxuan Wang, Yuqi Wang, Haoyang Li, Shaoxuan Xie, Guocai Yao, Hanbo Zhang, Xinlong Wang, Zhongyuan Wang, Xuguang Lan, Huaping Liu, Xinghang Li

AI总结:

本文提出分层VLA框架,利用大规模预训练世界模型提升机器人操作在分布外场景的泛化能力,实验显示性能提升显著。

摘要:

视觉-语言-动作(VLA)模型在通用机器人操作中具有前景,但在分布外(OOD)设置中仍然脆弱,尤其是在现实机器人数据有限的情况下。为解决泛化瓶颈,我们引入了一种分层视觉-语言-动作框架,利用大规模预训练世界模型的泛化能力,以实现稳健且可推广的VIsual Subgoal TAsk分解VISTA。我们的分层框架由世界模型作为高层规划器和VLA作为低层执行器组成。高层世界模型首先将操作任务分解为带有目标图像的子任务序列,而低层策略则遵循文本和视觉指导生成动作序列。与原始文本目标规范相比,这些合成的目标图像为低层策略提供了视觉和物理基础的细节,使其能够跨未见过的对象和新场景泛化。我们验证了视觉目标合成和我们的分层VLA策略在大规模分布外场景中的表现,相同结构的VLA在新场景中的性能可从14%提升到69%,通过世界模型生成的指导。结果表明,我们的方法在分布外场景中显著优于先前的基线方法。项目页面:https://vista-wm.github.io/

英文摘要:

Vision-Language-Action (VLA) models are promising for generalist robot manipulation but remain brittle in out-of-distribution (OOD) settings, especially with limited real-robot data. To resolve the generalization bottleneck, we introduce a hierarchical Vision-Language-Action framework \our{} that leverages the generalization of large-scale pre-trained world model for robust and generalizable VIsual Subgoal TAsk decomposition VISTA. Our hierarchical framework \our{} consists of a world model as the high-level planner and a VLA as the low-level executor. The high-level world model first divides manipulation tasks into subtask sequences with goal images, and the low-level policy follows the textual and visual guidance to generate action sequences. Compared to raw textual goal specification, these synthesized goal images provide visually and physically grounded details for low-level policies, making it feasible to generalize across unseen objects and novel scenarios. We validate both visual goal synthesis and our hierarchical VLA policies in massive out-of-distribution scenarios, and the performance of the same-structured VLA in novel scenarios could boost from 14% to 69% with the guidance generated by the world model. Results demonstrate that our method outperforms previous baselines with a clear margin, particularly in out-of-distribution scenarios. Project page: \href{https://vista-wm.github.io/}{https://vista-wm.github.io}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑