发表机构
Yonsei University; Seoul National University（延世大学; 首尔大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出SimVLA框架，基于仿真数据训练VLA，在补货等任务上零样本迁移至真实移动操作，性能优于50个真实演示训练的策略，凸显仿真在该领域的可扩展性价值。
AI 中文摘要
大规模多样化数据集推动了大语言模型（LLM）和视觉语言模型（VLM）的成功，但面向机器人的视觉语言动作模型（VLA）仍受限于真实世界数据采集的成本与复杂度。尽管仿真提供了可扩展的替代方案，但其在移动操作领域的仿真到真实VLA学习潜力仍未得到充分探索。本文提出SimVLA，一种完全基于合成仿真数据训练、无需遥操作的移动操作端到端VLA框架。SimVLA首先在两个互补的仿真衍生数据集上进行预训练：一是SimAction，这是一个涵盖35种不同移动操作任务的大规模机器人动作数据集，通过组合原子技能生成；二是SimVQA，该数据集利用特权仿真状态提供空间、几何及子任务级别的视觉语言监督。我们进一步在SimAction与SimDeploy的混合数据集上对SimVLA进行后训练，其中SimDeploy是从不同仿真环境中的策略 rollout 收集的数据集。我们在补货、倾倒、清洁等任务上对SimVLA进行评估，结果显示其可零样本迁移至真实世界移动操作，包括真实家庭环境。SimVLA的性能优于基于50个领域内真实世界演示训练的策略，表明仿真可实现可扩展的仿真到真实移动操作。我们还证明了多种互补监督形式对有效利用仿真进行VLA训练的价值。
英文摘要
Large-scale, diverse datasets have driven the success of LLMs and VLMs. But VLAs for robotics remain limited by the cost and complexity of real-world data collection. While simulation offers a scalable alternative, its potential for sim-to-real VLA learning in mobile manipulation remains largely underexplored. We introduce SimVLA, an end-to-end framework that trains VLAs entirely on synthetic simulation data without teleoperation for mobile manipulation. SimVLA is first pre-trained on two complementary simulation-derived datasets: SimAction, a large-scale robot action dataset spanning 35 diverse mobile manipulation tasks, generated by composing atomic skills, and SimVQA, which leverages privileged simulator state to provide spatial, geometric, and subtask-level visual-language supervision. We further post-train SimVLA on a mixture of SimAction and SimDeploy, a dataset collected from policy rollouts across diverse simulated environments. We evaluate SimVLA on tasks including restocking, pouring, and cleaning, and show zero-shot transfer to real-world mobile manipulation, including real home environments. SimVLA outperforms policies trained on 50 in-domain real-world demonstrations, suggesting that simulation can enable scalable sim-to-real mobile manipulation. We further demonstrate the value of multiple complementary forms of supervision for effectively leveraging simulation in VLA training.
CommentsProject page: https://kyounginbaik.github.io/simvla/
