arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.14687cs.RO

RealMirror:一个面向具身智能的全面开源视觉-语言-动作平台

RealMirror: A Comprehensive, Open-Source Vision-Language-Action Platform for Embodied AI

ZTE Corporation(中通服科技有限公司);The Chinese University of Hong Kong(香港中文大学)
Cong Tai, Zhaoyu Zheng, Haixu Long, Hansheng Wu, Haodong Xiang, Zhengbin Long, Jun Xiong, Rong Shi, Shizhuang Zhang, Gang Qiu, He Wang, Ruifeng Li, Jun Huang, B… 展开作者

Cong Tai, Zhaoyu Zheng, Haixu Long, Hansheng Wu, Haodong Xiang, Zhengbin Long, Jun Xiong, Rong Shi, Shizhuang Zhang, Gang Qiu, He Wang, Ruifeng Li, Jun Huang, Bin Chang, Shuai Feng, Tao Shen

AI总结:

RealMirror是一个开源具身智能VLA平台,通过低成本数据采集训练系统、人形机器人VLA基准以及基于生成模型和3D高斯泼溅的零样本Sim2Real迁移,加速人形机器人VLA模型研发。

摘要:

面向人形机器人的视觉-语言-动作(VLA)这一新兴领域面临若干根本性挑战,包括数据采集成本高昂、缺乏标准化基准,以及仿真与现实之间存在显著差距。为克服这些障碍,我们提出了RealMirror,一个全面、开源的具身智能VLA平台。RealMirror构建了一套高效、低成本的数据采集、模型训练和推理系统,无需真实机器人即可实现端到端的VLA研究。为促进模型演进和公平比较,我们还引入了一个专门面向人形机器人的VLA基准,包含多个场景、大量轨迹和多种VLA模型。此外,通过整合生成模型和3D高斯泼溅(3D Gaussian Splatting)来重建逼真的环境和机器人模型,我们成功展示了零样本Sim2Real迁移:仅在仿真数据上训练的模型能够无缝地在真实机器人上执行任务,无需任何微调。总之,通过统一这些关键组件,RealMirror提供了一个稳健的框架,可显著加速人形机器人VLA模型的开发。项目主页:https://terminators2025.github.io/RealMirror.github.io

英文摘要:

The emerging field of Vision-Language-Action (VLA) for humanoid robots faces several fundamental challenges, including the high cost of data acquisition, the lack of a standardized benchmark, and the significant gap between simulation and the real world. To overcome these obstacles, we propose RealMirror, a comprehensive, open-source embodied AI VLA platform. RealMirror builds an efficient, low-cost data collection, model training, and inference system that enables end-to-end VLA research without requiring a real robot. To facilitate model evolution and fair comparison, we also introduce a dedicated VLA benchmark for humanoid robots, featuring multiple scenarios, extensive trajectories, and various VLA models. Furthermore, by integrating generative models and 3D Gaussian Splatting to reconstruct realistic environments and robot models, we successfully demonstrate zero-shot Sim2Real transfer, where models trained exclusively on simulation data can perform tasks on a real robot seamlessly, without any fine-tuning. In conclusion, with the unification of these critical components, RealMirror provides a robust framework that significantly accelerates the development of VLA models for humanoid robots. Project page: https://terminators2025.github.io/RealMirror.github.io

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑