arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.14974cs.RO
收藏

DM0:一种面向物理AI的具身原生视觉-语言-动作模型

DM0: An Embodied-Native Vision-Language-Action Model towards Physical AI

DM0 Team(DM0团队);Dexmal;StepFun
En Yu, Haoran Lv, Jianjian Sun, Kangheng Lin, Ruitao Zhang, Yukang Shi, Yuyang Chen, Ze Chen, Ziheng Zhang, Fan Jia, Kaixin Liu, Meng Zhang, Ruitao Hao, Saike H… 展开作者

En Yu, Haoran Lv, Jianjian Sun, Kangheng Lin, Ruitao Zhang, Yukang Shi, Yuyang Chen, Ze Chen, Ziheng Zhang, Fan Jia, Kaixin Liu, Meng Zhang, Ruitao Hao, Saike Huang, Songhan Xie, Yu Liu, Zhao Wu, Bin Xie, Pengwei Zhang, Qi Yang, Xianchi Deng, Yunfei Wei, Enwen Zhang, Hongyang Peng, Jie Zhao, Kai Liu, Wei Sun, Yajun Wei, Yi Yang, Yunqiao Zhang, Ziwei Yan, Haitao Yang, Hao Liu, Haoqiang Fan, Haowei Zhang, Junwen Huang, Yang Chen, Yunchao Ma, Yunhuan Yang, Zhengyuan Du, Ziming Liu, Jiahui Niu, Yucheng Zhao, Daxin Jiang, Wenbin Tang, Xiangyu Zhang, Zheng Ge, Erjin Zhou, Tiancai Wang

AI总结:

DM0提出了一种面向物理AI的具身原生视觉-语言-动作模型,通过统一具身操作和导航,结合预训练、中训练和后训练流程,实现对复杂物理任务的高效处理。

摘要:

超越传统将互联网预训练模型适应到物理任务的范式,我们提出了DM0,一种面向物理AI的具身原生视觉-语言-动作(VLA)框架。不同于将物理基础视为微调后的次要步骤,DM0通过从异构数据源中学习,统一了具身操作和导航。我们的方法遵循一个全面的三阶段流程:预训练、中训练和后训练。首先,我们使用多样化的语料库对视觉-语言模型(VLM)进行大规模统一预训练,无缝整合网络文本、自动驾驶场景和具身交互日志,以共同获取语义知识和物理先验。随后,我们构建了一个基于VLM的流匹配动作专家。为了协调高层次推理与低层次控制,DM0采用了一种混合训练策略:对于具身数据,动作专家的梯度不回传至VLM以保持通用表示,而VLM在非具身数据上仍可训练。此外,我们引入了具身空间 scaffolding 策略来构建空间链式思维(CoT)推理,有效约束动作解决方案空间。在RoboChallenge基准测试中,DM0在Table30的 Specialist 和 Generalist 设置中均实现了最先进的性能。

英文摘要:

Moving beyond the traditional paradigm of adapting internet-pretrained models to physical tasks, we present DM0, an Embodied-Native Vision-Language-Action (VLA) framework designed for Physical AI. Unlike approaches that treat physical grounding as a fine-tuning afterthought, DM0 unifies embodied manipulation and navigation by learning from heterogeneous data sources from the onset. Our methodology follows a comprehensive three-stage pipeline: Pretraining, Mid-Training, and Post-Training. First, we conduct large-scale unified pretraining on the Vision-Language Model (VLM) using diverse corpora--seamlessly integrating web text, autonomous driving scenarios, and embodied interaction logs-to jointly acquire semantic knowledge and physical priors. Subsequently, we build a flow-matching action expert atop the VLM. To reconcile high-level reasoning with low-level control, DM0 employs a hybrid training strategy: for embodied data, gradients from the action expert are not backpropagated to the VLM to preserve generalized representations, while the VLM remains trainable on non-embodied data. Furthermore, we introduce an Embodied Spatial Scaffolding strategy to construct spatial Chain-of-Thought (CoT) reasoning, effectively constraining the action solution space. Experiments on the RoboChallenge benchmark demonstrate that DM0 achieves state-of-the-art performance in both Specialist and Generalist settings on Table30.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑