arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.02881cs.RO

MolmoAct2:面向现实部署的动作推理模型

MolmoAct2: Action Reasoning Models for Real-world Deployment

Allen Institute for AI(艾伦人工智能研究所);University of Washington(华盛顿大学);National University of Singapore(新加坡国立大学);University of Pennsylvania(宾夕法尼亚大学);Johns Hopkins University(约翰霍普金斯大学);Amazon(亚马逊公司);University of Michigan(密歇根大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, Shanli Xing, Jaemin Cho, Jae Sung… 展开作者

Haoquan Fang, Jiafei Duan, Donovan Clay, Sam Wang, Shuo Liu, Weikai Huang, Xiang Fan, Wei-Chuan Tsai, Shirui Chen, Yi Ru Wang, Shanli Xing, Jaemin Cho, Jae Sung Park, Ainaz Eftekhar, Peter Sushko, Karen Farley, Angad Wadhwa, Cole Harrison, Winson Han, Ying-Chun Lee, Eli VanderBilt, Rose Hendrix, Suveen Ellawela, Lucas Ngoo, Joyce Chai, Zhongzheng Ren, Ali Farhadi, Dieter Fox, Ranjay Krishna

AI总结:

本文提出MolmoAct2,一个完全开放的动作推理模型,通过改进架构和引入新数据集,在五个方面提升性能,展示了在多个基准测试中优于现有模型的成果。

摘要:

Vision-Language-Action (VLA)模型旨在为机器人提供单一通用控制器,但当前系统在现实部署中表现不足。Frontier模型封闭,开放权重替代方案依赖昂贵硬件,推理增强策略因接地而产生高延迟,微调成功率仍低于可靠使用的阈值。我们提出了MolmoAct2,一个为实际部署设计的完全开放动作推理模型,在五个方面改进其前代。我们引入了MolmoER,一个专门用于空间和具身推理的VLM基础模型,训练于330万个样本数据集。我们发布了三个新数据集,涵盖低至中等成本平台,包括MolmoAct2-BimanualYAM,720小时的遥控双臂轨迹,以及经过质量筛选的Franka (DROID)和SO100/101子集。我们提供了OpenFAST,一个开放权重、开放数据的动作分词器,训练于数百万条轨迹。我们重新设计了架构,通过每层KV缓存条件将流匹配连续动作专家连接到离散令牌VLM。最后,我们提出了MolmoThink,一种自适应深度推理变体,仅对场景中变化的区域重新预测深度令牌,保留几何基础,仅消耗先前延迟的分数。在迄今为止最广泛的开放VLA实证研究中,MolmoAct2在7个模拟和现实基准测试中优于强基线模型,包括Pi-05,而MolmoER在13个具身推理基准测试中超过GPT-5和Gemini Robotics ER-1.5。我们发布了模型权重、训练代码和完整训练数据。项目页面:https://allenai.org/blog/molmoact2

英文摘要:

Vision-Language-Action (VLA) models aim to provide a single generalist controller for robots, but today's systems fall short on the criteria that matter for real-world deployment. Frontier models are closed, open-weight alternatives are tied to expensive hardware, reasoning-augmented policies pay prohibitive latency for their grounding, and fine-tuned success rates remain below the threshold for dependable use. We present MolmoAct2, a fully open action reasoning model built for practical deployment, advancing its predecessor along five axes. We introduce MolmoER, a VLM backbone specialized for spatial and embodied reasoning, trained on a 3.3M-sample corpus with a specialize-then-rehearse recipe. We release three new datasets spanning low-to-medium cost platforms, including MolmoAct2-BimanualYAM, 720 hours of teleoperated bimanual trajectories that constitute the largest open bimanual dataset to date, together with quality-filtered Franka (DROID) and SO100/101 subsets. We provide OpenFAST, an open-weight, open-data action tokenizer trained on millions of trajectories across five embodiments. We redesign the architecture to graft a flow-matching continuous-action expert onto a discrete-token VLM via per-layer KV-cache conditioning. Finally, we propose MolmoThink, an adaptive-depth reasoning variant that re-predicts depth tokens only for scene regions that change between timesteps, retaining geometric grounding at a fraction of prior latency. In the most extensive empirical study of any open VLA to date, spanning 7 simulation and real-world benchmarks, MolmoAct2 outperforms strong baselines including Pi-05, while MolmoER surpasses GPT-5 and Gemini Robotics ER-1.5 across 13 embodied-reasoning benchmarks. We release model weights, training code, and complete training data. Project page: https://allenai.org/blog/molmoact2

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑