arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.02466cs.ROcs.AI
收藏

先学移动再学做事:面向VLA的任务无关预训练

Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs

Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院)

Junhao Shi, Siyin Wang, Xiaopeng Yu, Li Ji, Jingjing Gong, Xipeng Qiu

AI总结:

提出任务无关预训练(TAP)框架,先通过自监督逆动力学从廉价无标签交互数据学习可迁移运动先验,再用少量专家数据将先验与语言对齐,显著降低VLA模型对专家演示的依赖。

摘要:

视觉-语言-动作(VLA)模型根本上受限于专家演示的稀缺性——即观察、指令和动作的三元组,这些数据大规模收集成本高昂。我们认为,这一瓶颈源于混淆了两个不同的学习目标:获取物理能力(如何移动)和获取语义对齐(做什么)。关键在于,只有后者需要语言监督。基于这一分解假设,我们提出了任务无关预训练(TAP),这是一个两阶段框架:首先通过自监督逆动力学目标,从廉价的无标签交互数据(包括丢弃的离任务轨迹和自主机器人玩耍)中学习可迁移的运动先验;然后,第二阶段利用最少的专家数据将这些先验与语言对齐。在SIMPLER基准上,TAP匹配了使用超过100万条专家轨迹训练的模型,同时使用的标注数据量少几个数量级,相比标准行为克隆取得了10%的绝对提升。在真实世界的WidowX平台上,当相机发生扰动时,TAP保持了25%的成功率,而基于互联网规模的基线模型则降至0%,这表明任务无关预训练能够产生鲁棒、可迁移的物理表示,并为具身AI提供了一条可扩展的前进道路。

英文摘要:

Vision-Language-Action (VLA) models are fundamentally bottlenecked by the scarcity of expert demonstrations -- triplets of observations, instructions, and actions that are costly to collect at scale. We argue that this bottleneck stems from conflating two distinct learning objectives: acquiring physical competence (how to move) and acquiring semantic alignment (what to do). Crucially, only the latter requires language supervision. Building on this Decomposition Hypothesis, we propose Task-Agnostic Pretraining (TAP), a two-stage framework that first learns transferable motor priors from cheap, unlabeled interaction data -- including discarded off-task trajectories and autonomous robot play -- via a self-supervised Inverse Dynamics objective. A lightweight second stage then grounds these priors in language using minimal expert data. On the SIMPLER benchmark, TAP matches models trained on over 1M expert trajectories while using orders of magnitude less labeled data, yielding a 10% absolute gain over standard behavior cloning. On a real-world WidowX platform, TAP retains 25% success under camera perturbations where internet-scale baselines collapse to 0%, demonstrating that task-agnostic pretraining produces robust, transferable physical representations and offers a scalable path forward for Embodied AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑