arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.10098cs.ROcs.CV

VLA-JEPA: 通过潜在世界模型增强视觉-语言-动作模型

VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model

University of Science and Technology of China(中国科学技术大学);Zhongguancun Academy, Beijing, China(中关村学院);Shanghai Jiao Tong University(上海交通大学);Tsinghua University(清华大学);Eastern Institute of Technology, Ningbo(宁波东部科技研究院);University of Chinese Academy of Sciences(中国科学院大学);Nankai University(南开大学)

Jingwen Sun, Wenyao Zhang, Zekun Qi, Shaojie Ren, Zezhi Liu, Hanxin Zhu, Guangzhong Sun, Xin Jin, Zhibo Chen

AI总结:

VLA-JEPA通过潜在世界模型提升视觉-语言-动作模型的泛化与鲁棒性,采用无泄露状态预测和两阶段训练策略。

摘要:

在互联网级视频上预训练视觉-语言-动作(VLA)策略具有吸引力,但当前的潜在动作目标往往学习了错误的内容:它们仍然锚定在像素变化上,而不是与动作相关的状态转换,使其容易受到外观偏差、干扰运动和信息泄露的影响。我们引入了VLA-JEPA,一种类似于JEPA的预训练框架,通过设计规避这些陷阱。关键思想是无泄露的状态预测:目标编码器从未来帧中生成潜在表示,而学生路径只看到当前观察——未来信息仅用于监督目标,从不作为输入。通过在潜在空间而非像素空间中进行预测,VLA-JEPA学习到对相机运动和无关背景变化具有鲁棒性的动态抽象。这产生了一个简单的两阶段配方——JEPA预训练后接动作头微调——而无需先前潜在动作流水线的多阶段复杂性。在LIBERO、LIBERO-Plus、SimplerEnv和现实世界操作任务上的实验表明,VLA-JEPA在泛化性和鲁棒性上均优于现有方法。

英文摘要:

Pretraining Vision-Language-Action (VLA) policies on internet-scale video is appealing, yet current latent-action objectives often learn the wrong thing: they remain anchored to pixel variation rather than action-relevant state transitions, making them vulnerable to appearance bias, nuisance motion, and information leakage. We introduce VLA-JEPA, a JEPA-style pretraining framework that sidesteps these pitfalls by design. The key idea is leakage-free state prediction: a target encoder produces latent representations from future frames, while the student pathway sees only the current observation -- future information is used solely as supervision targets, never as input. By predicting in latent space rather than pixel space, VLA-JEPA learns dynamics abstractions that are robust to camera motion and irrelevant background changes. This yields a simple two-stage recipe -- JEPA pretraining followed by action-head fine-tuning -- without the multi-stage complexity of prior latent-action pipelines. Experiments on LIBERO, LIBERO-Plus, SimplerEnv and real-world manipulation tasks show that VLA-JEPA achieves consistent gains in generalization and robustness over existing methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑