联合对齐的潜在动作：迈向大规模野外VLA预训练
Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild
AI总结：
JALA通过联合对齐的潜在动作预训练框架，提升从人类数据中大规模预训练视觉-语言-动作模型的效率与性能。
摘要：
尽管取得了进展，视觉-语言-动作模型（VLAs）仍然受到大规模、多样化的机器人数据稀缺的限制。虽然人类操作视频提供了一个丰富的替代选择，但现有方法被迫在小而精确标注的数据集和大量野外 footage 之间做出选择，后者具有不可靠的手部跟踪标签。我们提出了JALA，一种预训练框架，学习联合对齐的潜在动作。JALA绕过了完整的视觉动态重建，而是学习一种预测性动作嵌入，与逆动态和真实动作对齐。这产生了具有过渡意识、行为导向的潜在空间，用于学习异构的人类数据。我们通过UniHand-Mix，一个包含750万视频库（>2000小时）的混合数据集，扩展了这一方法，结合实验室和野外 footage。实验表明，JALA在受控和非受控场景中都能生成更逼真的人手动作，显著提高了下游机器人操作在模拟和现实任务中的性能。这些结果表明，联合对齐的潜在动作为从人类数据中大规模预训练VLA提供了一条可行的路径。
英文摘要：
Despite progress, Vision-Language-Action models (VLAs) are limited by a scarcity of large-scale, diverse robot data. While human manipulation videos offer a rich alternative, existing methods are forced to choose between small, precisely-labeled datasets and vast in-the-wild footage with unreliable hand tracking labels. We present JALA, a pretraining framework that learns Jointly-Aligned Latent Actions. JALA bypasses full visual dynamic reconstruction, instead learns a predictive action embedding aligned with both inverse dynamics and real actions. This yields a transition-aware, behavior-centric latent space for learning from heterogeneous human data. We scale this approach with UniHand-Mix, a 7.5M video corpus (>2,000 hours) blending laboratory and in-the-wild footage. Experiments demonstrate that JALA generates more realistic hand motions in both controlled and unconstrained scenarios, significantly improving downstream robot manipulation performance in both simulation and real-world tasks. These results indicate that jointly-aligned latent actions offer a scalable pathway for VLA pretraining from human data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
