通过世界建模的潜在动作预训练
Latent Action Pretraining Through World Modeling
AI总结：
提出LAWM框架，通过世界建模从无标签视频中学习潜在动作表征，实现跨任务、环境和本体的迁移学习，在LIBERO基准和真实场景中优于使用真实动作预训练的方法。
摘要：
视觉-语言-动作（VLA）模型在遵循语言指令的机器人操作任务学习中越来越受欢迎。最先进的VLA模型，如OpenVLA和$\pi_{0}$，是在通过遥操作收集的大规模手动标注动作数据集上训练的。最近的方法，包括LAPA和villa-X，引入了潜在动作表示，通过建模帧间的抽象视觉变化，实现在无标签数据集上的无监督预训练。尽管这些方法展示了强大的结果，但它们的大模型尺寸使得在真实世界环境中部署具有挑战性。在这项工作中，我们提出了LAWM，一个模型无关的框架，通过世界建模从无标签视频数据中学习潜在动作表示，以自监督方式预训练模仿学习模型。这些视频可以来自机器人记录或人类使用日常物品执行动作的视频。我们的框架能够跨任务、环境和本体迁移所学知识。它在LIBERO基准和真实世界设置中优于使用真实机器人动作预训练的模型以及其他类似的预训练方法，同时在真实世界环境中高效且实用。
英文摘要：
Vision-Language-Action (VLA) models have gained popularity for learning robotic manipulation tasks that follow language instructions. State-of-the-art VLAs, such as OpenVLA and $π_{0}$, were trained on large-scale, manually labeled action datasets collected through teleoperation. More recent approaches, including LAPA and villa-X, introduce latent action representations that enable unsupervised pretraining on unlabeled datasets by modeling abstract visual changes between frames. Although these methods have shown strong results, their large model sizes make deployment in real-world settings challenging. In this work, we propose LAWM, a model-agnostic framework to pretrain imitation learning models in a self-supervised way, by learning latent action representations from unlabeled video data through world modeling. These videos can be sourced from robot recordings or videos of humans performing actions with everyday objects. Our framework is able to transfer learned knowledge across tasks, environments, and embodiments. It outperforms models pretrained with ground-truth robot actions and other similar pretraining methods on the LIBERO benchmark and real-world setup, while being efficient and practical for real-world settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
