用于可泛化机器人控制的原生视频动作预训练
Native Video-Action Pretraining for Generalizable Robot Control
浏览论文内容
中文总结 AI 辅助
研究针对机器人控制中视频动作模型应用于物理环境的不足，提出LingBot-VA 2.0，通过语义视觉动作分词器等四个核心设计原则构建基础模型，经真实世界部署验证其在复杂操作任务中的少样本泛化能力。
中文摘要 AI 辅助
视频动作模型的出现为机器人控制提供了一条有前景的路径。然而，将为数字内容创作设计的视频生成模型用于物理环境存在固有不足。为弥合这一差距，我们提出了LingBot-VA 2.0，一个为具身化全新构建的视频动作基础模型。介绍了四个核心设计原则，包括引入语义视觉动作分词器、采用因果预训练范式、使用稀疏混合专家骨干以及通过增强异步推理方案实现实时闭环控制。真实世界部署验证了其作为强大基础模型的有效性。
英文摘要
The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.
