Olaf-World: 面向视频世界模型的潜在动作定向
Olaf-World: Orienting Latent Actions for Video World Modeling
AI总结：
提出SeqΔ-REPA对齐目标，通过冻结自监督视频编码器的时序特征差异锚定潜在动作，实现无标签视频中可迁移的动作控制世界模型预训练。
摘要：
扩展动作可控世界模型受限于动作标签的稀缺性。虽然潜在动作学习有望从无标签视频中提取控制接口，但学习到的潜在表示往往难以跨上下文迁移：它们纠缠了场景特定线索，缺乏共享坐标系。这是因为标准目标仅在每个片段内操作，没有提供跨上下文对齐动作语义的机制。我们的关键洞察是，尽管动作未被观测到，但其语义效果是可观测的，可以作为共享参考。我们引入SeqΔ-REPA，一种序列级控制效果对齐目标，将集成潜在动作锚定到来自冻结自监督视频编码器的时序特征差异。基于此，我们提出Olaf-World，一个从大规模被动视频中预训练动作条件视频世界模型的流程。大量实验表明，我们的方法学习了更结构化的潜在动作空间，从而在零样本动作迁移和适应新控制接口的数据效率上优于最先进的基线方法。
英文摘要：
Scaling action-controllable world models is limited by the scarcity of action labels. While latent action learning promises to extract control interfaces from unlabeled video, learned latents often fail to transfer across contexts: they entangle scene-specific cues and lack a shared coordinate system. This occurs because standard objectives operate only within each clip, providing no mechanism to align action semantics across contexts. Our key insight is that although actions are unobserved, their semantic effects are observable and can serve as a shared reference. We introduce Seq$Δ$-REPA, a sequence-level control-effect alignment objective that anchors integrated latent action to temporal feature differences from a frozen, self-supervised video encoder. Building on this, we present Olaf-World, a pipeline that pretrains action-conditioned video world models from large-scale passive video. Extensive experiments demonstrate that our method learns a more structured latent action space, leading to stronger zero-shot action transfer and more data-efficient adaptation to new control interfaces than state-of-the-art baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
