说、梦、做：学习视频世界模型以驱动指令式机器人操作
Say, Dream, and Act: Learning Video World Models for Instruction-Driven Robot Manipulation
AI总结：
本文提出一种视频条件动作框架，通过生成稳健的视频模型和对抗性蒸馏，提升机器人操作中的预测能力和空间准确性。
摘要：
机器人操作需要预测环境对动作的响应方式，但大多数现有系统缺乏这种预测能力，往往导致错误和低效率。虽然视觉-语言模型（VLMs）提供高水平的指导，但它们无法明确预测未来状态，而现有世界模型要么只能预测短期范围，要么产生在空间上不一致的帧。为了解决这些挑战，我们提出了一种快速且预测性的视频条件动作框架。我们的方法首先选择并适应一个稳健的视频生成模型，以确保可靠的未来预测，然后应用对抗性蒸馏进行快速、少步的视频生成，最后训练一个动作模型，该模型利用生成的视频和真实的观察来纠正空间误差。广泛的实验表明，我们的方法能够产生在时间上连贯、在空间上准确的视频预测，这些预测能够直接支持精确的操作，显著提高了整体一致性、空间指认能力和任务完成度。代码和模型将被发布。
英文摘要：
Robotic manipulation requires anticipating how the environment evolves in response to actions, yet most existing systems lack this predictive capability, often resulting in errors and inefficiency. While Vision-Language Models (VLMs) provide high-level guidance, they cannot explicitly forecast future states, and existing world models either predict only short horizons or produce spatially inconsistent frames. To address these challenges, we propose a framework for fast and predictive video-conditioned action. Our approach first selects and adapts a robust video generation model to ensure reliable future predictions, then applies adversarial distillation for fast, few-step video generation, and finally trains an action model that leverages both generated videos and real observations to correct spatial errors. Extensive experiments show that our method produces temporally coherent, spatially accurate video predictions that directly support precise manipulation, achieving significant improvements in embodiment consistency, spatial referring ability, and task completion over existing baselines. Codes & Models will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
