Veo-Act：前沿视频模型在通用机器人操作中能走多远？
Veo-Act: Enhancing VLA Policies with Frontier Video Models
AI总结：
本文探讨前沿视频模型如Veo-3在通用机器人操作中的应用，提出Veo-Act框架结合高阶运动规划与低阶执行器，提升指令跟随性能。
摘要：
视频生成模型已迅速发展并开始展现对物理动态的强理解。本文研究了先进视频生成模型如Veo-3在通用机器人操作中的潜力。我们首先研究了一种零样本方法，其中Veo-3根据当前机器人观测预测未来图像序列，同时逆动力学模型IDM恢复对应的机器人动作。IDM仅在随机播放数据上训练，无需人工监督或专家示范。关键思想是，如果视频模型能生成在图像空间中物理合理的未来动作，IDM可以将这些视觉轨迹转换为可执行的机器人动作。我们通过高维灵巧手在仿真和现实世界中评估这一
英文摘要：
Video generation models can produce coherent vi- sual sequences depicting object motion and interactions. We in- vestigate how frontier video generation models can complement vision-language-action policies to enhance generalizable robotic manipulation. VLA policies have become a dominant paradigm for robot learning, but their action-oriented adaptation of pretrained VLMs can weaken semantic generalization, limiting robustness in ambiguous or out-of-distribution manipulation scenarios. We use video models as visual planners, motivated by their potential to generalize across complex scenes and their priors over hand motion. However, manipulation methods based on video models often lack the precision and temporal responsiveness needed for low-level dexterous interaction. To address this gap, we present Veo-Act, a hierarchical framework with Veo-3.1 as a high-level motion planner and a VLA policy as the low-level executor. A multi-head inverse dynamics model converts generated frame pairs into actions and learns an interaction gate to trigger the handoff to reactive VLA control. Experiments in simulation and on a real robot show improved instruction following and overall task success over the baseline VLA in novel and semantically complex manipulation settings, supporting the complementary roles of video planning and reactive interaction.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
