arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.02486cs.RO
收藏

拦截未来:用于动态VLA操作的潜在空间预测世界模型

Intercepting the Future: Latent-Space Predictive World Model for Dynamic VLA Manipulation

Robotics Institute, Carnegie Mellon University(卡内基梅隆大学机器人研究所)

Shahram Najam Syed, Arthur Jakobsson, Haoran Hao, Jeffrey Ichnowski

AI总结:

提出AHEAD框架,通过潜在空间世界模型预测未来视觉特征,使冻结的VLA模型在动态场景中实现高成功率操作。

摘要:

视觉-语言-动作(VLA)模型在静态操作中具有泛化能力,但当物体在任务执行过程中移动时则失效。它们将当前观测映射为动作,并假设观测与执行之间场景静止,因此在任何非平凡的物体速度下,产生的延迟都会超过可用的抓取时间。我们通过AHEAD(自适应动态预期视界外推)弥补了这一差距,这是一种先预测后执行的包装器,用运动感知的潜在世界模型增强冻结的VLA。一个在操作视频上训练的小型世界模型,基于光流计算的每个令牌的速度和加速度,预测VLA特征空间中的未来块令牌。语言和运动显著性掩码将预测集中在任务相关的块上,模型向前滚动自适应视界,当预测不确定性超过阈值时停止。然后冻结的动作解码器接收预测的未来令牌代替当前令牌。AHEAD为冻结的7B OpenVLA增加了4.9M参数,在20个动态模拟场景中达到79%至97%的成功率,而最强基线仅为31%至58%。在物理UFactory xArm 7上,AHEAD在三个传送带和滚球任务中成功率为29/30至30/30,在桨叶拦截任务中为23/30,在抛射物捕捉任务中为19/30,而所有基线均为0/30。

英文摘要:

Vision-Language-Action (VLA) models generalize across static manipulation but fail when objects move during task execution. They map the current observation to an action and assume the scene is stationary between observation and execution, so at any non-trivial object speed the resulting latency exceeds the time available to grasp. We close this gap with AHEAD (Anticipatory Horizon Extrapolation with Adaptive Dynamics), a predict-then-act wrapper that augments a frozen VLA with a motion-aware latent world model. A small world model trained on manipulation video forecasts future patch tokens in the VLA's feature space, conditioned on per-token velocity and acceleration from optical flow. A language-and-motion saliency mask concentrates prediction on task-relevant patches, and the model rolls forward for an adaptive horizon, halting when prediction uncertainty crosses a threshold. The frozen action decoder then receives the predicted future tokens in place of the current ones. AHEAD adds 4.9M parameters to a frozen 7B OpenVLA and reaches 79 to 97% success across 20 dynamic simulation scenarios where the strongest baseline reaches 31 to 58%. On a physical UFactory xArm 7, AHEAD succeeds on 29/30 to 30/30 on three conveyor and rolling-ball tasks, 23/30 on paddle interception, and 19/30 on projectile catching where every baseline scores 0/30.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑