arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.06192cs.CVcs.AIcs.RO
收藏

EA-WM:事件感知生成世界模型与结构运动-视觉动作场

EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields

Fudan University(复旦大学);Zhongguancun Academy(中关村学院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院);University of Science and Technology of China(中国科学技术大学);DeepCybo(深瞳)

Zhaoyang Yang, Yurun Jin, Lizhe Qi, Cong Huang, Kai Chen

AI总结:

EA-WM通过结构化运动-视觉动作场闭环连接运动控制与视觉感知,提升机器人空间几何和细粒度交互动态的生成精度。

摘要:

预训练视频扩散模型提供了强大的时空生成先验,使其成为机器人世界模型的自然基础。尽管最近的世界-动作模型联合优化未来视频和动作,但它们主要将视频生成作为辅助表示用于策略学习。因此,它们不足以探索逆向问题:利用动作信号指导视频合成,从而在生成的路径中往往无法精确保持机器人空间几何和细粒度机器人-物体交互动态。为弥合这一差距,我们提出了EA-WM,一种事件感知生成世界模型,能够有效闭合运动控制与视觉感知之间的闭环。而不是将关节或末端执行器动作注入为抽象的低维标记,EA-WM将动作和运动状态直接投影到目标摄像机视图中作为结构化运动-视觉动作场。为了充分利用这种基于几何的表示,我们引入了事件感知的双向融合块,调节跨分支注意力,捕捉物体状态变化和交互动态。在全面的WorldArena基准上评估,EA-WM实现了最先进的性能,显著优于现有基线。

英文摘要:

Pretrained video diffusion models provide powerful spatiotemporal generative priors, making them a natural foundation for robotic world models. While recent world-action models jointly optimize future videos and actions, they predominantly treat video generation as an auxiliary representation for policy learning. Consequently, they insufficiently explore the inverse problem: leveraging action signals to guide video synthesis, thereby often failing to preserve precise robot spatial geometry and fine-grained robot-object interaction dynamics in the generated rollouts. To bridge this gap, we present EA-WM, an Event-Aware Generative World Model that effectively closes the loop between kinematic control and visual perception. Rather than injecting joint or end-effector actions as abstract, low-dimensional tokens, EA-WM projects actions and kinematic states directly into the target camera view as Structured Kinematic-to-Visual Action Fields. To fully exploit this geometrically grounded representation, we introduce event-aware bidirectional fusion blocks that modulate cross-branch attention, capturing object state changes and interaction dynamics. Evaluated on the comprehensive WorldArena benchmark, EA-WM achieves state-of-the-art performance, outperforming existing baselines by a significant margin.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑