Cross-task generalization remains difficult for world action models (WAMs), even when their predicted futures depict plausible manipulation. This gap can be attributed directly: we make this gap directly testable by decoupling a WAM into a world model and an inverse-dynamics model (IDM), then holding the IDM fixed while replacing the source of future images. We introduce Align Once, Evaluate Everywhere (AoE), a camera-space inverse-dynamics model pretrained on large-scale robot data, which provides this recovery interface for different future sources across tasks and environments. Through it, we find that actions recovered from recorded futures of unseen tasks execute almost as well as replayed demonstrations, whereas futures predicted by world models collapse across benchmarks. The failures are not generic: they concentrate on the gripper and interaction geometry while generation capacity is spent on task-irrelevant background. This diagnosis suggests that supervision should target exactly these regions. We therefore introduce the vision-to-action (VITA) loss, which passes generated futures through the frozen AoE and penalizes discrepancies in the recovered actions, redirecting generation toward action-relevant detail. Across two world models and three manipulation benchmarks, this simple change raises the execution success of generated futures from near zero to 47.0%. The same loss also transfers to WAMs and yields consistent gains in native control.
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Guangdong Provincial Key Laboratory of Computility Microelectronics, Faculty of Computility Microelectronics, Shenzhen University of Advanced Technology(深圳先进技术研究院计算机微科学与技术学院、广东省计算微电子重点实验室);School of Aeronautics and Astronautics, Sichuan University(四川大学航空航天学院);Suzhou Evans Intelligent Technology Co., Ltd.(苏州伊文斯智能科技有限公司);Morphi Intelligence Technology Co., Ltd.(墨飞智能科技有限公司);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);Xiaomi EV(小米汽车)
Achieve What You Imagined: Learning to Align Actions with Visual Plans
KTH(瑞典皇家理工学院);Beihang University(北京航空航天大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Peking University(北京大学);Great Bay University(大湾区大学)