TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion
Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);State Key Laboratory of Multimedia Information Processing(多媒体信息处理国家重点实验室);School of Computer Science, Peking University(北京大学计算机科学学院)
The vision-language-action (VLA) paradigm has enabled powerful robotic control by leveraging vision-language models, but its reliance on large-scale, high-quality robot data limits its generalization. Generative world models offer a promising alternative for general-purpose embodied AI, yet a critical gap remains between their pixel-level plans and physically executable actions.
To this end, we propose the Tool-Centric Inverse Dynamics Model (TC-IDM). By focusing on the tool's imagined trajectory as synthesized by the world model, TC-IDM establishes a robust intermediate representation that bridges the gap between visual planning and physical control.
TC-IDM extracts the tool's point cloud trajectories via segmentation and 3D motion estimation from generated videos. Considering diverse tool attributes, our architecture employs decoupled action heads to project these planned trajectories into 6-DoF end-effector motions and corresponding control signals.
This plan-and-translate paradigm not only supports a wide range of end-effectors but also significantly improves viewpoint invariance. Furthermore, it exhibits strong generalization capabilities across long-horizon and out-of-distribution tasks, including interacting with deformable objects.
In real-world evaluations, the world model with TC-IDM achieves an average success rate of 61.11 percent, with 77.7 percent on simple tasks and 38.46 percent on zero-shot deformable object tasks. It substantially outperforms end-to-end VLA-style baselines and other inverse dynamics models.
KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding
Institute of Artificial Intelligence (TeleAI), China Telecom(中国电信人工智能研究院);National University of Singapore(新加坡国立大学);Fudan University(复旦大学);Tsinghua University(清华大学);Shenzhen Research Institute of Northwestern Polytechnical University(西北工业大学深圳研究院)
EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation
College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院智能信息处理上海市重点实验室);TeleAI, China Telecom(中国电信天翼人工智能公司)
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);Weiyang College, Tsinghua University(清华大学未央学院);Shanghai Qi Zhi Institute(上海期智研究院)
GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving
National Engineering Research Center of Electric Vehicles, Beijing Institute of Technology(北京理工大学电动车辆国家工程研究中心);Shenzhen Automotive Research Institute, Beijing Institute of Technology(北京理工大学深圳汽车研究院);Shenzhen Jiguangzhijie Technology Co., Ltd.(深圳极光智界科技有限公司);Nanyang Technological University(南洋理工大学)
HaReCAP: Habitual-action Grounding for Recursive Large Language Model Agents
North China Institute of Computer System Engineering(华北计算机系统工程研究所);University of Science and Technology of China(中国科学技术大学);China Information Security Research Institute Co., Ltd.(中国信息安全研究院有限公司)
GIFT: Guided Intermediate Feature Training via Action-Oriented Structural Supervision for Robotic Manipulation
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);National University of Singapore(新加坡国立大学);Tsinghua University(清华大学);Fudan University(复旦大学)
该研究提出 GIFT 框架,通过几何对齐、可供性预测和目标区域重建约束中间特征,在 LIBERO-Plus 和 RoboCasa 数据集上,其多个变体在机器人操作任务中显著优于基准模型,性能提升明显。
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全国家重点实验室);National Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所模式识别国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);XYZ Embodied AI(XYZ具身人工智能)
GenVid2Robot: From Video Generation to Robot Manipulation via Rigid-Geometric Consistency
School of Automation, Guangdong University of Technology(广东工业大学自动化学院);University of Liverpool(利物浦大学);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系);State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University(上海交通大学海洋地球科学国家重点实验室,自动化与智能感知学院)
ACE: Agentic Control for Embodied Manipulation via Zero-shot Workflow Reasoning
Department of Computer Science, Tsinghua University(清华大学计算机科学系);National College for Excellent Engineers, Beihang University(北京航空航天大学卓越工程师学院);Wuxi Dexteroushands Robotic Technology Co.(无锡灵犀机器人技术有限公司)
Translating Flow to Policy via Hindsight Online Imitation
Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院);University of California San Diego(加州大学圣地亚哥分校);Washington University in St. Louis(圣路易斯华盛顿大学);Shanghai Jiao Tong University(上海交通大学);Shanghai Qi Zhi Institute(上海启智研究所)
V-CAGE: Context-Aware Generation and Verification for Scalable Long-Horizon Embodied Tasks
Department of Computer Science and Technology, University of Cambridge, Cambridge, England.(计算机科学与技术系,剑桥大学,剑桥,英格兰);Shanghai Jiao Tong University, Shanghai, China(上海交通大学,上海,中国);Wuhan University, Wuhan, Hubei, China(武汉大学,武汉,湖北,中国)
GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
Harvard AI and Robotics Lab(哈佛人工智能与机器人实验室);Harvard University(哈佛大学);Media Lab and EECS(媒体实验室和电子工程与计算机科学系);MIT(麻省理工学院);Princeton University(普林斯顿大学);MIT-IBM Watson AI Lab(麻省理工-IBM沃森人工智能实验室)