MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation
MMLab, The Chinese University of Hong Kong, Hong Kong SAR(香港理工大学MMLab,香港中文大学,香港特别行政区);The Hong Kong University of Science(香港理工大学);The University of Hong Kong(香港大学);Tsinghua University(清华大学)
World-model-based imagine-then-act becomes a promising paradigm for robotic manipulation, yet existing approaches typically support either purely image-based forecasting or reasoning over partial 3D geometry, limiting their ability to predict complete 4D scene dynamics. This work proposes a novel embodied 4D world model that enables geometrically consistent, arbitrary-view RGBD generation: given only a single-view RGBD observation as input, the model imagines the remaining viewpoints, which can then be back-projected and fused to assemble a more complete 3D structure across time. To efficiently learn the multi-view, cross-modality generation, we explicitly design cross-view and cross-modality feature fusion that jointly encourage consistency between RGB and depth and enforce geometric alignment across views. Beyond prediction, converting generated futures into actions is often handled by inverse dynamics, which is ill-posed because multiple actions can explain the same transition. We address this with a test-time action optimization strategy that backpropagates through the generative model to infer a trajectory-level latent best matching the predicted future, and a residual inverse dynamics model that turns this trajectory prior into accurate executable actions. Experiments on three datasets demonstrate strong performance on both 4D scene generation and downstream manipulation, and ablations provide practical insights into the key design choices.
VLA-4D: Embedding 4D Awareness into Vision-Language-Action Models for SpatioTemporally Coherent Robotic Manipulation
School of Computing, National University of Singapore(新加坡国立大学计算机学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院)
TabletopGen: Tabletop Scene Generation and Interactive Simulation for Robotic Manipulation
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(中国科学院多模态人工智能系统国家重点实验室);D-Robotics;Horizon Robotics
End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representation
A*STAR Institute of High Performance Computing (IHPC)(新加坡科技研究局高性能计算研究所);A*STAR Institute of Information Research (IR)(新加坡科技研究局信息研究所);National University of Singapore (NUS)(新加坡国立大学);Alibaba DAMO Academy(阿里巴巴达摩院)
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(北京大学计算机学院多媒体信息处理国家重点实验室);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);The University of Hong Kong(香港大学);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Contrastive Imitation Learning for Language-guided Multi-Task Robotic Manipulation
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)AI Thrust);The Hong Kong University of Science and Technology(香港科技大学)
SKT: Integrating State-Aware Keypoint Trajectories with Vision-Language Models for Robotic Garment Manipulation
Shanghai Jiao Tong University(上海交通大学);The Hong Kong University of Science and Technology(香港科技大学);National University of Singapore(新加坡国立大学);Yuandao AI(元岛人工智能);Shanghai AI Lab(上海人工智能实验室)
DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence(人机混合增强智能国家重点实验室);National Engineering Research Center for Visual Information and Applications(视觉信息与应用国家工程研究中心);Institute of Artificial Intelligence and Robotics(人工智能与机器人研究院);Xi’an Jiaotong University(西安交通大学)
Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets
University of California, San Diego(加利福尼亚大学圣迭戈分校);Tongji University(同济大学);Shanghai Jiao Tong University(上海交通大学);University of Maryland, College Park(马里兰大学帕克分校);Tsinghua University(清华大学)
MQA: Answering the Question via Robotic Manipulation
Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Tsinghua University(清华大学)
Learning Reward Functions for Robotic Manipulation by Observing Humans
CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);Univ. Grenoble Alpes(格勒诺布尔阿尔卑斯大学);Grenoble INP(格勒诺布尔国立理工学院);Google Research(谷歌研究院);Center for Data Science, New York University(纽约大学数据科学中心)