Video-based world models have recently garnered increasing attention for their ability to synthesize diverse and dynamic visual environments. In this paper, we focus on shared world modeling, where a model generates multiple videos from a set of input images, each representing the same underlying world in different camera poses. We propose IC-World, a novel generation framework, enabling parallel generation for all input images via activating the inherent in-context generation capability of large video models. We further finetune IC-World via reinforcement learning, Group Relative Policy Optimization, together with two proposed novel reward models to enforce scene-level geometry consistency and object-level motion consistency among the set of generated videos. Extensive experiments demonstrate that IC-World substantially outperforms state-of-the-art methods in both geometry and motion consistency. To the best of our knowledge, this is the first work to systematically explore the shared world modeling problem with video-based world models.
King Abdullah University of Science and Technology (KAUST)(阿卜杜拉国王科技大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Beijing Zhongguancun Academy(北京中关村学院);Zhongguancun Institute of Artificial Intelligence(中关村人工智能研究院)
DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers
Centre for Artificial Intelligence and Robotics, HKISI, CAS(中国科学院香港中文大学信息工程中心人工智能与机器人中心);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update
University of Oxford(牛津大学);University of Cambridge(剑桥大学);University of Twente(特文特大学);National University of Singapore(新加坡国立大学);University of Bath(巴斯大学)
SGAM: Shared Gaussian Geometry with Implicit Amplitude Modeling for Scan-Specific 3D Multi-Contrast MRI Reconstruction
Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院劳特伯生物医学成像研究中心)