Imagine360: Immersive 360 Video Generation from Perspective Anchor
The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Nanyang Technological University(南洋理工大学)
$360^\circ$ videos offer a hyper-immersive experience that allows the viewers to explore a dynamic scene from full 360 degrees. To achieve more user-friendly and personalized content creation in $360^\circ$ video format, we seek to lift standard perspective videos into $360^\circ$ equirectangular videos. To this end, we introduce Imagine360, the first perspective-to-$360^\circ$ video generation framework that creates high-quality $360^\circ$ videos with rich and diverse motion patterns from video anchors. Imagine360 learns fine-grained spherical visual and motion patterns from limited $360^\circ$ video data with several key designs. 1) Firstly we adopt the dual-branch design, including a perspective and a panorama video denoising branch to provide local and global constraints for $360^\circ$ video generation, with motion module and spatial LoRA layers fine-tuned on extended web $360^\circ$ videos. 2) Additionally, an antipodal mask is devised to capture long-range motion dependencies, enhancing the reversed camera motion between antipodal pixels across hemispheres. 3) To handle diverse perspective video inputs, we propose elevation-aware designs that adapt to varying video masking due to changing elevations across frames. Extensive experiments show Imagine360 achieves superior graphics quality and motion coherence among state-of-the-art $360^\circ$ video generation methods. We believe Imagine360 holds promise for advancing personalized, immersive $360^\circ$ video creation.
SoLAR: Error-Resilient Streamable Long-Horizon Free-Viewpoint Video Reconstruction with Anchor Activation and Latent Recalibration
National Engineering Laboratory for Video Technology, School of Computer Science, Peking University(国家视频技术工程实验室,计算机科学学院,北京大学);School of Computer Science, Jilin University(吉林大学计算机科学学院);Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学)
TiP4GEN: Text to Immersive Panorama 4D Scene Generation
Institute of Information Science, Beijing Jiaotong University(信息科学研究院,北京交通大学);Visual Intelligence + X International Joint Laboratory(视觉智能+X国际联合实验室);University of Toronto(多伦多大学);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
OmniView-Space: Reinforcing Spatial Reasoning via Multi-Perspective Spatial Mapping
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(厦门大学多媒体可信感知与高效计算教育部重点实验室);Tencent Youtu Lab(腾讯优图实验室);Beijing Institute of Technology(北京理工大学)
Exploring Cognitive Paradoxes in Video Games: A Quantum Mechanical Perspective
Artificial Intelligence and Cyber Futures Institute, Charles Sturt University(查尔斯斯特大学人工智能与网络未来研究所);The Alan Turing Institute(阿兰·图灵研究所);The University of Sydney Business School(悉尼大学商学院)