Comprehensive panoramic scene understanding is critical for immersive applications, yet it remains challenging due to the scarcity of high-resolution, multi-task annotations. While perspective foundation models have achieved success through data scaling, directly adapting them to the panoramic domain often fails due to severe geometric distortions and coordinate system discrepancies. Furthermore, the underlying relations between diverse dense prediction tasks in spherical spaces are underexplored. To address these challenges, we propose MTPano, a robust multi-task panoramic foundation model established by a label-free training pipeline. First, to circumvent data scarcity, we leverage powerful perspective dense priors. We project panoramic images into perspective patches to generate accurate, domain-gap-free pseudo-labels using off-the-shelf foundation models, which are then re-projected to serve as patch-wise supervision. Second, to tackle the interference between task types, we categorize tasks into rotation-invariant (e.g., depth, segmentation) and rotation-variant (e.g., surface normals) groups. We introduce the Panoramic Dual BridgeNet, which disentangles these feature streams via geometry-aware modulation layers that inject absolute position and ray direction priors. To handle the distortion from equirectangular projections (ERP), we incorporate ERP token mixers followed by a dual-branch BridgeNet for interactions with gradient truncation, facilitating beneficial cross-task information sharing while blocking conflicting gradients from incompatible task attributes. Additionally, we introduce auxiliary tasks to fertilize the cross-task learning process. Extensive experiments demonstrate that MTPano achieves state-of-the-art performance on multiple benchmarks and delivers competitive results against task-specific panoramic specialist foundation models.
HoloTime: Taming Video Diffusion Models for Panoramic 4D Scene Generation
School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Peng Cheng Laboratory(鹏城实验室);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳))
Underwater360: Reconstructing Underwater Scenes from Panoramic Images with Omnidirectional Gaussian Splatting
School of Software, Dalian University of Technology(大连理工大学软件学院);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
OmniRoam: World Wandering via Long-Horizon Panoramic Video Generation
University of California, Irvine(加州大学尔湾分校);University of California, San Diego(加州大学圣地亚哥分校);City University of Hong Kong(香港城市大学);University of Pennsylvania(宾夕法尼亚大学);Adobe Research(Adobe研究院)
Panoramic Depth Estimation via Supervised and Unsupervised Learning in Indoor Scenes
Zhejiang University(浙江大学);Karlsruhe Institute of Technology(卡尔斯鲁厄理工学院);National Engineering Research Center of Optical Instrumentation, Zhejiang University(浙江大学国家光学仪器工程技术研究中心)
Aerial-PASS: Panoramic Annular Scene Segmentation in Drone Videos
State Key Laboratory of Optical Instrumentation, Zhejiang University, China(浙江大学光学仪器国家重点实验室);Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany(卡尔斯鲁厄理工学院人机与机器人研究所)
You Need a Transition Plane: Bridging Continuous Panoramic 3D Reconstruction with Perspective Gaussian Splatting
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学研究所);Visual Intelligence +X International Cooperation Joint Laboratory of MOE(教育部视觉智能+X国际合作联合实验室);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Geometry and Gradient-based Partitioning for Panoramic Outdoor Reconstruction
Insta360 Research(Insta360研究机构);Sun Yat-sen University(中山大学);South China University of Technology(华南理工大学);University of Chinese Academy of Sciences(中国科学院大学);Harbin Engineering University(哈尔滨工程大学);Wuhan University(武汉大学)
What's in my Room? Object Recognition on Indoor Panoramic Images
Université Bourgogne Franche-Comté(勃艮第-弗朗什-孔泰大学);Instituto de Investigación en Ingeniería de Aragón (I3A)(阿拉贡工程研究院(I3A));Universidad de Zaragoza(萨拉戈萨大学)
LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);Stanford University(斯坦福大学);Nanyang Technological University(南洋理工大学)
ExScene: Free-View 3D Scene Reconstruction with Gaussian Splatting from a Single Image
Shenzhen Future Network of Intelligence Institute(深圳未来智能网络研究院);School of Science and Engineering, The Chinese University of Hongkong, Shenzhen(香港中文大学(深圳)理工学院)
PanopticNeRF-360: Panoramic 3D-to-2D Label Transfer in Urban Scenes
Zhejiang University, China(浙江大学);Autonomous Vision Group (AVG) at the University of Tübingen and Tübingen AI Center, Germany(图宾根大学自主视觉组(AVG)及图宾根人工智能中心)
Hallucinating 360°: Panoramic Street-View Generation via Local Scenes Diffusion and Probabilistic Prompting
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(人工智能与机器人学院和机器人视觉感知与控制技术国家工程研究中心,湖南大学);Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology(人机学与机器人研究所,卡尔斯鲁厄技术大学)
Matrix-3D: Omnidirectional Explorable 3D World Generation
Skywork AI;Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
CM-EVS: Sparse Panoramic RGB-D-Pose Data for Complete Scene Coverage
Zhejiang University(浙江大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong University of Science and Technology(香港科学与技术大学);Vorynel(沃伊内尔);Xinjiang University(新疆大学);Wuhan Polytechnic University(武汉职业技术学院);Tianjin University(天津大学)