MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas
Washington University in St. Louis(华盛顿大学圣路易斯分校);The Chinese University of Hong Kong(香港中文大学);ShanghaiTech University(上海科技大学);University of California, Los Angeles(加州大学洛杉矶分校)
Omnidirectional depth estimation presents a significant challenge due to the inherent distortions in panoramic images. Despite notable advancements, the impact of projection methods remains underexplored. We introduce Multi-Cylindrical Panoramic Depth Estimation (MCPDepth), a novel two-stage framework designed to enhance omnidirectional depth estimation through stereo matching across multiple cylindrical panoramas. MCPDepth initially performs stereo matching using cylindrical panoramas, followed by a robust fusion of the resulting depth maps from different views. Unlike existing methods that rely on customized kernels to address distortions, MCPDepth utilizes standard network components, facilitating seamless deployment on embedded devices while delivering exceptional performance. To effectively address vertical distortions in cylindrical panoramas, MCPDepth incorporates a circular attention module, significantly expanding the receptive field beyond traditional convolutions. We provide a comprehensive theoretical and experimental analysis of common panoramic projections-spherical, cylindrical, and cubic-demonstrating the superior efficacy of cylindrical projection. Our method improves the mean absolute error (MAE) by 18.8% on the outdoor dataset Deep360 and by 19.9% on the real dataset 3D60. This work offers practical insights for other tasks and real-world applications, establishing a new paradigm in omnidirectional depth estimation. The code is available at https://github.com/Qjizhi/MCPDepth.
Distortion-aware Monocular Depth Estimation for Omnidirectional Images
School of Electronics and Communication Engineering, Sun Yat-sen University(中山大学电子与通信工程学院);University of Western Australia(西澳大利亚大学);Alibaba Group(阿里巴巴集团)
Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Chongqing University(重庆大学);Air Force Engineering University(空军工程大学)
LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Zhejiang University(浙江大学);Stanford University(斯坦福大学);Nanyang Technological University(南洋理工大学)
Spherical-GOF: Geometry-Aware Panoramic Gaussian Opacity Fields for 3D Scene Reconstruction
School of Artificial Intelligence and Robotics, Hunan University(人工智能与机器人学院,湖南大学);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(机器人视觉感知与控制技术国家工程研究中心,湖南大学)
Matrix-3D: Omnidirectional Explorable 3D World Generation
Skywork AI;Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);School of Artificial Intelligence, Beijing Normal University(北京师范大学人工智能学院)
UniTrackPLA: Unified Panorama-Language-Action Model for Instruction-Guided Navigation and Dynamic Person Tracking
College of Integrated Circuits, Hunan University(湖南大学集成电路学院);National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University(湖南大学机器人视觉感知与控制技术国家工程研究中心);School of Advanced Technology, Xi’an Jiaotong-Liverpool University(西交利物浦大学智能工程学院);Suzhou VSDeep Intelligent Technology Co., Ltd.(苏州维深智能科技有限公司)