arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.01653cs.CV
收藏

MCPDepth:基于多圆柱全景图的立体匹配全方位深度估计

MCPDepth: Omnidirectional Depth Estimation via Stereo Matching from Multi-Cylindrical Panoramas

Washington University in St. Louis(华盛顿大学圣路易斯分校);The Chinese University of Hong Kong(香港中文大学);ShanghaiTech University(上海科技大学);University of California, Los Angeles(加州大学洛杉矶分校)

Feng Qiao, Zhexiao Xiong, Xinge Zhu, Yuexin Ma, Qiumeng He, Nathan Jacobs

AI总结:

提出MCPDepth两阶段框架,通过圆柱全景图的立体匹配和融合,利用循环注意力模块处理垂直畸变,在标准网络组件上实现高效的全方位深度估计,在Deep360和3D60数据集上MAE分别降低18.8%和19.9%。

摘要:

全方位深度估计由于全景图像固有的畸变而面临重大挑战。尽管取得了显著进展,但投影方法的影响仍未得到充分探索。我们引入了多圆柱全景深度估计(MCPDepth),这是一种新颖的两阶段框架,旨在通过多个圆柱全景图之间的立体匹配来增强全方位深度估计。MCPDepth首先使用圆柱全景图进行立体匹配,然后对不同视图得到的深度图进行鲁棒融合。与现有方法依赖定制内核来处理畸变不同,MCPDepth利用标准网络组件,便于在嵌入式设备上无缝部署,同时提供卓越的性能。为了有效处理圆柱全景图中的垂直畸变,MCPDepth结合了循环注意力模块,显著扩展了传统卷积的感受野。我们对常见的全景投影——球面、圆柱和立方体——进行了全面的理论和实验分析,证明了圆柱投影的优越性。我们的方法在室外数据集Deep360上将平均绝对误差(MAE)降低了18.8%,在真实数据集3D60上降低了19.9%。这项工作为其他任务和实际应用提供了实用见解,建立了全方位深度估计的新范式。代码可在https://github.com/Qjizhi/MCPDepth获取。

英文摘要:

Omnidirectional depth estimation presents a significant challenge due to the inherent distortions in panoramic images. Despite notable advancements, the impact of projection methods remains underexplored. We introduce Multi-Cylindrical Panoramic Depth Estimation (MCPDepth), a novel two-stage framework designed to enhance omnidirectional depth estimation through stereo matching across multiple cylindrical panoramas. MCPDepth initially performs stereo matching using cylindrical panoramas, followed by a robust fusion of the resulting depth maps from different views. Unlike existing methods that rely on customized kernels to address distortions, MCPDepth utilizes standard network components, facilitating seamless deployment on embedded devices while delivering exceptional performance. To effectively address vertical distortions in cylindrical panoramas, MCPDepth incorporates a circular attention module, significantly expanding the receptive field beyond traditional convolutions. We provide a comprehensive theoretical and experimental analysis of common panoramic projections-spherical, cylindrical, and cubic-demonstrating the superior efficacy of cylindrical projection. Our method improves the mean absolute error (MAE) by 18.8% on the outdoor dataset Deep360 and by 19.9% on the real dataset 3D60. This work offers practical insights for other tasks and real-world applications, establishing a new paradigm in omnidirectional depth estimation. The code is available at https://github.com/Qjizhi/MCPDepth.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑