Elite360D：通过语义与距离感知的双投影融合实现高效360度深度估计
Elite360D: Towards Efficient 360 Depth Estimation via Semantic- and Distance-Aware Bi-Projection Fusion
AI总结：
提出Elite360D框架，通过ERP图像与ICOSAP点集的双投影双注意力融合，实现高效360度深度估计，在多个基准上超越现有方法。
摘要：
360度深度估计因其全方位视场（FoV）在三维重建中近来受到广泛关注。现有方法主要集中于基于几何重投影的跨投影融合：它们将360度图像与等距柱状投影（ERP）及另一种投影类型（如立方体投影）融合，以ERP格式估计深度。然而，这些方法存在两个问题：1）局部感受野有限，难以捕捉大视场场景；2）由于复杂的跨投影融合模块设计，计算成本过高。本文提出Elite360D，一种新颖框架，输入ERP图像和二十面体投影（ICOSAP）点集，该点集无畸变且空间连续。Elite360D在从局部到全局视角学习表示方面具有优越能力。通过灵活的ERP图像编码器，它包含一个ICOSAP点编码器和一个双投影双注意力融合（B2F）模块（总计约1M参数）。具体而言，ERP图像编码器可接受多种基于透视图像训练的骨干网络（如ResNet、Transformer）以提取局部特征。点编码器从ICOSAP中提取全局特征。然后，B2F模块捕获ERP特征每个像素与整个ICOSAP特征集之间的语义和距离感知依赖关系。无需特定骨干网络设计且无明显计算成本增加，Elite360D在多个基准数据集上优于先前方法。
英文摘要：
360 depth estimation has recently received great attention for 3D reconstruction owing to its omnidirectional field of view (FoV). Recent approaches are predominantly focused on cross-projection fusion with geometry-based re-projection: they fuse 360 images with equirectangular projection (ERP) and another projection type, e.g., cubemap projection to estimate depth with the ERP format. However, these methods suffer from 1) limited local receptive fields, making it hardly possible to capture large FoV scenes, and 2) prohibitive computational cost, caused by the complex cross-projection fusion module design. In this paper, we propose Elite360D, a novel framework that inputs the ERP image and icosahedron projection (ICOSAP) point set, which is undistorted and spatially continuous. Elite360D is superior in its capacity in learning a representation from a local-with-global perspective. With a flexible ERP image encoder, it includes an ICOSAP point encoder, and a Bi-projection Bi-attention Fusion (B2F) module (totally ~1M parameters). Specifically, the ERP image encoder can take various perspective image-trained backbones (e.g., ResNet, Transformer) to extract local features. The point encoder extracts the global features from the ICOSAP. Then, the B2F module captures the semantic- and distance-aware dependencies between each pixel of the ERP feature and the entire ICOSAP feature set. Without specific backbone design and obvious computational cost increase, Elite360D outperforms the prior arts on several benchmark datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
