SurfaceNet +：一种用于非常稀疏的多视图立体视觉的端到端3D神经网络
SurfaceNet+: An End-to-end 3D Neural Network for Very Sparse Multi-view Stereopsis
摘要：
多视点立体视觉（MVS）试图从2D图像恢复3D模型。随着观测变得稀疏，显著的3D信息损失使得MVS问题更具挑战性。而不是只专注于密集采样的条件下，我们调查稀疏MVS与大基线角度，因为稀疏的感觉是更实用，更具有成本效益。通过研究各种观测稀疏性，我们表明，经典的深度融合流水线变得无能为力的情况下，具有较大的基线角度，这是照片的一致性检查。作为解决方案的另一条线，我们提出了SurfaceNet+，一种体积方法来处理由非常稀疏的MVS设置引起的“不完整性”和“不准确性”问题。具体来说，前一个问题是由一种新的体积明智的视图选择方法来处理。该算法考虑了几何先验知识，在剔除无效遮挡视图的同时选择有效视图方面具有优越性。此外，后者的问题是通过一个多尺度的策略，从而细化周围的区域与重复模式的恢复几何处理。实验证明了SurfaceNet+和最先进的方法在精确度和召回率方面的巨大性能差距。在两个数据集中的极端稀疏MVS设置下，现有方法只能返回很少的点，SurfaceNet+仍然可以在密集MVS设置下工作。该基准和实施可在https://github.com/mjiUST/SurfaceNet-plus上公开获取。
英文摘要：
Multi-view stereopsis (MVS) tries to recover the 3D model from 2D images. As the observations become sparser, the significant 3D information loss makes the MVS problem more challenging. Instead of only focusing on densely sampled conditions, we investigate sparse-MVS with large baseline angles since the sparser sensation is more practical and more cost-efficient. By investigating various observation sparsities, we show that the classical depth-fusion pipeline becomes powerless for the case with a larger baseline angle that worsens the photo-consistency check. As another line of the solution, we present SurfaceNet+, a volumetric method to handle the 'incompleteness' and the 'inaccuracy' problems induced by a very sparse MVS setup. Specifically, the former problem is handled by a novel volume-wise view selection approach. It owns superiority in selecting valid views while discarding invalid occluded views by considering the geometric prior. Furthermore, the latter problem is handled via a multi-scale strategy that consequently refines the recovered geometry around the region with the repeating pattern. The experiments demonstrate the tremendous performance gap between SurfaceNet+ and state-of-the-art methods in terms of precision and recall. Under the extreme sparse-MVS settings in two datasets, where existing methods can only return very few points, SurfaceNet+ still works as well as in the dense MVS setting. The benchmark and the implementation are publicly available at https://github.com/mjiUST/SurfaceNet-plus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
