Multiview stereo aims to reconstruct scene depth from images acquired by a camera under arbitrary motion. Recent methods address this problem through deep learning, which can utilize semantic cues to deal with challenges such as textureless and reflective regions. In this paper, we present a convolutional neural network called DPSNet (Deep Plane Sweep Network) whose design is inspired by best practices of traditional geometry-based approaches for dense depth reconstruction. Rather than directly estimating depth and/or optical flow correspondence from image pairs as done in many previous deep learning methods, DPSNet takes a plane sweep approach that involves building a cost volume from deep features using the plane sweep algorithm, regularizing the cost volume via a context-aware cost aggregation, and regressing the dense depth map from the cost volume. The cost volume is constructed using a differentiable warping process that allows for end-to-end training of the network. Through the effective incorporation of conventional multiview stereo concepts within a deep learning framework, DPSNet achieves state-of-the-art reconstruction results on a variety of challenging datasets.
本文提出了一种基于长短期记忆（LSTM）的自适应聚合递归多视图立体视觉网络，即AA-RMVSNet。我们首先引入了一个视图内聚合模块，通过使用上下文感知卷积和多尺度聚合自适应地提取图像特征，有效地提高了具有挑战性的区域，如薄的物体和大的低纹理表面的性能。为了克服复杂场景中变化遮挡的困难，我们提出了一个视图间成本体积聚合模块，用于自适应像素视图聚合，该模块能够在所有视图之间保持更好的匹配对。提出的两个自适应聚合模块是轻量级的，有效的和互补的提高三维重建的准确性和完整性。代替传统的3D CNN，我们利用具有递归结构的混合网络进行成本体积正则化，这允许高分辨率重建和更精细的假设平面扫描。所提出的网络是端到端训练的，在各种数据集上都取得了优异的性能。该算法在Tanks and Temples基准测试中对$1^{st}$进行了排序，并在DTU数据集上取得了较好的结果，具有较强的泛化能力和鲁棒性。我们的方法的实现可在https://github.com/QT-Zhu/AA-RMVSNet上获得。
FaSS-MVS -- Fast Multi-View Stereo with Surface-Aware Semi-Global Matching from UAV-borne Monocular Imagery
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB)（弗劳恩霍夫光学技术、系统技术与图像利用研究所）；Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院）
