Prism：结合单目结构先验的半监督多视图立体匹配
Prism: Semi-Supervised Multi-View Stereo with Monocular Structure Priors
AI总结：
针对无监督MVS在真实复杂数据上性能差、合成数据训练的MVS泛化性弱的问题，提出半监督框架Prism，借助合成数据训练的单目深度先验与新型损失函数，实现性能显著提升，为多源数据训练MVS提供新思路。
摘要：
无监督多视图立体匹配（MVS）的前景在于利用大规模无标注数据集，但现有方法在复杂数据（如室内场景的手持智能手机视频）上训练时性能不佳。同时，高质量合成数据集已经可用，但在这类数据集上训练的MVS网络难以泛化到真实场景样本。为弥合这一差距，我们提出一种半监督学习框架，可同时在真实图像和渲染图像上训练，既从合成数据中获取结构先验，又确保与真实域的一致性。该框架的核心是一组新型损失函数，利用在合成数据集上训练的现有高性能单目相对深度估计器，将这种相对深度的丰富结构迁移到无标注数据的MVS预测结果中。受感知图像度量的启发，我们通过深度特征损失和多尺度统计损失对比MVS预测与单目预测结果。我们将完整框架命名为Prism，相较于现有无监督和合成监督MVS网络，该框架在定量和定性指标上均实现大幅提升。这是一个最优场景下的结果，为同时使用无标注智能手机视频和照片级真实感合成数据集训练MVS网络开辟了道路。
英文摘要：
The promise of unsupervised multi-view-stereo (MVS) is to leverage large unlabeled datasets, yet current methods underperform when training on difficult data, such as handheld smartphone videos of indoor scenes. Meanwhile, high-quality synthetic datasets are available but MVS networks trained on these datasets fail to generalize to real-world examples. To bridge this gap, we propose a semi-supervised learning framework that allows us to train on real and rendered images jointly, capturing structural priors from synthetic data while ensuring parity with the real-world domain. Central to our framework is a novel set of losses that leverages powerful existing monocular relative-depth estimators trained on the synthetic dataset, transferring the rich structure of this relative depth to the MVS predictions on unlabeled data. Inspired by perceptual image metrics, we compare the MVS and monocular predictions via a deep feature loss and a multi-scale statistical loss. Our full framework, which we call Prism, achieves large quantitative and qualitative improvements over current unsupervised and synthetic-supervised MVS networks. This is a best-case-scenario result, opening the door to using both unlabeled smartphone videos and photorealistic synthetic datasets for training MVS networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
