arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.10768cs.CV
收藏

去而复返:自监督多光谱对应估计

There and Back Again: Self-supervised Multispectral Correspondence Estimation

University of Surrey(萨里大学)

Celyn Walters, Oscar Mendez, Mark Johnson, Richard Bowden

摘要:

在从自动驾驶车辆到医学成像的广泛应用中,多光谱图像提供了提取彩色图像中不存在的附加信息的机会。使这些信息易于获得的最重要步骤之一是准确估计不同光谱之间的密集对应关系。由于交叉光谱图像的性质,视觉域的大多数对应求解技术根本不适用。此外,大多数交叉光谱技术利用光谱特定特性来执行对准。在这项工作中,我们的目标是解决密集对应估计问题的方式,推广到一个以上的频谱。我们通过引入一种新的周期一致性度量来实现这一点,该度量允许我们进行自我监督。这与我们的光谱不可知损失函数相结合,使我们能够在多个光谱上训练相同的网络。我们展示了我们的方法上的密集RGB-FIR对应估计的挑战性任务。我们还展示了未修改的网络在RGB-NIR和RGB-RGB情况下的性能,在这些情况下,我们比类似的自监督方法实现了更高的准确性。我们的工作表明,跨谱对应估计可以解决在一个共同的框架,学习概括跨谱对齐。

英文摘要:

Across a wide range of applications, from autonomous vehicles to medical imaging, multi-spectral images provide an opportunity to extract additional information not present in color images. One of the most important steps in making this information readily available is the accurate estimation of dense correspondences between different spectra. Due to the nature of cross-spectral images, most correspondence solving techniques for the visual domain are simply not applicable. Furthermore, most cross-spectral techniques utilize spectra-specific characteristics to perform the alignment. In this work, we aim to address the dense correspondence estimation problem in a way that generalizes to more than one spectrum. We do this by introducing a novel cycle-consistency metric that allows us to self-supervise. This, combined with our spectra-agnostic loss functions, allows us to train the same network across multiple spectra. We demonstrate our approach on the challenging task of dense RGB-FIR correspondence estimation. We also show the performance of our unmodified network on the cases of RGB-NIR and RGB-RGB, where we achieve higher accuracy than similar self-supervised approaches. Our work shows that cross-spectral correspondence estimation can be solved in a common framework that learns to generalize alignment across spectra.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑