面向可泛化神经辐射场的显式对应匹配
Explicit Correspondence Matching for Generalizable Neural Radiance Fields
AI总结：
本文提出一种可泛化NeRF方法，通过显式建模跨视图对应匹配信息，利用Transformer交叉注意力提升特征匹配质量，仅需两个源视图即可实现新视角合成，并在多个评估设置中取得最先进结果。
摘要：
我们提出了一种新的可泛化NeRF方法，能够直接泛化到未见过的场景，并仅使用少至两个源视图执行新视角合成。我们方法的关键在于显式建模的对应匹配信息，从而为体积渲染中NeRF颜色和密度预测提供几何先验。显式对应匹配通过计算一个3D点在不同视图上的2D投影处采样的图像特征之间的余弦相似度来量化，能够提供关于表面几何的可靠线索。与以往对每个视图独立提取图像特征的方法不同，我们考虑通过Transformer交叉注意力建模跨视图交互，这极大地提高了特征匹配质量。我们的方法在不同的评估设置下取得了最先进的结果，实验表明学习到的余弦特征相似度与体积密度之间存在强相关性，证明了所提方法的有效性和优越性。代码和模型位于我们的项目页面：https://donydchen.github.io/matchnerf
英文摘要：
We present a new generalizable NeRF method that is able to directly generalize to new unseen scenarios and perform novel view synthesis with as few as two source views. The key to our approach lies in the explicitly modeled correspondence matching information, so as to provide the geometry prior to the prediction of NeRF color and density for volume rendering. The explicit correspondence matching is quantified with the cosine similarity between image features sampled at the 2D projections of a 3D point on different views, which is able to provide reliable cues about the surface geometry. Unlike previous methods where image features are extracted independently for each view, we consider modeling the cross-view interactions via Transformer cross-attention, which greatly improves the feature matching quality. Our method achieves state-of-the-art results on different evaluation settings, with the experiments showing a strong correlation between our learned cosine feature similarity and volume density, demonstrating the effectiveness and superiority of our proposed method. The code and model are on our project page: https://donydchen.github.io/matchnerf
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
