发表机构
Chalmers University of Technology; Mobile Perception Lab, ShanghaiTech University; LIGM, Ecole des Ponts, Univ. Gustave Eiffel, CNRS（查尔姆斯理工大学; 上海科技大学移动感知实验室; LIGM，巴黎高科路桥学校，古斯塔夫·埃菲尔大学，法国国家科学研究中心）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文探究前馈3D模型对图像匹配的认知，发现其表示可用于线性探测和完整匹配，并基于VGGT-$\Omega$重训RoMa v2，在多个基准上超越现有匹配器。
AI 中文摘要
近年来，学习式图像匹配取得了显著进展，最终出现了如RoMa这样鲁棒且精确的匹配器，其鲁棒性通常归因于使用冻结的DINO特征。与此同时，前馈重建模型（如VGGT）在日益增长的数据集上训练，以准确回归密集3D点图和相机位姿。随着MASt3R和VGGT-$\Omega$等模型中引入匹配损失，匹配器与前馈重建模型之间的区别变得越来越模糊。这引发了一个自然的问题：前馈3D模型对图像匹配了解多少？在这项工作中，我们通过分析三种场景来回答这个问题：（i）补丁特征的零样本匹配，（ii）3D点预测的直接匹配，以及（iii）在学习表示之上训练完整的匹配器。我们发现，尽管在零样本匹配中表现不佳（尤其是在较深层），前馈重建模型为线性探测和完整匹配流程提供了强大的表示。我们进一步表明，即使没有任何训练，它们的原始预测单独也能实现有竞争力的匹配，但仅在适度的视角变化和模态差距下。基于这些见解，我们通过用VGGT-$\Omega$替换其DINO骨干网络来重新训练RoMa v2。我们得到的模型\ours在广泛的基准测试中优于最先进的匹配器，例如在WxBS上相比RoMa v2提高了+8.1 mAA。
英文摘要
Learned image matching has experienced significant progress in recent years, culminating in robust and accurate matchers such as RoMa, whose robustness is often attributed to its use of frozen DINO features. In a parallel development, feed-forward reconstruction models, such as VGGT, have been trained on ever-growing datasets to accurately regress dense 3D point maps and camera poses. The distinction between matchers and feed-forward reconstruction models has become increasingly blurred with the introduction of matching losses in models such as MASt3R and VGGT-$Ω$. This raises a natural question: what do feed-forward 3D models know about image matching? In this work, we answer this question by analyzing three scenarios: (i) zero-shot matching of patch features, (ii) direct matching of 3D point predictions, and (iii) training a full matcher on top of the learned representations. We find that, despite performing poorly in zero-shot matching, especially in later layers, feed-forward reconstruction models provide strong representations for linear probing and full matching pipelines. We further show that, even without any training, their raw predictions alone enable competitive matching, albeit only under moderate viewpoint changes and modality gaps. Based on these insights, we retrain RoMa v2 by replacing its DINO backbone with VGGT-$Ω$. Our resulting model, \ours, outperforms state-of-the-art matchers on a wide range of benchmarks, e.g. +8.1 mAA compared to RoMa v2 on WxBS.
