第三种对应关系：基于物体反射的相机位姿估计
Correspondences of the Third Kind: Camera Pose Estimation from Object Reflection
AI总结：
本文提出反射对应关系这一新概念，通过神经估计器和RANSAC算法联合利用像素、三维及反射对应关系，仅凭物体外观实现鲁棒的相机位姿与形状估计，无需依赖背景。
摘要：
计算机视觉长期以来依赖两种对应关系：图像中的像素对应关系和物体表面上的三维对应关系。是否还存在另一种对应关系？如果存在，它们能为我们做什么？在本文中，我们引入了第三种对应关系，称之为反射对应关系，并证明它们可以在不依赖背景的情况下，仅通过观察物体来帮助估计相机位姿。反射对应关系是反射世界中的点对应关系，即由物体表面反射的场景。物体几何形状和反射率分别从几何和辐射度量上改变了场景，导致错误的像素对应关系。从每幅图像中恢复的几何形状也受到畸变的干扰，即广义浅浮雕歧义，导致错误的三维对应关系。我们证明反射对应关系可以解决由这些畸变引起的歧义。我们引入了一个神经对应关系估计器和一个RANSAC算法，该算法充分利用所有三种对应关系，仅从物体外观实现鲁棒且准确的联合相机位姿和物体形状估计。该方法拓展了众多下游任务的视野，包括用于外观建模（例如NeRF）的相机位姿估计和反射物体（例如道路上的汽车）的运动估计等，因为它放宽了对重叠背景的要求。
英文摘要：
Computer vision has long relied on two kinds of correspondences: pixel correspondences in images and 3D correspondences on object surfaces. Is there another kind, and if there is, what can they do for us? In this paper, we introduce correspondences of the third kind we call reflection correspondences and show that they can help estimate camera pose by just looking at objects without relying on the background. Reflection correspondences are point correspondences in the reflected world, i.e., the scene reflected by the object surface. The object geometry and reflectance alters the scene geometrically and radiometrically, respectively, causing incorrect pixel correspondences. Geometry recovered from each image is also hampered by distortions, namely generalized bas-relief ambiguity, leading to erroneous 3D correspondences. We show that reflection correspondences can resolve the ambiguities arising from these distortions. We introduce a neural correspondence estimator and a RANSAC algorithm that fully leverages all three kinds of correspondences for robust and accurate joint camera pose and object shape estimation just from the object appearance. The method expands the horizon of numerous downstream tasks, including camera pose estimation for appearance modeling (e.g., NeRF) and motion estimation of reflective objects (e.g., cars on the road), to name a few, as it relieves the requirement of overlapping background.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
