Matching two images while estimating their relative geometry is a key step in many computer vision applications. For decades, a well-established pipeline, consisting of SIFT, RANSAC, and 8-point algorithm, has been used for this task. Recently, many new approaches were proposed and shown to outperform previous alternatives on standard benchmarks, including the learned features, correspondence pruning algorithms, and robust estimators. However, whether it is beneficial to incorporate them into the classic pipeline is less-investigated. To this end, we are interested in i) evaluating the performance of these recent algorithms in the context of image matching and epipolar geometry estimation, and ii) leveraging them to design more practical registration systems. The experiments are conducted in four large-scale datasets using strictly defined evaluation metrics, and the promising results provide insight into which algorithms suit which scenarios. According to this, we propose three high-quality matching systems and a Coarse-to-Fine RANSAC estimator. They show remarkable performances and have potentials to a large part of computer vision tasks. To facilitate future research, the full evaluation pipeline and the proposed methods are made publicly available.
University of Adelaide（阿德莱德大学）；Nankai University（南开大学）；Advanced Digital Sciences Center（先进数字科学中心）；Huawei Technologies（华为技术有限公司）
Feature matching is one of the most fundamental and active research areas in computer vision. A comprehensive evaluation of feature matchers is necessary, since it would advance both the development of this field and also high-level applications such as Structure-from-Motion or Visual SLAM. However, to the best of our knowledge, no previous work targets the evaluation of feature matchers while they only focus on evaluating feature detectors and descriptors. This leads to a critical absence in this field that there is no standard datasets and evaluation metrics to evaluate different feature mat…
GMatch: A Lightweight, Geometry-Constrained Keypoint Matcher for Zero-Shot 6DoF Pose Estimation in Robotic Grasp Tasks
The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院，北京）
Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives
School of Automation, Southeast University（东南大学自动化学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Institute of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能研究院）；School of Robotics, Wuhan University（武汉大学机器人学院）；Graduate School of Information, Production and Systems, Waseda University（早稻田大学信息生产系统研究科）
IIIT Hyderabad（海得拉巴国际信息技术学院）；Queensland University of Technology（昆士兰科技大学）
Sparse local feature matching is pivotal for many computer vision and robotics tasks. To improve their invariance to challenging appearance conditions and viewing angles, and hence their usefulness, existing learning-based methods have primarily focused on data augmentation-based training. In this work, we propose an alternative, complementary approach that centers on inducing bias in the model architecture itself to generate `rotation-specific' features using Steerable E2-CNNs, that are then group-pooled to achieve rotation-invariant local features. We demonstrate that this high performance, …
Deep Learning Reforms Image Matching: A Survey and Outlook
Electronic Information School, Wuhan University（武汉大学电子信息学院）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
The University of Western Australia（西澳大学）；Murdoch University（莫道克大学）
This paper proposes a novel concept to directly match feature descriptors extracted from 2D images with feature descriptors extracted from 3D point clouds. We use this concept to directly localize images in a 3D point cloud. We generate a dataset of matching 2D and 3D points and their corresponding feature descriptors, which is used to learn a Descriptor-Matcher classifier. To localize the pose of an image at test time, we extract keypoints and feature descriptors from the query image. The trained Descriptor-Matcher is then used to match the features from the image and the point cloud. The loc…
The University of Western Australia（西澳大学）；Murdoch University（默多克大学）
We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D keypoints and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D keypoints by matching the 2…
Cornell University（康奈尔大学）；Cornell Tech（康奈尔科技校区）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）
Estimating fundamental matrices is a classic problem in computer vision. Traditional methods rely heavily on the correctness of estimated key-point correspondences, which can be noisy and unreliable. As a result, it is difficult for these methods to handle image pairs with large occlusion or significantly different camera poses. In this paper, we propose novel neural network architectures to estimate fundamental matrices in an end-to-end manner without relying on point correspondences. New modules and layers are introduced in order to preserve mathematical properties of the fundamental matrix …
Total S.A.（道达尔公司）；Sorbonne Université（索邦大学）；CNRS Laboratoire d’Informatique de Paris 6（法国国家科学研究中心巴黎第六大学计算机实验室）
This paper presents a robust and efficient method for tracking topological features in time-varying scalar data. Structures are tracked based on the optimal matching between persistence diagrams with respect to the Wasserstein metric. This fundamentally relies on solving the assignment problem, a special case of optimal transport, for all consecutive timesteps. Our approach relies on two main contributions. First, we revisit the seminal assignment algorithm by Kuhn and Munkres which we specifically adapt to the problem of matching persistence diagrams in an efficient way. Second, we propose an…
Feature matching is a challenging computer vision task that involves finding correspondences between two images of a 3D scene. In this paper we consider the dense approach instead of the more common sparse paradigm, thus striving to find all correspondences. Perhaps counter-intuitively, dense methods have previously shown inferior performance to their sparse and semi-sparse counterparts for estimation of two-view geometry. This changes with our novel dense method, which outperforms both dense and sparse methods on geometry estimation. The novelty is threefold: First, we propose a kernel regres…
Semi-Dense Matching Uncertainty Is Not Just Local Confidence
University of Engineering and Technology, Vietnam National University（越南国家大学工程技术大学）；Optoelectronics Center, Viettel Aerospace Institute, Viettel Group（Viettel集团Viettel宇航研究院光电中心）；National Yang Ming Chiao Tung University（国立阳明交通大学）
In computer vision, the estimation of the fundamental matrix is a basic problem that has been extensively studied. The accuracy of the estimation imposes a significant influence on subsequent tasks such as the camera trajectory determination and 3D reconstruction. In this paper we propose a new method for fundamental matrix estimation that makes use of clustering a group of 4D vectors. The key insight is the observation that among the 4D vectors constructed from matching pairs of points obtained from the SIFT algorithm, well-defined cluster points tend to be reliable inliers suitable for funda…
In this paper, we present a novel end-to-end network architecture to estimate fundamental matrix directly from stereo images. To establish a complete working pipeline, different deep neural networks in charge of finding correspondences in images, performing outlier rejection and calculating fundamental matrix, are integrated into an end-to-end network architecture. To well train the network and preserve geometry properties of fundamental matrix, a new loss function is introduced. To evaluate the accuracy of estimated fundamental matrix more reasonably, we design a new evaluation metric which i…
DreamSat-Pose: Spacecraft Pose Estimation from Single-View 3D Reconstructions and Learned 2D-3D Feature Matching
Wellesley College（韦尔斯利学院）；Massachusetts Institute of Technology（麻省理工学院）；Politecnico di Milano（米兰理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Universidad Politécnica de Madrid（马德里理工大学）
Centre for Machine Perception, Czech Technical University in Prague（布拉格捷克理工大学机器感知中心）；Machine Perception Research Laboratory, MTA SZTAKI（MTA SZTAKI机器感知研究实验室）
We review the most recent RANSAC-like hypothesize-and-verify robust estimators. The best performing ones are combined to create a state-of-the-art version of the Universal Sample Consensus (USAC) algorithm. A recent objective is to implement a modular and optimized framework, making future RANSAC modules easy to be included. The proposed method, USACv20, is tested on eight publicly available real-world datasets, estimating homographies, fundamental and essential matrices. On average, USACv20 leads to the most geometrically accurate models and it is the fastest in comparison to the state-of-the…
Generating robust and reliable correspondences across images is a fundamental task for a diversity of applications. To capture context at both global and local granularity, we propose ASpanFormer, a Transformer-based detector-free matcher that is built on hierarchical attention structure, adopting a novel attention operation which is capable of adjusting attention span in a self-adaptive manner. To achieve this goal, first, flow maps are regressed in each cross attention phase to locate the center of search region. Next, a sampling grid is generated around the center, whose size, instead of be…
The State Key Lab. LIESMARS, Wuhan University（武汉大学国家重点实验室LIESMARS）；École Polytechnique Fédérale de Lausanne (EPFL)（瑞士联邦理工学院（EPFL））；Chinese Academy of Sciences（中国科学院）；China University of Geosciences（中国地质大学）
The fundamental matrix (FM) describes the geometric relations that exist between two images of the same scene. Different error criteria are used for estimating FMs from an input set of correspondences. In this paper, the accuracy and efficiency aspects of the different error criteria were studied. We mathematically and experimentally proved that the most popular error criterion, the symmetric epipolar distance, is biased. It was also shown that despite the similarity between the algebraic expressions of the symmetric epipolar distance and Sampson distance, they have different accuracy properti…
University of Maryland, College Park（马里兰大学帕克分校）；Weizmann Institute of Science（魏茨曼科学研究学院）；Princeton University（普林斯顿大学）
Accurate estimation of camera matrices is an important step in structure from motion algorithms. In this paper we introduce a novel rank constraint on collections of fundamental matrices in multi-view settings. We show that in general, with the selection of proper scale factors, a matrix formed by stacking fundamental matrices between pairs of images has rank 6. Moreover, this matrix forms the symmetric part of a rank 3 matrix whose factors relate directly to the corresponding camera matrices. We use this new characterization to produce better estimations of fundamental matrices by optimizing …
A Comparative Study of Vision Transformers and CNNs for Few-Shot Rigid Transformation and Fundamental Matrix Estimation
Department of Computer Science, Ben-Gurion University of the Negev（本·古里安大学内盖夫分校计算机科学系）；School of Electrical and Computer Engineering, Ben-Gurion University of the Negev（本·古里安大学内盖夫分校电气与计算机工程学院）；GE Healthcare Science and Technology（通用电气医疗科技）
Machine Perception Research Laboratory（机器感知研究实验室）；MTA SZTAKI（匈牙利科学院计算机与自动化研究所（MTA SZTAKI））；Budapest, Hungary（匈牙利布达佩斯（注意：这是地址信息，不应作为机构提取，此处原文本中机构已完整列出，此条为冗余地址，按要求应忽略））
We aim at estimating the fundamental matrix in two views from five correspondences of rotation invariant features obtained by e.g.\ the SIFT detector. The proposed minimal solver first estimates a homography from three correspondences assuming that they are co-planar and exploiting their rotational components. Then the fundamental matrix is obtained from the homography and two additional point pairs in general position. The proposed approach, combined with robust estimators like Graph-Cut RANSAC, is superior to other state-of-the-art algorithms both in terms of accuracy and number of iteration…
Statistical Multiresolution Dantzig Estimation in Imaging: Fundamental Concepts and Algorithmic Framework
Institute for Mathematical Stochastics University of G \"o ttingen Goldschmidtstra e 7, 37077 G \"o ttingen；Max Planck Institute for Biophysical Chemistry Am Fa berg 11, 37077 G \"o ttingen
