We develop new statistics for robustly filtering corrupted keypoint matches in the structure from motion pipeline. The statistics are based on consistency constraints that arise within the clustered structure of the graph of keypoint matches. The statistics are designed to give smaller values to corrupted matches and than uncorrupted matches. These new statistics are combined with an iterative reweighting scheme to filter keypoints, which can then be fed into any standard structure from motion pipeline. This filtering method can be efficiently implemented and scaled to massive datasets as it only requires sparse matrix multiplication. We demonstrate the efficacy of this method on synthetic and real structure from motion datasets and show that it achieves state-of-the-art accuracy and speed in these tasks.
Shanghai Jiao Tong University（上海交通大学）；IBM Research – China（IBM中国研究院）；INRIA（法国国家信息与自动化研究所）；Ecole Normale Superieure（巴黎高等师范学院）；East China Normal University（华东师范大学）
This paper addresses the problem of matching $N$ weighted graphs referring to an identical object or category. More specifically, matching the common node correspondences among graphs. This multi-graph matching problem involves two ingredients affecting the overall accuracy: i) the local pairwise matching affinity score among graphs; ii) the global matching consistency that measures the uniqueness of the pairwise matching results by different chaining orders. Previous studies typically either enforce the matching consistency constraints in the beginning of iterative optimization, which may pro…
West Virginia University（西弗吉尼亚大学）；Huazhong University of Science and Technology（华中科技大学）
Matching corresponding features between two images is a fundamental task to computer vision with numerous applications in object recognition, robotics, and 3D reconstruction. Current state of the art in image feature matching has focused on establishing a single consistency in static scenes; by contrast, finding multiple consistencies in dynamic scenes has been under-researched. In this paper, we present an end-to-end optimization framework named "iterative clustering with Game-Theoretic Matching" (ic-GTM) for robust multi-consistency correspondence. The key idea is to formulate multi-consiste…
Beihang University（北京航空航天大学）；RIKEN AIP（理化学研究所先进智能项目中心）；The University of Tokyo（东京大学）；Griffith University（格里菲斯大学）；University of York（约克大学）
Despite recent stereo matching networks achieving impressive performance given sufficient training data, they suffer from domain shifts and generalize poorly to unseen domains. We argue that maintaining feature consistency between matching pixels is a vital factor for promoting the generalization capability of stereo matching networks, which has not been adequately considered. Here we address this issue by proposing a simple pixel-wise contrastive learning across the viewpoints. The stereo contrastive feature loss function explicitly constrains the consistency between learned features of match…
German Research Center for Artificial Intelligence – DFKI（德国人工智能研究中心）
Despite the remarkable progress of deep learning in stereo matching, there exists a gap in accuracy between real-time models and slower state-of-the-art models which are suitable for practical applications. This paper presents an iterative multi-scale coarse-to-fine refinement (iCFR) framework to bridge this gap by allowing it to adopt any stereo matching network to make it fast, more efficient and scalable while keeping comparable accuracy. To reduce the computational cost of matching, we use multi-scale warped features to estimate disparity residuals and push the disparity search range in th…
We propose a novel multi-scale template matching method which is robust against both scaling and rotation in unconstrained environments. The key component behind is a similarity measure referred to as scalable diversity similarity (SDS). Specifically, SDS exploits bidirectional diversity of the nearest neighbor (NN) matches between two sets of points. To address the scale-robustness of the similarity measure, local appearance and rank information are jointly used for the NN search. Furthermore, by introducing penalty term on the scale change, and polar radius term into the similarity measure, …
Seoul National University（首尔大学）；Facebook, Inc.（Facebook公司）
Semi-supervised video object segmentation (semi-VOS) is widely used in many applications. This task is tracking class-agnostic objects from a given target mask. For doing this, various approaches have been developed based on online-learning, memory networks, and optical flow. These methods show high accuracy but are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To resolve this problem, template matching methods are devised for fast processing speed but sacrificing lots of performance in previous models. We introduce a novel semi-VOS model …
Topology-aware Pathological Consistency Matching for Weakly-Paired IHC Virtual Staining
Guangdong Research Center of Polarization Imaging and Measurement Engineering Technology（广东极化成像与测量工程研究中心）；Shenzhen Key Laboratory for Minimal Invasive Medical Technologies（深圳微创医疗技术重点实验室）；Institute of Biopharmaceutical and Health Engineering（生物制药与健康工程研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际 Graduate School）；Tsinghua University（清华大学）
This paper proposes a strategy for efficient geometrical verification in unmanned aerial vehicle (UAV) image matching. First, considering the complex transformation model between correspondence set in the image-space, feature points of initial candidate matches are projected onto an elevation plane in the object-space, with assistant of UAV flight control data and camera mounting angles. Spatial relationships are simplified as a 2D-translation in which a motion establishes the relation of two correspondence points. Second, a hierarchical motion consistency constraint, termed HMCC, is designed …
Academia Sinica（中央研究院）；National Taiwan University（台湾大学）；Virginia Tech（弗吉尼亚理工学院暨州立大学）；University of California, Merced（加州大学默塞德分校）；Google Cloud（谷歌云）
Establishing dense semantic correspondences between object instances remains a challenging problem due to background clutter, significant scale and pose differences, and large intra-class variations. In this paper, we address weakly supervised semantic matching based on a deep network where only image pairs without manual keypoint correspondence annotations are provided. To facilitate network training with this weaker form of supervision, we 1) explicitly estimate the foreground regions to suppress the effect of background clutter and 2) develop cycle-consistent losses to enforce the predicted…
Pose estimation and map building are central ingredients of autonomous robots and typically rely on the registration of sensor data. In this paper, we investigate a new metric for registering images that builds upon on the idea of the photometric error. Our approach combines a gradient orientation-based metric with a magnitude-dependent scaling term. We integrate both into stereo estimation as well as visual odometry systems and show clear benefits for typical disparity and direct image registration tasks when using our proposed metric. Our experimental evaluation indicats that our metric lead…
The matching of multiple objects (e.g. shapes or images) is a fundamental problem in vision and graphics. In order to robustly handle ambiguities, noise and repetitive patterns in challenging real-world settings, it is essential to take geometric consistency between points into account. Computationally, the multi-matching problem is difficult. It can be phrased as simultaneously solving multiple (NP-hard) quadratic assignment problems (QAPs) that are coupled via cycle-consistency constraints. The main limitations of existing multi-matching methods are that they either ignore geometric consiste…
TU Munich（慕尼黑工业大学）；University of Bonn（波恩大学）；MPI Informatics（马克斯·普朗克信息学研究所）
We present a scalable combinatorial algorithm for globally optimizing over the space of geometrically consistent mappings between 3D shapes. We use the mathematically elegant formalism proposed by Windheuser et al. (ICCV 2011) where 3D shape matching was formulated as an integer linear program over the space of orientation-preserving diffeomorphisms. Until now, the resulting formulation had limited practical applicability due to its complicated constraint structure and its large size. We propose a novel primal heuristic coupled with a Lagrange dual problem that is several orders of magnitudes …
ViBA: Implicit Bundle Adjustment with Geometric and Temporal Consistency for Robust Visual Matching
GNSS Research Center, Wuhan University（武汉大学卫星导航定位技术研究中心）；Electronic Information School, Wuhan University（武汉大学电子信息学院）；Hubei Technology Innovation Center for Spatiotemporal Information and Positioning Navigation（湖北省时空信息与定位导航技术创新中心）；Hubei Luojia Laboratory（湖北珞珈实验室）
We present a relocalization pipeline, which combines an absolute pose regression (APR) network with a novel view synthesis based direct matching module, offering superior accuracy while maintaining low inference time. Our contribution is twofold: i) we design a direct matching module that supplies a photometric supervision signal to refine the pose regression network via differentiable rendering; ii) we modify the rotation representation from the classical quaternion to SO(3) in pose regression, removing the need for balancing rotation and translation loss terms. As a result, our network Direc…
Consistency-Driven Calibration and Matching for Few-Shot Class-Incremental Learning
School of Automation, Central South University, China（中南大学自动化学院）；Big Data Institute, Central South University, China（中南大学大数据研究院）；School of Computer Science, Faculty of Engineering, The University of Sydney, Australia（悉尼大学工程学院计算机科学系）
FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching
Indian Institute of Technology Bombay（印度理工学院班加罗尔分校）；University of California, Berkeley（加州大学伯克利分校）；Inception Labs（Inception实验室）；Adobe Research India（Adobe印度研究）
3D-Guided Scalable Flow Matching for Generating Volumetric Tissue Spatial Transcriptomics from Serial Histology
Cambridge Stem Cell Institute, University of Cambridge（剑桥干细胞研究所，剑桥大学）；Cambridge Center for AI in Medicine, University of Cambridge（剑桥人工智能医学中心，剑桥大学）；Department of Medicine, University of Cambridge（剑桥大学医学系）；Computer Science Department, University of Oxford（牛津大学计算机科学系）；Computer Science Department, University of Helsinki（赫尔辛基大学计算机科学系）；Institute for Molecular Medicine Finland, University of Helsinki（芬兰分子医学研究所，赫尔辛基大学）；Wellcome Sanger Institute（沃森桑格研究所）
The complementary fusion of light detection and ranging (LiDAR) data and image data is a promising but challenging task for generating high-precision and high-density point clouds. This study proposes an innovative LiDAR-guided stereo matching approach called LiDAR-guided stereo matching (LGSM), which considers the spatial consistency represented by continuous disparity or depth changes in the homogeneous region of an image. The LGSM first detects the homogeneous pixels of each LiDAR projection point based on their color or intensity similarity. Next, we propose a riverbed enhancement function…
Image feature matching is a fundamental part of many geometric computer vision applications, and using multiple images can improve performance. In this work, we formulate multi-image matching as a graph embedding problem then use a Graph Convolutional Network to learn an appropriate embedding function for aligning image features. We use cycle consistency to train our network in an unsupervised fashion, since ground truth correspondence is difficult or expensive to aquire. In addition, geometric consistency losses can be added at training time, even if the information is not available in the te…
University of Science and Technology of China（中国科学技术大学）
Local feature matching aims at finding correspondences between a pair of images. Although current detector-free methods leverage Transformer architecture to obtain an impressive performance, few works consider maintaining local consistency. Meanwhile, most methods struggle with large scale variations. To deal with the above issues, we propose Adaptive Spot-Guided Transformer (ASTR) for local feature matching, which jointly models the local consistency and scale variations in a unified coarse-to-fine architecture. The proposed ASTR enjoys several merits. First, we design a spot-guided aggregati…
Harbin Institute of Technology（哈尔滨工业大学）；Pengcheng Lab（鹏城实验室）
The constraint of neighborhood consistency or local consistency is widely used for robust image matching. In this paper, we focus on learning neighborhood topology consistent descriptors (TCDesc), while former works of learning descriptors, such as HardNet and DSM, only consider point-to-point Euclidean distance among descriptors and totally neglect neighborhood information of descriptors. To learn topology consistent descriptors, first we propose the linear combination weights to depict the topological relationship between center descriptor and its kNN descriptors, where the difference betwee…
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）
As hashing becomes an increasingly appealing technique for large-scale image retrieval, multi-label hashing is also attracting more attention for the ability to exploit multi-level semantic contents. In this paper, we propose a novel deep hashing method for scalable multi-label image search. Unlike existing approaches with conventional objectives such as contrast and triplet losses, we employ a rank list, rather than pairs or triplets, to provide sufficient global supervision information for all the samples. Specifically, a new rank-consistency objective is applied to align the similarity orde…
University of Science Technology of China（中国科学技术大学）
In this paper, we study the problem of stereo matching from a pair of images with different resolutions, e.g., those acquired with a tele-wide camera system. Due to the difficulty of obtaining ground-truth disparity labels in diverse real-world systems, we start from an unsupervised learning perspective. However, resolution asymmetry caused by unknown degradations between two views hinders the effectiveness of the generally assumed photometric consistency. To overcome this challenge, we propose to impose the consistency between two views in a feature space instead of the image space, named fea…
Austrian Institute of Technology（奥地利技术研究所）；Graz University of Technology（格拉茨技术大学）；Czech Technical University in Prague（布拉格捷克理工大学）
We propose a method for large displacement optical flow in which local matching costs are learned by a convolutional neural network (CNN) and a smoothness prior is imposed by a conditional random field (CRF). We tackle the computation- and memory-intensive operations on the 4D cost volume by a min-projection which reduces memory complexity from quadratic to linear and binary descriptors for efficient matching. This enables evaluation of the cost on the fly and allows to perform learning and CRF inference on high resolution images without ever storing the 4D cost volume. To address the problem …
LUCID: Learned Undersampling-Adaptive Consistency-Guided Inference with Deterministic Flow Matching for Sparse-View CT Reconstruction
School of Mathematical Sciences, Capital Normal University（首都师范大学数学科学学院）；National Center for Applied Mathematics Beijing, Capital Normal University（首都师范大学北京国家应用数学中心）；Academy for Multidisciplinary Studies, Capital Normal University（首都师范大学交叉科学研究院）
Subspace clustering methods based on $\ell_1$, $\ell_2$ or nuclear norm regularization have become very popular due to their simplicity, theoretical guarantees and empirical success. However, the choice of the regularizer can greatly impact both theory and practice. For instance, $\ell_1$ regularization is guaranteed to give a subspace-preserving affinity (i.e., there are no connections between points from different subspaces) under broad conditions (e.g., arbitrary subspaces and corrupted data). However, it requires solving a large scale convex optimization problem. On the other hand, $\ell_2…
