Change in viewpoint is one of the major factors for variation in object appearance across different images. Thus, view-invariant object recognition is a challenging and important image understanding task. In this paper, we propose a method that can match objects in images taken under different viewpoints. Unlike most methods in the literature, no restriction on camera orientations or internal camera parameters are imposed and no prior knowledge of 3D structure of the object is required. We prove that when two cameras take pictures of the same object from two different viewing angels, the relationship between every quadruple of points reduces to the special case of homography with two equal eigenvalues. Based on this property, we formulate the problem as an error function that indicates how likely two sets of 2D points are projections of the same set of 3D points under two different cameras. Comprehensive set of experiments were conducted to prove the robustness of the method to noise, and evaluate its performance on real-world applications, such as face and object recognition.
Aerial image registration or matching is a geometric process of aligning two aerial images captured in different environments. Estimating the precise transformation parameters is hindered by various environments such as time, weather, and viewpoints. The characteristics of the aerial images are mainly composed of a straight line owing to building and road. Therefore, the straight lines are distorted when estimating homography parameters directly between two images. In this paper, we propose a deep homography alignment network to precisely match two aerial images by progressively estimating the…
Geometric matching is a key step in computer vision tasks. Previous learning-based methods for geometric matching concentrate more on improving alignment quality, while we argue the importance of naturalness issue simultaneously. To deal with this, firstly, Pearson correlation is applied to handle large intra-class variations of features in feature matching stage. Then, we parametrize homography transformation with 9 parameters in full connected layer of our network, to better characterize large viewpoint variations compared with affine transformation. Furthermore, a novel loss function with G…
Precise homography estimation between multiple images is a pre-requisite for many computer vision applications. One application that is particularly relevant in today's digital era is the alignment of scanned or camera-captured document images such as insurance claim forms for information extraction. Traditional learning based approaches perform poorly due to the absence of an appropriate gradient. Feature based keypoint extraction techniques for homography estimation in real scene images either detect an extremely large number of inconsistent keypoints due to sharp textual edges, or produce i…
HomoFM: Deep Homography Estimation with Flow Matching
Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；School of Instrument Science and Opto-Electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）；School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学航空航天工程学院）
In computer vision, finding point correspondence among images plays an important role in many applications, such as image stitching, image retrieval, visual localization, etc. Most of the research worksfocus on the matching of local feature before a sampling method is employed, such as RANSAC, to verify initial matching results via repeated fitting of certain global transformation among the images. However, incorrect matches may still exist, while careful examination of such problems is often skipped. Accordingly, a geometrically constrained algorithm is proposed in this work to verify the cor…
Research Center for Information Technology Innovation, Academia Sinica（中央研究院资讯科技创新研究中心）；Department of Computer Science and Information Engineering, National Taiwan University（国立台湾大学资讯工程学系）
With the aim to improve the performance of feature matching, we present an unsupervised approach to fuse various local descriptors in the space of homographies. Inspired by the observation that the homographies of correct feature correspondences vary smoothly along the spatial domain, our approach stands on the unsupervised nature of feature matching, and can select a good descriptor for matching each feature point. Specifically, the homography space serves as the common domain, in which a correspondence obtained by any descriptor is considered as a point, for integrating various heterogeneous…
Estimating homography to align image pairs captured by different sensors or image pairs with large appearance changes is an important and general challenge for many computer vision applications. In contrast to others, we propose a generic solution to pixel-wise align multimodal image pairs by extending the traditional Lucas-Kanade algorithm with networks. The key contribution in our method is how we construct feature maps, named as deep Lucas-Kanade feature map (DLKFM). The learned DLKFM can spontaneously recognize invariant features under various appearance-changing conditions. It also has tw…
Megvii Technology（旷视科技）；Sichuan University（四川大学）；Beijing Jiaotong University（北京交通大学）；University of Electronic Science and Technology of China（电子科技大学）
Estimating homography from an image pair is a fundamental problem in image alignment. Unsupervised learning methods have received increasing attention in this field due to their promising performance and label-free training. However, existing methods do not explicitly consider the problem of plane-induced parallax, which will make the predicted homography compromised on multiple planes. In this work, we propose a novel method HomoGAN to guide unsupervised homography estimation to focus on the dominant plane. First, a multi-scale transformer network is designed to predict homography from the fe…
ETH Zurich（苏黎世联邦理工学院）；CTU in Prague（布拉格捷克理工大学）；University Bonn（波恩大学）
We present a large-scale dataset of Planes in 3D, Pi3D, of roughly 1000 planes observed in 10 000 images from the 1DSfM dataset, and HEB, a large-scale homography estimation benchmark leveraging Pi3D. The applications of the Pi3D dataset are diverse, e.g. training or evaluating monocular depth, surface normal estimation and image matching algorithms. The HEB dataset consists of 226 260 homographies and includes roughly 4M correspondences. The homographies link images that often undergo significant viewpoint and illumination changes. As applications of HEB, we perform a rigorous evaluation of a…
View-Centric Multi-Object Tracking with Homographic Matching in Moving UAV
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China（脑启发智能感知与认知教育部重点实验室，中国科学技术大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Alibaba Group（阿里巴巴集团）；Department of Computer Science and Engineering, Shanghai Jiao Tong University（上海交通大学计算机科学与工程系）
University of Electronic Science and Technology of China（电子科技大学）；Megvii Technology（旷视科技）
Homography estimation is a basic image alignment method in many applications. It is usually conducted by extracting and matching sparse feature points, which are error-prone in low-light and low-texture images. On the other hand, previous deep homography approaches use either synthetic images for supervised learning or aerial images for unsupervised learning, both ignoring the importance of handling depth disparities and moving objects in real world applications. To overcome these problems, in this work we propose an unsupervised deep homography method with a new architecture design. In the sp…
1 Research School of Engineering, The \ National University (ANU), Australia；2 Australia Centre of Excellence for \ Vision (ACRV), Australia
Large field-of-view fisheye lens cameras have attracted more and more researchers' attention in the field of robotics. However, there does not exist a convenient off-the-shelf stereo rectification approach which can be applied directly to fisheye stereo rig. One obvious drawback of existing methods is that the resampling distortion (which is defined as the loss of pixels due to under-sampling and the creation of new pixels due to over-sampling during rectification process) is severe if we want to obtain a rectification with epipolar line (not epipolar circle) constraint. To overcome this weakn…
SeFENet: Robust Deep Homography Estimation via Semantic-Driven Feature Enhancement
School of Software Technology, Dalian University of Technology（大连理工大学软件学院）；School of Mechanical Engineering, Dalian University of Technology（大连理工大学机械工程学院）；College of Information Science and Technology, Dalian Maritime University（大连海事大学信息科学技术学院）
Coverage of image features play an important role in many vision algorithms since their distribution affect the estimated homography. This paper presents a Genetic Algorithm (GA) in order to select the optimal set of features yielding maximum coverage of the image which is measured by a robust method based on spatial statistics. It is shown with statistical tests on two datasets that the metric yields better coverage and this is also confirmed by an accuracy test on the computed homography for the original set and the newly selected set of features. Results have demonstrated that the new set h…
Template matching is one of the most prevalent pattern recognition methods worldwide. It has found uses in most visual concept detection fields. In this work, we investigate methods for improving template matching by adjusting the weights of different regions of the template. We compare several weight maps and test the methods using the FERET face test set in the context of human eye detection.
This article provides a step by step development of designing a Object Detection scheme using the HOG based Feature Pyramid aligned with the concept of Template Matching.
National Technical University of Ukraine "Kyiv Polytechnic Institute"（乌克兰国立技术大学"基辅理工学院"）
This paper reviews Kunchenko's polynomials using as template matching method to recognize template in one-dimensional input signal. Kunchenko's polynomials method is compared with classical methods - cross-correlation and sum of squared differences according to numerical statistical example.
A novel template matching algorithm that can incorporate the concept of deformable parts, is presented in this paper. Unlike the deformable part model (DPM) employed in object recognition, the proposed template-matching approach called Deformable Template Matching (DTM) does not require a training step. Instead, deformation is achieved by a set of predefined basic rules (e.g. the left sub-patch cannot pass across the right patch). Experimental evaluation of this new method using the PASCAL VOC 07 dataset demonstrated substantial performance improvement over conventional template matching algor…
Finding a template in a search image is one of the core problems many computer vision, such as semantic image semantic, image-to-GPS verification \etc. We propose a novel quality-aware template matching method, QATM, which is not only used as a standalone template matching algorithm, but also a trainable layer that can be easily embedded into any deep neural network. Specifically, we assess the quality of a matching pair using soft-ranking among all matching pairs, and thus different matching scenarios such as 1-to-1, 1-to-many, and many-to-many will be all reflected to different values. Our e…
Brute force matching of binary image feature descriptors is conventionally performed using the Hamming distance. This paper assesses the use of alternative metrics in order to see whether they can produce feature correspondences that yield more accurate homography matrices. Two statistical tests, namely ANOVA (Analysis of Variance) and McNemar's test were employed for evaluation. Results show that Jackard-Needham and Dice metrics can display better performance for some descriptors. Yet, these performance differences were not found to be statistically significant.
We propose a novel measure for template matching named Deformable Diversity Similarity -- based on the diversity of feature matches between a target image window and the template. We rely on both local appearance and geometric information that jointly lead to a powerful approach for matching. Our key contribution is a similarity measure, that is robust to complex deformations, significant background clutter, and occlusions. Empirical evaluation on the most up-to-date benchmark shows that our method outperforms the current state-of-the-art in its detection accuracy while improving computational…
Finding a template in a search image is an important task underlying many computer vision applications. Recent approaches perform template matching in a deep feature-space, produced by a convolutional neural network (CNN), which is found to provide more tolerance to changes in appearance. In this article we investigate if enhancing the CNN's encoding of shape information can produce more distinguishable features that improve the performance of template matching. This investigation results in a new template matching method that produces state-of-the-art results on a standard benchmark. To confi…
This paper presents a generic pre-processor for expediting conventional template matching techniques. Instead of locating the best matched patch in the reference image to a query template via exhaustive search, the proposed algorithm rules out regions with no possible matches with minimum computational efforts. While working on simple patch features, such as mean, variance and gradient, the fast pre-screening is highly discriminative. Its computational efficiency is gained by using a novel octagonal-star-shaped template and the inclusion-exclusion principle to extract and compare patch feature…
King Faisal University（费萨尔大学）；Mansoura University（曼苏拉大学）
Template matching is a basic method in image analysis to extract useful information from images. In this paper, we suggest a new method for pattern matching. Our method transform the template image from two dimensional image into one dimensional vector. Also all sub-windows (same size of template) in the reference image will transform into one dimensional vectors. The three similarity measures SAD, SSD, and Euclidean are used to compute the likeness between template and all sub-windows in the reference image to find the best match. The experimental results show the superior performance of the …
Template matching by normalized cross correlation (NCC) is widely used for finding image correspondences. We improve the robustness of this algorithm by preprocessing images with "siamese" convolutional networks trained to maximize the contrast between NCC values of true and false matches. The improvement is quantified using patches of brain images from serial section electron microscopy. Relative to a parameter-tuned bandpass filter, siamese convolutional networks significantly reduce false matches. Furthermore, all false matches can be eliminated by removing a tiny fraction of all matches ba…
Shanghai Jiao Tong University（上海交通大学）；Nanjing University of Science and Technology（南京理工大学）；University of Sydney（悉尼大学）
In this paper, we propose a novel matching based tracker by investigating the relationship between template matching and the recent popular correlation filter based trackers (CFTs). Compared to the correlation operation in CFTs, a sophisticated similarity metric termed "mutual buddies similarity" (MBS) is proposed to exploit the relationship of multiple reciprocal nearest neighbors for target matching. By doing so, our tracker obtains powerful discriminative ability on distinguishing target and background as demonstrated by both empirical and theoretical analyses. Besides, instead of utilizing…
Tokyo Metropolitan University（东京都立大学）；Tokai University（东海大学）
In this paper, we propose a novel template matching method with a white balancing adjustment, called N-white balancing, which was proposed for multi-illuminant scenes. To reduce the influence of lighting effects, N-white balancing is applied to images for multi-illumination color constancy, and then a template matching method is carried out by using adjusted images. In experiments, the effectiveness of the proposed method is demonstrated to be effective in object detection tasks under various illumination conditions.
An Efficient Deep Template Matching and In-Plane Pose Estimation Method via Template-Aware Dynamic Convolution
School of Computer Science, Chengdu University of Information Technology（成都信息工程大学计算机科学学院）；Faculty of Mechanical Engineering and Mechanics, Ningbo University（宁波大学机械与力学学院）
Deep Homography Estimation for Visual Place Recognition
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；Chongqing Institute of Green and Intelligent Technology, Chinese Academy of Sciences（中国科学院重庆绿色智能技术研究院）；Southern University of Science and Technology（南方科技大学）
Georgia Institute of Technology（佐治亚理工学院）；Microsoft Research（微软研究院）
Image alignment tasks require accurate pixel correspondences, which are usually recovered by matching local feature descriptors. Such descriptors are often derived using supervised learning on existing datasets with ground truth correspondences. However, the cost of creating such datasets is usually prohibitive. In this paper, we propose a new approach to align two images related by an unknown 2D homography where the local descriptor is learned from scratch from the images and the homography is estimated simultaneously. Our key insight is that a siamese convolutional neural network can be trai…
Department of Computer Science and Engineering（计算机科学与工程系）；Michigan State University（密歇根州立大学）；East Lansing（东兰辛（城市名，非机构））；Michigan 48824（密歇根州 48824（邮编，非机构））
We propose a texture template approach, consisting of a set of virtual minutiae, to improve the overall latent fingerprint recognition accuracy. To compensate for the lack of sufficient number of minutiae in poor quality latent prints, we generate a set of virtual minutiae. However, due to a large number of these regularly placed virtual minutiae, texture based template matching has a large computational requirement compared to matching true minutiae templates. To improve both the accuracy and efficiency of the texture template matching, we investigate: i) both original and enhanced fingerprin…
Seoul National University（首尔大学）；Facebook, Inc.（Facebook公司）
Semi-supervised video object segmentation (semi-VOS) is widely used in many applications. This task is tracking class-agnostic objects from a given target mask. For doing this, various approaches have been developed based on online-learning, memory networks, and optical flow. These methods show high accuracy but are hard to be utilized in real-world applications due to slow inference time and tremendous complexity. To resolve this problem, template matching methods are devised for fast processing speed but sacrificing lots of performance in previous models. We introduce a novel semi-VOS model …
Sun Yat-Sen University（中山大学）；University of California, Los Angeles（加州大学洛杉矶分校）
This paper aims at a newly raising task in visual surveillance: re-identifying people at a distance by matching body information, given several reference examples. Most of existing works solve this task by matching a reference template with the target individual, but often suffer from large human appearance variability (e.g. different poses/views, illumination) and high false positives in matching caused by conjunctions, occlusions or surrounding clutters. Addressing these problems, we construct a simple yet expressive template from a few reference images of a certain individual, which represe…
Tel-Aviv University（特拉维夫大学）；MIT Computer Science and Artificial Intelligence Lab（麻省理工学院计算机科学与人工智能实验室）；Google（谷歌）
We propose a novel method for template matching in unconstrained environments. Its essence is the Best-Buddies Similarity (BBS), a useful, robust, and parameter-free similarity measure between two sets of points. BBS is based on counting the number of Best-Buddies Pairs (BBPs)--pairs of points in source and target sets, where each point is the nearest neighbor of the other. BBS has several key features that make it robust against complex geometric deformations and high levels of outliers, such as those arising from background clutter and occlusions. We study these properties, provide a statist…
Non-planar Object Detection and Identification by Features Matching and Triangulation Growth
POLITECNICO DI MILANO School of Industrial and Information Engineering（米兰理工大学工业与信息工程学院）；Master of Science Degree in Computer Science and Engineering Department of Electronics, Information and Bioengineering（电子、信息与生物工程学院）
Face recognition technology has demonstrated tremendous progress over the past few years, primarily due to advances in representation learning. As we witness the widespread adoption of these systems, it is imperative to consider the security of face representations. In this paper, we explore the practicality of using a fully homomorphic encryption based framework to secure a database of face templates. This framework is designed to preserve the privacy of users and prevent information leakage from the templates, while maintaining their utility through template matching directly in the encrypte…
SZTAKI（匈牙利科学院计算机与自动化研究所）；Czech Technical University in Prague（布拉格捷克理工大学）；University of Bonn（波恩大学）；Chalmers University of Technology（查尔姆斯理工大学）
Local features e.g. SIFT and its affine and learned variants provide region-to-region rather than point-to-point correspondences. This has recently been exploited to create new minimal solvers for classical problems such as homography, essential and fundamental matrix estimation. The main advantage of such solvers is that their sample size is smaller, e.g., only two instead of four matches are required to estimate a homography. Works proposing such solvers often claim a significant improvement in run-time thanks to fewer RANSAC iterations. We show that this argument is not valid in practice if…
We propose a novel multi-scale template matching method which is robust against both scaling and rotation in unconstrained environments. The key component behind is a similarity measure referred to as scalable diversity similarity (SDS). Specifically, SDS exploits bidirectional diversity of the nearest neighbor (NN) matches between two sets of points. To address the scale-robustness of the similarity measure, local appearance and rank information are jointly used for the NN search. Furthermore, by introducing penalty term on the scale change, and polar radius term into the similarity measure, …
XJTLU（西交利物浦大学）；University of Liverpool（利物浦大学）；Beijing Jiaotong University（北京交通大学）
In this paper, the main task we aim to tackle is the multi-instance semi-supervised video object segmentation across a sequence of frames where only the first-frame box-level ground-truth is provided. Detection-based algorithms are widely adopted to handle this task, and the challenges lie in the selection of the matching method to predict the result as well as to decide whether to update the target template using the newly predicted result. The existing methods, however, make these selections in a rough and inflexible way, compromising their performance. To overcome this limitation, we propos…
Weizmann Institute of Science（魏茨曼科学研究所）；UCLA Vision Lab（加州大学洛杉矶分校视觉实验室）；Northrop Grumman（诺斯罗普·格鲁曼公司）
We present a novel approach to template matching that is efficient, can handle partial occlusions, and comes with provable performance guarantees. A key component of the method is a reduction that transforms the problem of searching a nearest neighbor among $N$ high-dimensional vectors, to searching neighbors among two sets of order $\sqrt{N}$ vectors, which can be found efficiently using range search techniques. This allows for a quadratic improvement in search complexity, and makes the method scalable in handling large search spaces. The second contribution is a hashing scheme based on conse…
University of Copenhagen（哥本哈根大学）；Rigshospitalet, University Hospital of Copenhagen（哥本哈根大学医院）；Erasmus MC Rotterdam（鹿特丹伊拉斯姆斯大学医学中心）
Knowledge of airway tree morphology has important clinical applications in diagnosis of chronic obstructive pulmonary disease. We present an automatic tree extraction method based on multiple hypothesis tracking and template matching for this purpose and evaluate its performance on chest CT images. The method is adapted from a semi-automatic method devised for vessel segmentation. Idealized tubular templates are constructed that match airway probability obtained from a trained classifier and ranked based on their relative significance. Several such regularly spaced templates form the local hyp…
University of Verona（维罗纳大学）；Sapienza University of Rome（罗马萨皮恩扎大学）
We propose a new approach for 3D shape matching of deformable human shapes. Our approach is based on the joint adoption of three different tools: an intrinsic spectral matching pipeline, a morphable model, and an extrinsic details refinement. By operating in conjunction, these tools allow us to greatly improve the quality of the matching while at the same time resolving the key issues exhibited by each tool individually. In this paper we present an innovative High-Resolution Augmentation (HRA) strategy that enables highly accurate correspondence even in the presence of significant mesh resolut…
The Hong Kong University of Science and Technology（香港科技大学）；Tencent（腾讯）
Significant progress has been made in Video Object Segmentation (VOS), the video object tracking task in its finest level. While the VOS task can be naturally decoupled into image semantic segmentation and video object tracking, significantly much more research effort has been made in segmentation than tracking. In this paper, we introduce "tracking-by-detection" into VOS which can coherently integrate segmentation into tracking, by proposing a new temporal aggregation network and a novel dynamic time-evolving template matching mechanism to achieve significantly improved performance. Notably, …
In most computer vision and image analysis problems, it is necessary to define a similarity measure between two or more different objects or images. Template matching is a classic and fundamental method used to score similarities between objects using certain mathematical algorithms. In this paper, we reviewed the basic concept of matching, as well as advances in template matching and applications such as invariant features or novel applications in medical image analysis. Additionally, deformable models and templates originating from classic template matching were discussed. These models have …
