This paper presents a unique approach for the dichotomy between useful and adverse variations of key-point descriptors, namely the identity and the expression variations in the descriptor (feature) space. The descriptors variations are learned from training examples. Based on the labels of the training data, the equivalence relations among the descriptors are established. Both types of descriptor variations are represented by a graph embedded in the descriptor manifold. The invariant recognition is then conducted as a graph search problem. A heuristic graph search algorithm suitable for the recognition under this setup was devised. The proposed approach was tests on the FRGC v2.0, the Bosphorus and the 3D TEC datasets. It has shown to enhance the recognition performance, under expression variations in particular, by considerable margins.
This work presents a novel descriptor for texture images based on fractal geometry and its application to image analysis. The descriptors are provided by estimating the triangular prism fractal dimension under different scales with a weight exponential parameter, followed by dimensionality reduction using Karhunen-Loève transform. The efficiency of the proposed descriptors is tested on two well-known texture data sets, that is, Brodatz and Vistex, both for classification and image retrieval. The novel method is also tested concerning invariances in situations when the textures are rotated or a…
We investigate the problem of image retrieval based on visual queries when the latter comprise arbitrary regions-of-interest (ROI) rather than entire images. Our proposal is a compact image descriptor that combines the state-of-the-art in content-based descriptor extraction with a multi-level, Voronoi-based spatial partitioning of each dataset image. The proposed multi-level Voronoi-based encoding uses a spatial hierarchical K-means over interest-point locations, and computes a content-based descriptor over each cell. In order to reduce the matching complexity with minimal or no sacrifice in r…
Microsoft Mixed Reality and AI Zurich lab（微软混合现实与人工智能苏黎世实验室）；Department of Computer Science, ETH Zurich（苏黎世联邦理工学院计算机科学系）
To be invariant, or not to be invariant: that is the question formulated in this work about local descriptors. A limitation of current feature descriptors is the trade-off between generalization and discriminative power: more invariance means less informative descriptors. We propose to overcome this limitation with a disentanglement of invariance in local descriptors and with an online selection of the most appropriate invariance given the context. Our framework consists in a joint learning of multiple local descriptors with different levels of invariance and of meta descriptors encoding the r…
Illumination-insensitive Binary Descriptor for Visual Measurement Based on Local Inter-patch Invariance
University of Electronic Science and Technology of China (UESTC)（电子科技大学）；Sichuan University（四川大学）；institut superieur d’electronique de Paris - ISEP（巴黎高等电子学院（ISEP））
Institute for Infocomm Research (A*STAR)（信息通信研究院（新加坡科技研究局））；Université Pierre et Marie Curie（皮埃尔和玛丽·居里大学）
With deep learning becoming the dominant approach in computer vision, the use of representations extracted from Convolutional Neural Nets (CNNs) is quickly gaining ground on Fisher Vectors (FVs) as favoured state-of-the-art global image descriptors for image instance retrieval. While the good performance of CNNs for image classification are unambiguously recognised, which of the two has the upper hand in the image retrieval context is not entirely clear yet. In this work, we propose a comprehensive study that systematically evaluates FVs and CNNs for image retrieval. The first part compares th…
Image search systems based on local descriptors typically achieve orientation invariance by aligning the patches on their dominant orientations. Albeit successful, this choice introduces too much invariance because it does not guarantee that the patches are rotated consistently. This paper introduces an aggregation strategy of local descriptors that achieves this covariance property by jointly encoding the angle in the aggregation stage in a continuous manner. It is combined with an efficient monomial embedding to provide a codebook-free method to aggregate local descriptors into a single vect…
University of Otago（奥塔哥大学）；Indian Statistical Institute（印度统计研究所）
This letter introduces the LOOP binary descriptor (local optimal oriented pattern) that encodes rotation invariance into the main formulation itself. This makes any post processing stage for rotation invariance redundant and improves on both accuracy and time complexity. We consider fine-grained lepidoptera (moth/butterfly) species recognition as the representative problem since it involves repetition of localized patterns and textures that may be exploited for discrimination. We evaluate the performance of LOOP against its predecessors as well as few other popular descriptors. Besides experim…
VNI-Net: Vector Neurons-based Rotation-Invariant Descriptor for LiDAR Place Recognition
Department of Computer Science and Technology, School of Electronics and Information Engineering, Tongji University（同济大学电子信息工程学院计算机科学与技术系）；MOE Key Lab of Embedded System and Service Computing, Tongji University（同济大学嵌入式系统与服务计算教育部重点实验室）；Institute of Intelligent Vehicles, Tongji University（同济大学智能汽车研究所）
School of Computer Science, University of Sydney（悉尼大学计算机科学学院）
Recent investigations on rotation invariance for 3D point clouds have been devoted to devising rotation-invariant feature descriptors or learning canonical spaces where objects are semantically aligned. Examinations of learning frameworks for invariance have seldom been looked into. In this work, we review rotation invariance in terms of point cloud registration and propose an effective framework for rotation invariance learning via three sequential stages, namely rotation-invariant shape encoding, aligned feature integration, and deep feature registration. We first encode shape descriptors co…
Finding local correspondences between images with different viewpoints requires local descriptors that are robust against geometric transformations. An approach for transformation invariance is to integrate out the transformations by pooling the features extracted from transformed versions of an image. However, the feature pooling may sacrifice the distinctiveness of the resulting descriptors. In this paper, we introduce a novel visual descriptor named Group Invariant Feature Transform (GIFT), which is both discriminative and robust to geometric transformations. The key idea is that the featur…
Most image instance retrieval pipelines are based on comparison of vectors known as global image descriptors between a query image and the database images. Due to their success in large scale image classification, representations extracted from Convolutional Neural Networks (CNN) are quickly gaining ground on Fisher Vectors (FVs) as state-of-the-art global descriptors for image instance retrieval. While CNN-based descriptors are generally remarked for good retrieval performance at lower bitrates, they nevertheless present a number of drawbacks including the lack of robustness to common object …
We present a new smooth, Gaussian-like kernel that allows the kernel density estimate for an angular distribution to be exactly represented by a finite number of its Fourier series coefficients. Distributions of angular quantities, such as gradients, are a central part of several state-of-the-art image processing algorithms, but these distributions are usually described via histograms and therefore lack rotation invariance due to binning artifacts. Replacing histograming with kernel density estimation removes these binning artifacts and can provide a finite-dimensional descriptor of the distri…
Université Pierre et Marie Curie（皮埃尔和玛丽·居里大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
The goal of this work is the computation of very compact binary hashes for image instance retrieval. Our approach has two novel contributions. The first one is Nested Invariance Pooling (NIP), a method inspired from i-theory, a mathematical theory for computing group invariant transformations with feed-forward neural networks. NIP is able to produce compact and well-performing descriptors with visual representations extracted from convolutional neural networks. We specifically incorporate scale, translation and rotation invariances but the scheme can be extended to any arbitrary sets of transf…
Utrecht University（乌得勒支大学）；University of Amsterdam（阿姆斯特丹大学）；SIM University（SIM大学）
In this work, we aim to segment and detect water in videos. Water detection is beneficial for appllications such as video search, outdoor surveillance, and systems such as unmanned ground vehicles and unmanned aerial vehicles. The specific problem, however, is less discussed compared to general texture recognition. Here, we analyze several motion properties of water. First, we describe a video pre-processing step, to increase invariance against water reflections and water colours. Second, we investigate the temporal and spatial properties of water and derive corresponding local descriptors. Th…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
A number of recent approaches have used deep convolutional neural networks (CNNs) to build texture representations. Nevertheless, it is still unclear how these models represent texture and invariances to categorical variations. This work conducts a systematic evaluation of recent CNN-based texture descriptors for recognition and attempts to understand the nature of invariances captured by these representations. First we show that the recently proposed bilinear CNN model [25] is an excellent general-purpose texture descriptor and compares favorably to other CNN-based descriptors on various text…
Istituto Dalle Molle di Studi sull’Intelligenza Artificiale (IDSIA)（达勒莫勒人工智能研究所（IDSIA））；Faculty of Informatics, Università della Svizzera Italiana (USI)（瑞士意大利语区大学信息学院（USI））；Scuola Universitaria Professionale della Svizzera Italiana (SUPSI)（瑞士意大利语区高等专业学院（SUPSI））；Evidence Srl（Evidence 有限公司）
Feature matching in omnidirectional vision systems is a challenging problem, mainly because complicated optical systems make the theoretical modelling of invariance and construction of invariant feature descriptors hard or even impossible. In this paper, we propose learning invariant descriptors using a training set of similar and dissimilar descriptor pairs. We use the similarity-preserving hashing framework, in which we are trying to map the descriptor data to the Hamming space preserving the descriptor similarity on the training set. A neural network is used to solve the underlying optimiza…
We address the problem of generating video features for action recognition. The spatial pyramid and its variants have been very popular feature models due to their success in balancing spatial location encoding and spatial invariance. Although it seems straightforward to extend spatial pyramid to the temporal domain (spatio-temporal pyramid), the large spatio-temporal diversity of unconstrained videos and the resulting significantly higher dimensional representations make it less appealing. This paper introduces the space-time extended descriptor, a simple but efficient alternative way to incl…
This paper presents a novel approach to exploit the distinctive invariant features in convolutional neural network. The proposed CNN model uses Scale Invariant Feature Transform (SIFT) descriptor instead of the max-pooling layer. Max-pooling layer discards the pose, i.e., translational and rotational relationship between the low-level features, and hence unable to capture the spatial hierarchies between low and high level features. The SIFT descriptor layer captures the orientation and the spatial relationship of the features extracted by convolutional layer. The proposed SIFT Descriptor CNN t…
In this paper we propose a new descriptor for birds search. First, our work was carried on the choice of a descriptor. This choice is usually driven by the application requirements such as robustness to noise, stability with respect to bias, the invariance to geometrical transformations or tolerance to occlusions. In this context, we introduce a descriptor which combines the shape and color descriptors to have an effectiveness description of birds. The proposed descriptor is an adaptation of a descriptor based on the contours defined in article Belongie et al. [5] combined with color moments […
Rensselaer Polytechnic Institute（伦斯勒理工学院）；IBM TJ Watson Research Center（IBM TJ 沃森研究中心）
In this paper, we introduce a shape-based, time-scale invariant feature descriptor for 1-D sensor signals. The time-scale invariance of the feature allows us to use feature from one training event to describe events of the same semantic class which may take place over varying time scales such as walking slow and walking fast. Therefore it requires less training set. The descriptor takes advantage of the invariant location detection in the scale space theory and employs a high level shape encoding scheme to capture invariant local features of events. Based on this descriptor, a scale-invariant …
Pohang University of Science and Technology (POSTECH)（浦项科技大学）
Extracting discriminative local features that are invariant to imaging variations is an integral part of establishing correspondences between images. In this work, we introduce a self-supervised learning framework to extract discriminative rotation-invariant descriptors using group-equivariant CNNs. Thanks to employing group-equivariant CNNs, our method effectively learns to obtain rotation-equivariant features and their orientations explicitly, without having to perform sophisticated data augmentations. The resultant features and their orientations are further processed by group aligning, a n…
Shapes and texture image recognition usage is an essential branch of pattern recognition. It is made up of techniques that aim at extracting information from images via human knowledge and works. Local Binary Pattern (LBP) ensures encoding global and local information and scaling invariance by introducing a look-up table to reflect the uniformity structure of an object. However, edge direction matrixes (EDMS) only apply global invariant descriptor which employs first and secondary order relationships. The main idea behind this methodology is the need of improved recognition capabilities, a goa…
University of Wisconsin-Milwaukee（威斯康星大学密尔沃基分校）；Marshfield Clinic Research Institute（马什菲尔德诊所研究所）
With the rapid expansion of applied 3D computational vision, shape descriptors have become increasingly important for a wide variety of applications and objects from molecules to planets. Appropriate shape descriptors are critical for accurate (and efficient) shape retrieval and 3D model classification. Several spectral-based shape descriptors have been introduced by solving various physical equations over a 3D surface model. In this paper, for the first time, we incorporate a specific group of techniques in statistics and machine learning, known as manifold learning, to develop a global shape…
Despite the importance of image representations such as histograms of oriented gradients and deep Convolutional Neural Networks (CNN), our theoretical understanding of them remains limited. Aiming at filling this gap, we investigate three key mathematical properties of representations: equivariance, invariance, and equivalence. Equivariance studies how transformations of the input image are encoded by the representation, invariance being a special case where a transformation has no effect. Equivalence studies whether two representations, for example two different parametrisations of a CNN, cap…
Technische Universität München（慕尼黑工业大学）；Siemens AG（西门子股份公司）；National University of Defense Technology（国防科技大学）
We present PPF-FoldNet for unsupervised learning of 3D local descriptors on pure point cloud geometry. Based on the folding-based auto-encoding of well known point pair features, PPF-FoldNet offers many desirable properties: it necessitates neither supervision, nor a sensitive local reference frame, benefits from point-set sparsity, is end-to-end, fast, and can extract powerful rotation invariant descriptors. Thanks to a novel feature visualization, its evolution can be monitored to provide interpretable insights. Our extensive experiments demonstrate that despite having six degree-of-freedom …
Guangdong University of Technology（广东工业大学）；Southern University of Science and Technology（南方科技大学）
Visual place recognition (VPR) in condition-varying environments is still an open problem. Popular solutions are CNN-based image descriptors, which have been shown to outperform traditional image descriptors based on hand-crafted visual features. However, there are two drawbacks of current CNN-based descriptors: a) their high dimension and b) lack of generalization, leading to low efficiency and poor performance in applications. In this paper, we propose to use a convolutional autoencoder (CAE) to tackle this problem. We employ a high-level layer of a pre-trained CNN to generate features, and …
