Ground texture based localization is a promising approach to achieve high-accuracy positioning of vehicles. We present a self-contained method that can be used for global localization as well as for subsequent local localization updates, i.e. it allows a robot to localize without any knowledge of its current whereabouts, but it can also take advantage of a prior pose estimate to reduce computation time significantly. Our method is based on a novel matching strategy, which we call identity matching, that is based on compact binary feature descriptors. Identity matching treats pairs of features as matches only if their descriptors are identical. While other methods for global localization are faster to compute, our method reaches higher localization success rates, and can switch to local localization after the initial localization.
Shantou University（汕头大学）；Texas State University（德克萨斯州立大学）
Object recognition and grasping plays a key role in robotic systems, especially for the autonomous robots to implement object sorting tasks in a warehouse. In this paper, we present a global texture-shape 3D feature descriptor which can be utilized in a system of object recognition and grasping, and can perform object sorting tasks well. Our proposed descriptor stems from the clustered viewpoint feature histogram (CVFH), which relies on the geometrical information of the whole 3D object surface only, and can not perform well in recognizing the objects with similar geometrical information. Ther…
This work presents a novel descriptor for texture images based on fractal geometry and its application to image analysis. The descriptors are provided by estimating the triangular prism fractal dimension under different scales with a weight exponential parameter, followed by dimensionality reduction using Karhunen-Loève transform. The efficiency of the proposed descriptors is tested on two well-known texture data sets, that is, Brodatz and Vistex, both for classification and image retrieval. The novel method is also tested concerning invariances in situations when the textures are rotated or a…
São Carlos Institute of Physics, University of São Paulo (USP)（圣保罗大学圣卡洛斯物理研究所）
In this work, we propose a novel technique for obtaining descriptors of gray-level texture images. The descriptors are provided by applying a multiscale transform to the fractal dimension of the image estimated through the probability (Voss) method. The effectiveness of the descriptors is verified in a classification task using benchmark over texture datasets. The results obtained demonstrate the efficiency of the proposed method as a tool for the description and discrimination of texture images.
This work proposes the combination of multiscale transform with fractal descriptors employed in the classification of gray-level texture images. We apply the space-scale transform (derivative + Gaussian filter) over the Bouligand-Minkowski fractal descriptors, followed by a threshold over the filter response, aiming at attenuating noise effects caused by the final part of this response. The method is tested in the classification of a well-known data set (Brodatz) and compared with other classical texture descriptor techniques. The results demonstrate the advantage of the proposed approach, ach…
Instituto de Física de São Carlos (IFSC)（圣卡洛斯物理研究所（IFSC））；Universidade de São Paulo（圣保罗大学）
A gray-level image texture descriptors based on fractal dimension estimation is proposed in this work. The proposed method estimates the fractal dimension using probability (Voss) method. The descriptors are computed applying a multiscale transform to the fractal dimension curves of the texture image. The proposed texture descriptor method is evaluated in a classification task of well known benchmark texture datasets. The results show the great performance of the proposed method as a tool for texture images analysis and characterization.
École de Technologie Supérieure, Université du Québec（魁北克大学高等技术学院）
Texture can be defined as the change of image intensity that forms repetitive patterns, resulting from physical properties of the object's roughness or differences in a reflection on the surface. Considering that texture forms a complex system of patterns in a non-deterministic way, biodiversity concepts can help texture characterization in images. This paper proposes a novel approach capable of quantifying such a complex system of diverse patterns through species diversity and richness and taxonomic distinctiveness. The proposed approach considers each image channel as a species ecosystem and…
São Carlos Institute of Physics, University of São Paulo (USP)（圣保罗大学圣卡洛斯物理研究所）
This work proposes a texture descriptor based on fractal theory. The method is based on the Bouligand-Minkowski descriptors. We decompose the original image recursively into 4 equal parts. In each recursion step, we estimate the average and the deviation of the Bouligand-Minkowski descriptors computed over each part. Thus, we extract entropy features from both average and deviation. The proposed descriptors are provided by the concatenation of such measures. The method is tested in a classification experiment under well known datasets, that is, Brodatz and Vistex. The results demonstrate that …
Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
We present a novel Affine-Gradient based Local Binary Pattern (AGLBP) descriptor for texture classification. It is very hard to describe complicated texture using single type information, such as Local Binary Pattern (LBP), which just utilizes the sign information of the difference between the pixel and its local neighbors. Our descriptor has three characteristics: 1) In order to make full use of the information contained in the texture, the Affine-Gradient, which is different from Euclidean-Gradient and invariant to affine transformation is incorporated into AGLBP. 2) An improved method is pr…
Identifying the origin of a sample image in biometric systems can be beneficial for data authentication in case of attacks against the system and for initiating sensor-specific processing pipelines in sensor-heterogeneous environments. Motivated by shortcomings of the photo response non-uniformity (PRNU) based method in the biometric context, we use a texture classification approach to detect the origin of finger vein sample images. Based on eight publicly available finger vein datasets and applying eight classical yet simple texture descriptors and SVM classification, we demonstrate excellent…
Indian Statistical Institute（印度统计研究所）；National Institute of Technology Rourkela（鲁尔凯拉国家技术学院）
In this paper we propose a novel texture descriptor called Fractal Weighted Local Binary Pattern (FWLBP). The fractal dimension (FD) measure is relatively invariant to scale-changes, and presents a good correlation with human viewpoint of surface roughness. We have utilized this property to construct a scale-invariant descriptor. Here, the input image is sampled using an augmented form of the local binary pattern (LBP) over three different radii, and then used an indexing operation to assign FD weights to the collected samples. The final histogram of the descriptor has its features calculated …
Computer Aided Diagnosis (CAD) system has been developed for the early detection of breast cancer, one of the most deadly cancer for women. The benign of mammogram has different texture from malignant. There are fifty mammogram images used in this work which are divided for training and testing. Therefore, the selection of the right texture to determine the level of accuracy of CAD system is important. The first and second order statistics are the texture feature extraction methods which can be used on a mammogram. This work classifies texture descriptor into nine groups where the extraction o…
Institute of Engineering & Management（工程与管理学院）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Indian Institute of Technology Ropar（印度理工学院罗巴尔分校）
In this paper, we have proposed a novel feature descriptors combining color and texture information collectively. In our proposed color descriptor component, the inter-channel relationship between Hue (H) and Saturation (S) channels in the HSV color space has been explored which was not done earlier. We have quantized the H channel into a number of bins and performed the voting with saturation values and vice versa by following a principle similar to that of the HOG descriptor, where orientation of the gradient is quantized into a certain number of bins and voting is done with gradient magnitu…
In this paper, we present an efficient and distinctive local descriptor, namely block intensity and gradient difference (BIGD). In an image patch, we randomly sample multi-scale block pairs and utilize the intensity and gradient differences of pairwise blocks to construct the local BIGD descriptor. The random sampling strategy and the multi-scale framework help BIGD descriptors capture the distinctive patterns of patches at different orientations and spatial granularity levels. We use vectors of locally aggregated descriptors (VLAD) or improved Fisher vector (IFV) to encode local BIGD descript…
Institute of Engineering & Management（工程与管理学院）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；Indian Institute of Technology Ropar（印度理工学院罗巴尔分校）
In this paper, a new texture descriptor based on the local neighborhood intensity difference is proposed for content based image retrieval (CBIR). For computation of texture features like Local Binary Pattern (LBP), the center pixel in a 3*3 window of an image is compared with all the remaining neighbors, one pixel at a time to generate a binary bit pattern. It ignores the effect of the adjacent neighbors of a particular pixel for its binary encoding and also for texture description. The proposed method is based on the concept that neighbors of a particular pixel hold a significant amount of t…
Second layer scattering descriptors are known to provide good classification performance on natural quasi-stationary processes such as visual textures due to their sensitivity to higher order moments and continuity with respect to small deformations. In a functional Magnetic Resonance Imaging (fMRI) experiment we present visual textures to subjects and evaluate the predictive power of these descriptors with respect to the predictive power of simple contour energy - the first scattering layer. We are able to conclude not only that invariant second layer scattering coefficients better encode vox…
University of Pavia（帕维亚大学）；University of Milano-Bicocca（米兰比可卡大学）
The recognition of color texture under varying lighting conditions is still an open issue. Several features have been proposed for this purpose, ranging from traditional statistical descriptors to features extracted with neural networks. Still, it is not completely clear under what circumstances a feature performs better than the others. In this paper we report an extensive comparison of old and new texture features, with and without a color normalization step, with a particular focus on how they are affected by small and large variation in the lighting conditions. The evaluation is performed …
Along with feature points for image matching, line features provide additional constraints to solve visual geometric problems in robotics and computer vision (CV). Although recent convolutional neural network (CNN)-based line descriptors are promising for viewpoint changes or dynamic environments, we claim that the CNN architecture has innate disadvantages to abstract variable line length into the fixed-dimensional descriptor. In this paper, we effectively introduce Line-Transformers dealing with variable lines. Inspired by natural language processing (NLP) tasks where sentences can be underst…
Instituto de Física de São Carlos - Universidade de São Paulo（圣保罗大学圣卡洛斯物理研究所）
This work proposes the development and study of a novel technique for the generation of fractal descriptors used in texture analysis. The novel descriptors are obtained from a multiscale transform applied to the Fourier technique of fractal dimension calculus. The power spectrum of the Fourier transform of the image is plotted against the frequency in a log- log scale and a multiscale transform is applied to this curve. The obtained values are taken as the fractal descriptors of the image. The validation of the propose is performed by the use of the descriptors for the classification of a data…
St. Pölten University of Applied Sciences（圣珀尔滕应用科学大学）；Jagiellonian University（雅盖隆大学）
Methods from computational topology are becoming more and more popular in computer vision and have shown to improve the state-of-the-art in several tasks. In this paper, we investigate the applicability of topological descriptors in the context of 3D surface analysis for the classification of different surface textures. We present a comprehensive study on topological descriptors, investigate their robustness and expressiveness and compare them with state-of-the-art methods including Convolutional Neural Networks (CNNs). Results show that class-specific information is reflected well in topologi…
University for Health Sciences, Medical Informatics and Technology (UMIT)（健康科学、医学信息与技术大学（UMIT））
We study the applicability of a set of texture descriptors introduced in recent work by the author to texture-based segmentation of images. The texture descriptors under investigation result from applying graph indices from quantitative graph theory to graphs encoding the local structure of images. The underlying graphs arise from the computation of morphological amoebas as structuring elements for adaptive morphology, either as weighted or unweighted Dijkstra search trees or as edge-weighted pixel graphs within structuring elements. In the present paper we focus on texture descriptors in whic…
Indian Institute of Technology Indore（印度理工学院印多尔分校）
Breast cancer is becoming pervasive with each passing day. Hence, its early detection is a big step in saving the life of any patient. Mammography is a common tool in breast cancer diagnosis. The most important step here is classification of mammogram patches as normal-abnormal and benign-malignant. Texture of a breast in a mammogram patch plays a significant role in these classifications. We propose a variation of Histogram of Gradients (HOG) and Gabor filter combination called Histogram of Oriented Texture (HOT) that exploits this fact. We also revisit the Pass Band - Discrete Cosine Transfo…
Geo-informatics and Space Technology Development Agency (GISTDA)（地理信息与空间技术发展局）
Traditional image classification techniques often produce unsatisfactory results when applied to high spatial resolution data because classes in high resolution images are not spectrally homogeneous. Texture offers an alternative source of information for classifying these images. This paper evaluates a recently developed, computationally simple texture metric called Weber Local Descriptor (WLD) for use in classifying high resolution QuickBird panchromatic data. We compared WLD with state-of-the art texture descriptors (TD) including Local Binary Pattern (LBP) and its rotation-invariant versio…
University of Oulu（奥卢大学）；Shenzhen University（深圳大学）
In this paper, we propose a novel local feature, called Local Orientation Adaptive Descriptor (LOAD), to capture regional texture in an image. In LOAD, we proposed to define point description on an Adaptive Coordinate System (ACS), adopt a binary sequence descriptor to capture relationships between one point and its neighbors and use multi-scale strategy to enhance the discriminative power of the descriptor. The proposed LOAD enjoys not only discriminative power to capture the texture information, but also has strong robustness to illumination variation and image rotation. Extensive experiment…
ETH Zurich（苏黎世联邦理工学院）；Sevensense Robotics AG（七感机器人股份公司）
Precisely estimating a robot's pose in a prior, global map is a fundamental capability for mobile robotics, e.g. autonomous driving or exploration in disaster zones. This task, however, remains challenging in unstructured, dynamic environments, where local features are not discriminative enough and global scene descriptors only provide coarse information. We therefore present SegMap: a map representation solution for localization and mapping based on the extraction of segments in 3D point clouds. Working at the level of segments offers increased invariance to view-point and local structural ch…
The present work proposes the development of a novel method to provide descriptors for colored texture images. The method consists in two steps. In the first, we apply a linear transform in the color space of the image aiming at highlighting spatial structuring relations among the color of pixels. In a second moment, we apply a multiscale approach to the calculus of fractal dimension based on Fourier transform. From this multiscale operation, we extract the descriptors used to discriminate the texture represented in digital images. The accuracy of the method is verified in the classification o…
Nowadays, 2D barcodes have been widely used for advertisement, mobile payment, and product authentication. However, in applications related to product authentication, an authentic 2D barcode can be illegally copied and attached to a counterfeited product in such a way to bypass the authentication scheme. In this paper, we employ a proprietary 2D barcode pattern and use multimedia forensics methods to analyse the scanning and printing artefacts resulting from the copy (rebroadcasting) attack. A diverse and complementary feature set is proposed to quantify the barcode texture distortions introdu…
University of Campinas（坎皮纳斯大学）；State University of Campinas（坎皮纳斯州立大学）
Texture recognition is one of the most important tasks in computer vision and, despite the recent success of learning-based approaches, there is still need for model-based solutions. This is especially the case when the amount of data available for training is not sufficiently large, a common situation in several applied areas, or when computational resources are limited. In this context, here we propose a method for texture descriptors that combines the representation power of complex objects by cellular automata with the known effectiveness of local descriptors in texture analysis. The metho…
Few-shot medical image classification with simple shape and texture text descriptors using vision-language models
RIKEN Center for Brain Science（理化学研究所脑科学中心）；Institute of Fundamental Technological Research, Polish Academy of Sciences（波兰科学院基础技术研究所）；Faculty of Computer Science, Universitas Indonesia（印度尼西亚大学计算机学院）
National Database And Registration Authority（国家数据库与注册局）
Minutia Cylinder Codes (MCC) are minutiae based fingerprint descriptors that take into account minutiae information in a fingerprint image for fingerprint matching. In this paper, we present a modification to the underlying information of the MCC descriptor and show that using different features, the accuracy of matching is highly affected by such changes. MCC originally being a minutia only descriptor is transformed into a texture descriptor. The transformation is from minutiae angular information to orientation, frequency and energy information using Short Time Fourier Transform (STFT) analy…
Indian Statistical Institute（印度统计研究所）；National Institute of Technology, Rourkela（鲁尔凯拉国家技术学院）；Indian Institute of Information Technology, Sri City（斯里城印度信息技术学院）
Methods based on local image features have recently shown promise for texture classification tasks, especially in the presence of large intra-class variation due to illumination, scale, and viewpoint changes. Inspired by the theories of image structure analysis, this paper presents a simple, efficient, yet robust descriptor namely local jet pattern (LJP) for texture classification. In this approach, a jet space representation of a texture image is derived from a set of derivatives of Gaussian (DtGs) filter responses up to second order, so called local jet vectors (LJV), which also satisfy the …
This paper presents an efficient method for texture retrieval using multiscale feature extraction and embedding based on the local extrema keypoints. The idea is to first represent each texture image by its local maximum and local minimum pixels. The image is then divided into regular overlapping blocks and each one is characterized by a feature vector constructed from the radiometric, geometric and structural information of its local extrema. All feature vectors are finally embedded into a covariance matrix which will be exploited for dissimilarity measurement within retrieval task. Thanks to…
Image X Institute, University of Sydney（悉尼大学 Image X 研究所）；Jiangxi University of Finance and Economics（江西财经大学）；Sany（三一集团）
The existing state-of-the-art point descriptor relies on structure information only, which omit the texture information. However, texture information is crucial for our humans to distinguish a scene part. Moreover, the current learning-based point descriptors are all black boxes which are unclear how the original points contribute to the final descriptor. In this paper, we propose a new multimodal fusion method to generate a point cloud registration descriptor by considering both structure and texture information. Specifically, a novel attention-fusion module is designed to extract the weighte…
Osaka University（大阪大学）；Japan Science and Technology Agency（日本科学技术振兴机构）
This paper proposes a multi-view extension of instance segmentation without relying on texture or shape descriptor matching. Multi-view instance segmentation becomes challenging for scenes with repetitive textures and shapes, e.g., plant leaves, due to the difficulty of multi-view matching using texture or shape descriptors. To this end, we propose a multi-view region matching method based on epipolar geometry, which does not rely on any feature descriptors. We further show that the epipolar region matching can be easily integrated into instance segmentation and effective for instance-wise 3D …
In this work, we propose a method for large-scale topological localization based on radar scan images using learned descriptors. We present a simple yet efficient deep network architecture to compute a rotationally invariant discriminative global descriptor from a radar scan image. The performance and generalization ability of the proposed method is experimentally evaluated on two large scale driving datasets: MulRan and Oxford Radar RobotCar. Additionally, we present a comparative evaluation of radar-based and LiDAR-based localization using learned global descriptors. Our code and trained mod…
In recent years, camera-based localization has been widely used for robotic applications, and most proposed algorithms rely on local features extracted from recorded images. For better performance, the features used for open-loop localization are required to be short-term globally static, and the ones used for re-localization or loop closure detection need to be long-term static. Therefore, the motion attribute of a local feature point could be exploited to improve localization performance, e.g., the feature points extracted from moving persons or vehicles can be excluded from these systems du…
Vision is one of the most important of the senses, and humans use it extensively during navigation. We evaluated different types of image and video frame descriptors that could be used to determine distinctive visual landmarks for localizing a person based on what is seen by a camera that they carry. To do this, we created a database containing over 3 km of video-sequences with ground-truth in the form of distance travelled along different corridors. Using this database, the accuracy of localization - both in terms of knowing which route a user is on - and in terms of position along a certain …
VRVis Center for Virtual Reality and Visualization（VRVis虚拟现实与可视化中心）；Braincon Handels-GmbH（Braincon贸易有限公司）；NOVA Medical School, Universidade Nova de Lisboa（新里斯本大学NOVA医学院）；Instituto de Medicina Molecular（分子医学研究所）；Centro Hospitalar Lisboa Ocidental（里斯本西部医疗中心）
The relationship between knee osteoarthritis progression and changes in tibial bone structure has long been recognized and various texture descriptors have been proposed to detect early osteoarthritis (OA) from radiographs. This work aims to investigate (1) femoral textures as an OA indicator and (2) the potential of entropy as a computationally efficient alternative to established texture descriptors. We design a robust semi-automatically placed layout for regions of interest (ROI), compute the Hurst coefficient and the entropy in each ROI, and employ statistical and machine learning methods …
Federal University of Mato Grosso do Sul（南马托格罗索联邦大学）；University of São Paulo（圣保罗大学）
Texture is an important visual attribute used to describe images. There are many methods available for texture analysis. However, they do not capture the details richness of the image surface. In this paper, we propose a new method to describe textures using the artificial crawler model. This model assumes that each agent can interact with the environment and each other. Since this swarm system alone does not achieve a good discrimination, we developed a new method to increase the discriminatory power of artificial crawlers, together with the fractal dimension theory. Here, we estimated the fr…
Despite the recent success of convolutional neural networks in texture recognition, model-based descriptors are still competitive, especially when we do not have access to large amounts of annotated data for training and the interpretation of the model is an important issue. Among the model-based approaches, fractal geometry has been one of the most popular, especially in biological applications. Nevertheless, fractals are part of a much broader family of models, which are the non-linear operators, studied in chaos theory. In this context, we propose here a chaos-based local descriptor for tex…
Location-aware applications play an increasingly critical role in everyday life. However, satellite-based localization (e.g., GPS) has limited accuracy and can be unusable in dense urban areas and indoors. We introduce an image-based global localization system that is accurate to a few millimeters and performs reliable localization both indoors and outside. The key idea is to capture and index distinctive local keypoints in ground textures. This is based on the observation that ground textures including wood, carpet, tile, concrete, and asphalt may look random and homogeneous, but all contain …
Robert Bosch GmbH（罗伯特·博世公司）；Norwegian Open AI Lab（挪威开放人工智能实验室）；NTNU（挪威科技大学）
Ground texture based vehicle localization using feature-based methods is a promising approach to achieve infrastructure-free high-accuracy localization. In this paper, we provide the first extensive evaluation of available feature extraction methods for this task, using separately taken image pairs as well as synthetic transformations. We identify AKAZE, SURF and CenSurE as best performing keypoint detectors, and find pairings of CenSurE with the ORB, BRIEF and LATCH feature descriptors to achieve greatest success rates for incremental localization, while SIFT stands out when considering sever…
The University of Western Australia（西澳大学）；Murdoch University（默多克大学）
We propose a novel concept to directly match feature descriptors extracted from RGB images, with feature descriptors extracted from 3D point clouds. We use this concept to localize the position and orientation (pose) of the camera of a query image in dense point clouds. We generate a dataset of matching 2D and 3D descriptors, and use it to train a proposed Descriptor-Matcher algorithm. To localize a query image in a point cloud, we extract 2D keypoints and descriptors from the query image. Then the Descriptor-Matcher is used to find the corresponding pairs 2D and 3D keypoints by matching the 2…
University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
A number of recent approaches have used deep convolutional neural networks (CNNs) to build texture representations. Nevertheless, it is still unclear how these models represent texture and invariances to categorical variations. This work conducts a systematic evaluation of recent CNN-based texture descriptors for recognition and attempts to understand the nature of invariances captured by these representations. First we show that the recently proposed bilinear CNN model [25] is an excellent general-purpose texture descriptor and compares favorably to other CNN-based descriptors on various text…
Federal University of Mato Grosso do Sul（南马托格罗索联邦大学）
A new method based on complex networks is proposed for color-texture analysis. The proposal consists on modeling the image as a multilayer complex network where each color channel is a layer, and each pixel (in each color channel) is represented as a network vertex. The network dynamic evolution is accessed using a set of modeling parameters (radii and thresholds), and new characterization techniques are introduced to capt information regarding within and between color channel spatial interaction. An automatic and adaptive approach for threshold selection is also proposed. We conduct classific…
Robert Bosch GmbH（罗伯特·博世有限公司）；Goethe University（歌德大学）；NTNU Trondheim（挪威科技大学特隆赫姆校区）；Norwegian Open AI Lab（挪威开放人工智能实验室）
A promising approach to accurate positioning of robots is ground texture based localization. It is based on the observation that visual features of ground images enable fingerprint-like place recognition. We tackle the issue of efficient parametrization of such methods, deriving a prediction model for localization performance, which requires only a small collection of sample images of an application area. In a first step, we examine whether the model can predict the effects of changing one of the most important parameters of feature-based localization methods: the number of extracted features.…
Landmark-based robot self-localization has recently garnered interest as a highly-compressive domain-invariant approach for performing visual place recognition (VPR) across domains (e.g., time of day, weather, and season). However, landmark-based self-localization can be an ill-posed problem for a passive observer (e.g., manual robot control), as many viewpoints may not provide an effective landmark view. In this study, we consider an active self-localization task by an active observer and present a novel reinforcement learning (RL)-based next-best-view (NBV) planner. Our contributions are as …
Shanghai Institute of Microsystem and Information Technology (SIMIT), Chinese Academy of Sciences (CAS)（中国科学院上海微系统与信息技术研究所（SIMIT））
The detection of road and free space remains challenging for non-flat plane, especially with the varying latitudinal and longitudinal slope or in the case of multi-ground plane. In this paper, we propose a framework of the ground plane detection with stereo vision. The main contribution of this paper is a newly proposed descriptor which is implemented in the disparity image to obtain a disparity texture image. The ground plane regions can be distinguished from their surroundings effectively in the disparity texture image. Because the descriptor is implemented in the local area of the image, it…
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；National Laboratory of Pattern Recognition（模式识别国家重点实验室）
Visual localization to compute 6DoF camera pose from a given image has wide applications such as in robotics, virtual reality, augmented reality, etc. Two kinds of descriptors are important for the visual localization. One is global descriptors that extract the whole feature from each image. The other is local descriptors that extract the local feature from each image patch usually enclosing a key point. More and more methods of the visual localization have two stages: at first to perform image retrieval by global descriptors and then from the retrieval feedback to make 2D-3D point corresponde…
ETH Zurich（苏黎世联邦理工学院）；The University of Queensland（昆士兰大学）；Arrival Ltd（Arrival有限公司）
Current descriptors for global localization often struggle under vast viewpoint or appearance changes. One possible improvement is the addition of topological information on semantic objects. However, handcrafted topological descriptors are hard to tune and not robust to environmental noise, drastic perspective changes, object occlusion or misdetections. To solve this problem, we formulate a learning-based approach by modelling semantically meaningful object constellations as graphs and using Deep Graph Convolution Networks to map a constellation to a descriptor. We demonstrate the effectivene…
INSA Centre Val de Loire（卢瓦尔河谷国立应用科学学院）；Université d’Orléans（奥尔良大学）；Laboratoire PRISME EA 4229（PRISME实验室EA 4229）
Texture is an important cue for different computer vision tasks and applications. Local Binary Pattern (LBP) is considered one of the best yet efficient texture descriptors. However, LBP has some notable limitations, mostly the sensitivity to noise. In this paper, we address these criteria by introducing a novel texture descriptor, Robust Adaptive Median Binary Pattern (RAMBP). RAMBP based on classification process of noisy pixels, adaptive analysis window, scale analysis and image regions median comparison. The proposed method handles images with high noisy textures, and increases the discrim…
Visual localization is an attractive problem that estimates the camera localization from database images based on the query image. It is a crucial task for various applications, such as autonomous vehicles, assistive navigation and augmented reality. The challenging issues of the task lie in various appearance variations between query and database images, including illumination variations, dynamic object variations and viewpoint variations. In order to tackle those challenges, Panoramic Annular Localizer into which panoramic annular lens and robust deep image descriptors are incorporated is pr…
École de Technologie Supérieure, University of Québec（魁北克大学高等技术学院）；Instituto Federal do Sul de Minas Gerais（南米纳斯吉拉斯联邦学院）；Pontificia Universidade Catolica do Paraná（巴拉那天主教大学）
Information from an image occurs over multiple and distinct spatial scales. Image pyramid multiresolution representations are a useful data structure for image analysis and manipulation over a spectrum of spatial scales. This paper employs the Gaussian-Laplacian pyramid to treat different spatial frequency bands of a texture separately. First, we generate three images corresponding to three levels of the Gaussian-Laplacian pyramid for an input image to capture intrinsic details. Then we aggregate features extracted from gray and color texture images using bio-inspired texture descriptors, info…
Many robotics applications require precise pose estimates despite operating in large and changing environments. This can be addressed by visual localization, using a pre-computed 3D model of the surroundings. The pose estimation then amounts to finding correspondences between 2D keypoints in a query image and 3D points in the model using local descriptors. However, computational power is often limited on robotic platforms, making this task challenging in large-scale environments. Binary feature descriptors significantly speed up this 2D-3D matching, and have become popular in the robotics comm…
Universidad Nacional Autónoma de México（墨西哥国立自治大学）；Instituto de Óptica（光学研究所）
In many image processing applications, such as segmentation and classification, the selection of robust features descriptors is crucial to improve the discrimination capabilities in real world scenarios. In particular, it is well known that image textures constitute power visual cues for feature extraction and classification. In the past few years the local binary pattern (LBP) approach, a texture descriptor method proposed by Ojala et al., has gained increased acceptance due to its computational simplicity and more importantly for encoding a powerful signature for describing textures. However…
ETH Zürich（苏黎世联邦理工学院）；University of Bonn（波恩大学）；Lund University（隆德大学）；Microsoft（微软）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
A distinctive representation of image patches in form of features is a key component of many computer vision and robotics tasks, such as image matching, image retrieval, and visual localization. State-of-the-art descriptors, from hand-crafted descriptors such as SIFT to learned ones such as HardNet, are usually high dimensional; 128 dimensions or even more. The higher the dimensionality, the larger the memory consumption and computational time for approaches using such descriptors. In this paper, we investigate multi-layer perceptrons (MLPs) to extract low-dimensional but high-quality descript…
This paper introduces a simple but highly efficient ensemble for robust texture classification, which can effectively deal with translation, scale and changes of significant viewpoint problems. The proposed method first inherits the spirit of spatial pyramid matching model (SPM), which is popular for encoding spatial distribution of local features, but in a flexible way, partitioning the original image into different levels and incorporating different overlapping patterns of each level. This flexible setup helps capture the informative features and produces sufficient local feature codes by so…
