Visual localization in large and complex indoor scenes, dominated by weakly textured rooms and repeating geometric patterns, is a challenging problem with high practical relevance for applications such as Augmented Reality and robotics. To handle the ambiguities arising in this scenario, a common strategy is, first, to generate multiple estimates for the camera pose from which a given query image was taken. The pose with the largest geometric consistency with the query image, e.g., in the form of an inlier count, is then selected in a second stage. While a significant amount of research has concentrated on the first stage, there is considerably less work on the second stage. In this paper, we thus focus on pose verification. We show that combining different modalities, namely appearance, geometry, and semantics, considerably boosts pose verification and consequently pose accuracy. We develop multiple hand-crafted as well as a trainable approach to join into the geometric-semantic verification and show significant improvements over state-of-the-art on a very challenging indoor dataset.
University of Bristol（布里斯托大学）；Zhejiang University（浙江大学）
We describe a novel approach to indoor place recognition from RGB point clouds based on aggregating low-level colour and geometry features with high-level implicit semantic features. It uses a 2-stage deep learning framework, in which the first stage is trained for the auxiliary task of semantic segmentation and the second stage uses features from layers in the first stage to generate discriminate descriptors for place recognition. The auxiliary task encourages the features to be semantically meaningful, hence aggregating the geometry and colour in the RGB point cloud data with implicit semant…
Tokyo Institute of Technology（东京工业大学）；ETH Zürich（苏黎世联邦理工学院）；CIIRC, CTU in Prague（布拉格捷克理工大学信息学、机器人学与网络学捷克研究所）；Inria（法国国家信息与自动化研究所）；Microsoft（微软）
We seek to predict the 6 degree-of-freedom (6DoF) pose of a query photograph with respect to a large indoor 3D map. The contributions of this work are three-fold. First, we develop a new large-scale visual localization method targeted for indoor environments. The method proceeds along three steps: (i) efficient retrieval of candidate poses that ensures scalability to large-scale environments, (ii) pose estimation using dense matching rather than local features to deal with textureless indoor scenes, and (iii) pose verification by virtual view synthesis to cope with significant changes in viewp…
Université Bourgogne Franche-Comté（勃艮第-弗朗什-孔泰大学）；Instituto de Investigación en Ingeniería de Aragón (I3A)（阿拉贡工程研究院（I3A））；Universidad de Zaragoza（萨拉戈萨大学）
In the last few years, there has been a growing interest in taking advantage of the 360 panoramic images potential, while managing the new challenges they imply. While several tasks have been improved thanks to the contextual information these images offer, object recognition in indoor scenes still remains a challenging problem that has not been deeply investigated. This paper provides an object recognition system that performs object detection and semantic segmentation tasks by using a deep learning model adapted to match the nature of equirectangular images. From these results, instance segm…
CIIRC - Czech Institute of Informatics, Robotics and Cybernetics（捷克信息学、机器人学与控制论研究所（CIIRC））；Czech Technical University in Prague（捷克布拉格技术大学）；Faculty of Electrical Engineering, Czech Technical University in Prague（捷克布拉格技术大学电气工程学院）
Most state-of-the-art localization algorithms rely on robust relative pose estimation and geometry verification to obtain moving object agnostic camera poses in complex indoor environments. However, this approach is prone to mistakes if a scene contains repetitive structures, e.g., desks, tables, boxes, or moving people. We show that the movable objects incorporate non-negligible localization error and present a new straightforward method to predict the six-degree-of-freedom (6DoF) pose more robustly. We equipped the localization pipeline InLoc with real-time instance segmentation network YOLA…
Future advancements in robot autonomy and sophistication of robotics tasks rest on robust, efficient, and task-dependent semantic understanding of the environment. Semantic segmentation is the problem of simultaneous segmentation and categorization of a partition of sensory data. The majority of current approaches tackle this using multi-class segmentation and labeling in a Conditional Random Field (CRF) framework or by generating multiple object hypotheses and combining them sequentially. In practical settings, the subset of semantic labels that are needed depend on the task and particular sc…
Visual Place Recognition (VPR) is generally concerned with localizing outdoor images. However, localizing indoor scenes that contain part of an outdoor scene can be of large value for a wide range of applications. In this paper, we introduce Inside Out Visual Place Recognition (IOVPR), a task aiming to localize images based on outdoor scenes visible through windows. For this task we present the new large-scale dataset Amsterdam-XXXL, with images taken in Amsterdam, that consists of 6.4 million panoramic street-view images and 1000 user-generated indoor queries. Additionally, we introduce a new…
Moscow Institute of Physics and Technology（莫斯科物理技术学院）；AIRI (Artificial Intelligence Research Institute)（AIRI（人工智能研究院））；LLC Integrant（Integrant有限责任公司）；Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences（俄罗斯科学院“计算机科学与控制”联邦研究中心）
We present a novel dataset named as HPointLoc, specially designed for exploring capabilities of visual place recognition in indoor environment and loop detection in simultaneous localization and mapping. The loop detection sub-task is especially relevant when a robot with an on-board RGB-D camera can drive past the same place (``Point") at different angles. The dataset is based on the popular Habitat simulator, in which it is possible to generate photorealistic indoor scenes using both own sensor data and open datasets, such as Matterport3D. To study the main stages of solving the place recogn…
Plug to Place: Indoor Multimedia Geolocation from Electrical Sockets for Digital Investigation
Forensics and Security Research Group, School of Computer Science（犯罪学与安全研究组，计算机科学学院）；School of Engineering（工程学院）；Case Western Reserve University（凯斯西储大学）
GOPI: Generation-Oriented 3D Pose Inference for Furniture Insertion from Single-View RGB-D Indoor Scenes
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；X-Era AI Lab（X-Era AI实验室）
In this paper, we present a comprehensive study on the utility of deep convolutional neural networks with two state-of-the-art pooling layers which are placed after convolutional layers and fine-tuned in an end-to-end manner for visual place recognition task in challenging conditions, including seasonal and illumination variations. We compared extensively the performance of deep learned global features with three different loss functions, e.g. triplet, contrastive and ArcFace, for learning the parameters of the architectures in terms of fraction of the correct matches during deployment. To ver…
IIIT Hyderabad（海得拉巴国际信息技术研究所）；Queensland University of Technology（昆士兰科技大学）
Significant advances have been made recently in Visual Place Recognition (VPR), feature correspondence, and localization due to the proliferation of deep-learning-based methods. However, existing approaches tend to address, partially or fully, only one of two key challenges: viewpoint change and perceptual aliasing. In this paper, we present novel research that simultaneously addresses both challenges by combining deep-learned features with geometric transformations based on reasonable domain assumptions about navigation on a ground-plane, whilst also removing the requirement for specialized h…
University of Maryland, College Park（马里兰大学帕克分校）；Department of Electrical and Computer Engineering, University of Maryland, College Park（马里兰大学帕克分校电气与计算机工程系）；Center for Automation Research, University of Maryland, College Park（马里兰大学帕克分校自动化研究中心）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Covariates are factors that have a debilitating influence on face verification performance. In this paper, we comprehensively study two covariate related problems for unconstrained face verification: first, how covariates affect the performance of deep neural networks on the large-scale unconstrained face verification problem; second, how to utilize covariates to improve verification performance. To study the first problem, we implement five state-of-the-art deep convolutional networks (DCNNs) for face verification and evaluate them on three challenging covariates datasets. In total, seven cov…
Chongqing Technology and Business University（重庆工商大学）；Zhejiang University（浙江大学）
With the expansive aging of global population, service robot with living assistance applied in indoor scenes will serve as a crucial role in the field of elderly care and health in the future. Service robots need to detect multiple targets when completing auxiliary tasks. However, indoor scenes are usually complex and there are many types of interference factors, leading to great challenges in the multiple targets detection. To overcome this technical difficulty, a novel improved Mask RCNN method for multiple targets detection in the indoor complex scenes is proposed in this paper. The improve…
SVS College of Engineering and Technology（SVS工程技术学院）；SSK College of Engineering & Technology（SSK工程技术学院）
In the beginning stage, face verification is done using easy method of geometric algorithm models, but the verification route has now developed into a scientific progress of complicated geometric representation and identical procedure. In recent years the technologies have boosted face recognition system into the healthy focus. Researchers currently undergoing strong research on finding face recognition system for wider area information taken under hysterical elucidation dissimilarity. The proposed face recognition system consists of a narrative expositionindiscreet preprocessing method, a hyb…
Learning Geometric and Photometric Features from Panoramic LiDAR Scans for Outdoor Place Categorization
Graduate School of Information Science and Electrical Engineering, Kyushu University（九州大学信息科学与电子工程研究生院）；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）；Faculty of Information Science and Electrical Engineering, Kyushu University（九州大学信息科学与电子工程学部）；Technical University of Cartagena（卡塔赫纳技术大学）
University of Essex（埃塞克斯大学）；Queensland University of Technology（昆士兰科技大学）
This paper presents a cognition-inspired agnostic framework for building a map for Visual Place Recognition. This framework draws inspiration from human-memorability, utilizes the traditional image entropy concept and computes the static content in an image; thereby presenting a tri-folded criterion to assess the 'memorability' of an image for visual place recognition. A dataset namely 'ESSEX3IN1' is created, composed of highly confusing images from indoor, outdoor and natural scenes for analysis. When used in conjunction with state-of-the-art visual place recognition methods, the proposed fra…
The paper presents an approach to indoor personal localization on a mobile device based on visual place recognition. We implemented on a smartphone two state-of-the-art algorithms that are representative to two different approaches to visual place recognition: FAB-MAP that recognizes places using individual images, and ABLE-M that utilizes sequences of images. These algorithms are evaluated in environments of different structure, focusing on problems commonly encountered when a mobile device camera is used. The conclusions drawn from this evaluation are guidelines to design the FastABLE system…
Washington University in St. Louis（圣路易斯华盛顿大学）；University of Massachusetts at Amherst（马萨诸塞大学阿默斯特分校）
Modern AI tools, such as generative adversarial networks, have transformed our ability to create and modify visual data with photorealistic results. However, one of the deleterious side-effects of these advances is the emergence of nefarious uses in manipulating information in visual data, such as through the use of deep fakes. We propose a novel architecture for preserving the provenance of semantic information in images to make them less susceptible to deep fake attacks. Our architecture includes semantic signing and verification steps. We apply this architecture to verifying two types of se…
