The primary challenge for any autonomous system operating in realistic, rather unconstrained scenarios is to manage the complexity and uncertainty of the real world. While it is unclear how exactly humans and other higher animals master these problems, it seems evident, that abstraction plays an important role. The use of abstract concepts allows to define the system behavior on higher levels. In this paper we focus on the semantic mapping of indoor environments. We propose a method to extract an abstracted floor plan from typical grid maps using Bayesian reasoning. The result of this procedure is a probabilistic generative model of the environment defined over abstract concepts. It is well suited for higher-level reasoning and communication purposes. We demonstrate the effectiveness of the approach using real-world data.
We present a new method of learning a continuous occupancy field for use in robot navigation. Occupancy grid maps, or variants of, are possibly the most widely used and accepted method of building a map of a robot's environment. Various methods have been developed to learn continuous occupancy maps and have successfully resolved many of the shortcomings of grid mapping, namely, priori discretisation and spatial correlation. However, most methods for producing a continuous occupancy field remain computationally expensive or heuristic in nature. Our method explores a generalisation of the so-cal…
School of Electrical Engineering, Aalto University（阿尔托大学电气工程学院）
Most mobile robots for indoor use rely on 2D laser scanners for localization, mapping and navigation. These sensors, however, cannot detect transparent surfaces or measure the full occupancy of complex objects such as tables. Deep Neural Networks have recently been proposed to overcome this limitation by learning to estimate object occupancy. These estimates are nevertheless subject to uncertainty, making the evaluation of their confidence an important issue for these measures to be useful for autonomous navigation and mapping. In this work we approach the problem from two sides. First we disc…
Institute for Experiential Robotics, Northeastern University（东北大学体验机器人研究所）；Khoury College of Computer Sciences, Northeastern University（东北大学胡里计算机学院）；Department of Electrical and Computer Engineering, Northeastern University（东北大学电气与计算机工程系）
Obstacle detection is a safety-critical problem in robot navigation, where stereo matching is a popular vision-based approach. While deep neural networks have shown impressive results in computer vision, most of the previous obstacle detection works only leverage traditional stereo matching techniques to meet the computational constraints for real-time feedback. This paper proposes a computationally efficient method that employs a deep neural network to detect occupancy from stereo images directly. Instead of learning the point cloud correspondence from the stereo data, our approach extracts t…
The Chinese University of Hong Kong（香港中文大学）；The University of British Columbia（不列颠哥伦比亚大学）；National Institute of Advanced Industrial Science and Technology（产业技术综合研究所）
Robots are increasingly operating in indoor environments designed for and shared with people. However, robots working safely and autonomously in uneven and unstructured environments still face great challenges. Many modern indoor environments are designed with wheelchair accessibility in mind. This presents an opportunity for wheeled robots to navigate through sloped areas while avoiding staircases. In this paper, we present an integrated software and hardware system for autonomous mobile robot navigation in uneven and unstructured indoor environments. This modular and reusable software framew…
Monocular Occupancy Prediction for Scalable Indoor Scenes
School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；Centre for Artificial Intelligence and Robotics (HKISI_CAS)（人工智能与机器人中心）
FLM-Occ: Feed-forward Likelihood Maximization for Efficient Indoor Occupancy Prediction
Southern University of Science and Technology（南方科技大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen)（广东省人工智能与数字经济实验室（深圳））；Shenzhen Key Laboratory of Robotics and Computer Vision（深圳市机器人视觉与感知重点实验室）
Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）；Beijing Institute of Technology（北京理工大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Robust Dynamic Object Detection in Cluttered Indoor Scenes via Learned Spatiotemporal Cues
Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente（代尔夫特理工大学电子工程、数学与计算机科学学院）；Department of Electrical Engineering, Wright State University（怀特塞德大学电子工程系）；Massachusetts Institute of Technology（麻省理工学院）
Iowa State University（爱荷华州立大学）；University of Washington（华盛顿大学）
Reliable detection of human occupancy in indoor environments is critical for various energy efficiency, security, and safety applications. We consider this challenge of occupancy detection using extremely low-quality, privacy-preserving images from low power image sensors. We propose a combined few shot learning and clustering algorithm to address this challenge that has very low commissioning and maintenance cost. While the few shot learning concept enables us to commission our system with a few labeled examples, the clustering step serves the purpose of online adaptation to changing imaging …
A key proficiency an autonomous mobile robot must have to perform high-level tasks is a strong understanding of its environment. This involves information about what types of objects are present, where they are, what their spatial extend is, and how they can be reached, i.e., information about free space is also crucial. Semantic maps are a powerful instrument providing such information. However, applying semantic segmentation and building 3D maps with high spatial resolution is challenging given limited resources on mobile robots. In this paper, we incorporate semantic information into effici…
Group-wise Supervision with Focal-Dice Loss for Long-Tailed Indoor Semantic Occupancy Prediction
College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Guangdong Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）
GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory
The Hong Kong Polytechnic University（香港理工大学）；Eastern Institute of Technology（东部理工学院）；Shanghai Jiao Tong University（上海交通大学）；Wuhan University（武汉大学）；The University of Hong Kong（香港大学）；Georgia Institute of Technology（佐治亚理工学院）
Scene text is an important feature to be extracted, especially in vision-based mobile robot navigation as many potential landmarks such as nameplates and information signs contain text. In this paper, a novel two-step text localization method for Indoor Mobile Robot Navigation is introduced. This method is based on morphological operators and machine learning techniques and can be used in real time environments. The proposed method has two steps. At First, a new set of morphological operators is applied with a particular sequence to extract high contrast areas that have high probability of tex…
Experimental study on surveillance video-based indoor occupancy measurement with occupant-centric control
Center for Intelligent and Networked Systems, Department of Automation, BNRist, Tsinghua University（清华大学自动化系智能与网络化系统中心，北京信息科学与技术国家研究中心）；Urban Mobility Lab, Massachusetts Institute of Technology（麻省理工学院城市移动实验室）；Singapore-MIT Alliance for Research and Technology Centre, Massachusetts Institute of Technology（麻省理工学院新加坡-麻省理工学院研究与技术联盟中心）
College of AI, Tsinghua University（清华大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；AIR, Tsinghua University（清华大学人工智能研究院）；TARS；College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院）
University of Luxembourg（卢森堡大学）；Universidad de Zaragoza（萨拉戈萨大学）
Mobile robots extract information from its environment to understand their current situation to enable intelligent decision making and autonomous task execution. In our previous work, we introduced the concept of Situation Graphs (S-Graphs) which combines in a single optimizable graph, the robot keyframes and the representation of the environment with geometric, semantic and topological abstractions. Although S-Graphs were built and optimized in real-time and demonstrated state-of-the-art results, they are limited to specific structured environments with specific hand-tuned dimensions of rooms…
EKF-Based Fusion of Wi-Fi/LiDAR/IMU for Indoor Localization and Navigation
School of Advanced Technology, Xi'an Jiaotong-Liverpool University（西安交通大学利物浦大学先进技术学院）；Department of Electrical Engineering and Electronics, University of Liverpool（利物浦大学电子工程系）；Zhejiang University of Technology（浙江工业大学）；School of Artificial Intelligence, Suzhou Vocational Institute of Industrial Technology（苏州工业职业技术学院人工智能学院）
EmbodiedOcc++: Boosting Embodied 3D Occupancy Prediction with Plane Regularization and Uncertainty Sampler
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；School of electronic science and engineering, Nanjing University（电子科学与工程学院，南京大学）；Berkeley Artificial Intelligence Research Lab, Department of EECS, University of California, Berkeley（伯克利人工智能研究实验室，电子工程与计算机科学系，加州大学伯克利分校）
University of Zurich（苏黎世大学）；ETH Zürich（苏黎世联邦理工学院）；University College London（伦敦大学学院）；Adobe Research（奥多比研究中心）
Recent years have seen a proliferation of new digital products for the efficient management of indoor spaces, with important applications like emergency management, virtual property showcasing and interior design. These products rely on accurate 3D models of the environments considered, including information on both architectural and non-permanent elements. These models must be created from measured data such as RGB-D images or 3D point clouds, whose capture and consolidation involves lengthy data workflows. This strongly limits the rate at which 3D models can be produced, preventing the adopt…
Today's mobile robots are expected to operate in complex environments they share with humans. To allow intuitive human-robot collaboration, robots require a human-like understanding of their surroundings in terms of semantically classified instances. In this paper, we propose a new approach for deriving such instance-based semantic maps purely from occupancy grids. We employ a combination of deep learning techniques to detect, segment and extract door hypotheses from a random-sized map. The extraction is followed by a post-processing chain to further increase the accuracy of our approach, as w…
We present a novel dataset for training and benchmarking semantic SLAM methods. The dataset consists of 200 long sequences, each one containing 3000-5000 data frames. We generate the sequences using realistic home layouts. For that we sample trajectories that simulate motions of a simple home robot, and then render the frames along the trajectories. Each data frame contains a) RGB images generated using physically-based rendering, b) simulated depth measurements, c) simulated IMU readings and d) ground truth occupancy grid of a house. Our dataset serves a wider range of purposes compared to ex…
OccScene: Semantic Occupancy-based Cross-task Mutual Learning for 3D Scene Generation
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（电子信息与电气工程学院，上海交通大学）；Ningbo Institute of Digital Twin, Eastern Institute of Technology（宁波数字孪生研究所，东部技术研究所）；PhiGent Robotics, Beijing, China（PhiGent Robotics，北京，中国）
University of São Paulo（圣保罗大学）；Toyota Research Institute（丰田研究院）
This paper addresses the problem of single image depth estimation (SIDE), focusing on improving the quality of deep neural network predictions. In a supervised learning scenario, the quality of predictions is intrinsically related to the training labels, which guide the optimization process. For indoor scenes, structured-light-based depth sensors (e.g. Kinect) are able to provide dense, albeit short-range, depth maps. On the other hand, for outdoor scenes, LiDARs are considered the standard sensor, which comparatively provides much sparser measurements, especially in areas further away. Rather…
Indoor image features extraction is a fundamental problem in multiple fields such as image processing, pattern recognition, robotics and so on. Nevertheless, most of the existing feature extraction methods, which extract features based on pixels, color, shape/object parts or objects on images, suffer from limited capabilities in describing semantic information (e.g., object association). These techniques, therefore, involve undesired classification performance. To tackle this issue, we propose the notion of high-level semantic features and design four steps to extract them. Specifically, we fi…
University of New South Wales（新南威尔士大学）；Agriculture Victoria（维多利亚州农业局）
Spatially and geometrically accurate laser scans are essential in modelling infrastructure for applications in civil, mining and transportation. Monitoring of underground or indoor environments such as mines or tunnels is challenging due to unavailability of a sensor positioning framework, complicated structurally symmetric layouts, repetitive features and occlusions. Current practices largely include a manual selection of discernable reference points for georeferencing and coregistration purpose. This study aims at overcoming these practical challenges in underground or indoor laser scanning.…
Normal Transformer: Extracting Surface Geometry from LiDAR Points Enhanced by Visual Semantics
School of Computer Science, Australian Artificial Intelligence Institute (AAII), University of Technology Sydney（悉尼科技大学计算机科学学院、澳大利亚人工智能研究所）；AI Department of SunnyWay Tech LLC（SunnyWay科技有限责任公司人工智能部门）；Autonavi of Alibaba Group（阿里巴巴集团高德（高德是阿里巴巴旗下的高德，此处按原文提取））
