This paper addresses a multi-robot planning problem in environments with partially unknown semantics. The environment is assumed to have known geometric structure (e.g., walls) and to be occupied by static labeled landmarks with uncertain positions and classes. This modeling approach gives rise to an uncertain semantic map generated by semantic SLAM algorithms. Our goal is to design control policies for robots equipped with noisy perception systems so that they can accomplish collaborative tasks captured by global temporal logic specifications. To specify missions that account for environmental and perceptual uncertainty, we employ a fragment of Linear Temporal Logic (LTL), called co-safe LTL, defined over perception-based atomic predicates modeling probabilistic satisfaction requirements. The perception-based LTL planning problem gives rise to an optimal control problem, solved by a novel sampling-based algorithm, that generates open-loop control policies that are updated online to adapt to a continuously learned semantic map. We provide extensive experiments to demonstrate the efficiency of the proposed planning architecture.
University of Perugia（佩鲁贾大学）；University of Zurich（苏黎世大学）
In this paper, we give a double twist to the problem of planning under uncertainty. State-of-the-art planners seek to minimize the localization uncertainty by only considering the geometric structure of the scene. In this paper, we argue that motion planning for vision-controlled robots should be perception aware in that the robot should also favor texture-rich areas to minimize the localization uncertainty during a goal-reaching task. Thus, we describe how to optimally incorporate the photometric information (i.e., texture) of the scene, in addition to the the geometric one, to compute the un…
Bridging Perception and Planning: Towards End-to-End Planning for Signal Temporal Logic Tasks
School of Automation & Intelligent Sensing, Shanghai Jiao Tong University（自动化与智能感知学院，上海交通大学）；University of Minnesota, Twin Cities（明尼苏达大学，双城分校）；School of Information Science and Technology, University of Science and Technology of China（信息科学与技术学院，中国科学技术大学）
Istanbul Technical University（伊斯坦布尔理工大学）；Turkish-German University（土耳其-德国大学）
We investigate how to utilize predictive models for selecting appropriate motion planning strategies based on perception uncertainty estimation for agile unmanned aerial vehicle (UAV) navigation tasks. Although there are variety of motion planning and perception algorithms for such tasks, the impact of perception uncertainty is not explicitly handled in many of the current motion algorithms, which leads to performance loss in real-life scenarios where the measurement are often noisy due to external disturbances. We develop a novel framework for embedding perception uncertainty to high level mo…
University of California at Berkeley（加州大学伯克利分校）；California PATH, University of California at Berkeley（加州大学伯克利分校加州PATH）
Autonomous cars have to navigate in dynamic environment which can be full of uncertainties. The uncertainties can come either from sensor limitations such as occlusions and limited sensor range, or from probabilistic prediction of other road participants, or from unknown social behavior in a new area. To safely and efficiently drive in the presence of these uncertainties, the decision-making and planning modules of autonomous cars should intelligently utilize all available information and appropriately tackle the uncertainties so that proper driving strategies can be generated. In this paper, …
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
In this paper we provide an overview of a new framework for robot perception, real-world modelling, and navigation that uses a stochastic tesselated representation of spatial information called the Occupancy Grid. The Occupancy Grid is a multi-dimensional random field model that maintains probabilistic estimates of the occupancy state of each cell in a spatial lattice. Bayesian estimation mechanisms employing stochastic sensor models allow incremental updating of the Occupancy Grid using multi-view, multi-sensor data, composition of multiple maps, decision-making, and incorporation of robot an…
Gaussian Mixture-Based Inverse Perception Contract for Uncertainty-Aware Robot Navigation
Department of Computer Science, Columbia University（计算机科学系，哥伦比亚大学）；Department of Computer Science and Engineering, Texas A&M University（计算机科学与工程系，德克萨斯大学）
Dalle Molle Institute for Artificial Intelligence (IDSIA)（达莱·莫勒人工智能研究所（IDSIA））；USI-SUPSI（提契诺大学-瑞士南方应用科学与艺术大学）
We propose a general self-supervised learning approach for spatial perception tasks, such as estimating the pose of an object relative to the robot, from onboard sensor readings. The model is learned from training episodes, by relying on: a continuous state estimate, possibly inaccurate and affected by odometry drift; and a detector, that sporadically provides supervision about the target pose. We demonstrate the general approach in three different concrete scenarios: a simulated robot arm that visually estimates the pose of an object of interest; a small differential drive robot using 7 infra…
Uncertainty-Informed Active Perception for Open Vocabulary Object Goal Navigation
Center for Robotics, University of Bonn（波恩大学机器人中心）；MAVLab, TU Delft（代尔夫特理工大学MAVLab）；Lamarr Institute for Machine Learning and Artificial Intelligence（机器学习与人工智能拉马尔研究所）
Virginia Tech（弗吉尼亚理工大学）；University of Maryland（马里兰大学）
We propose a risk-aware framework for multi-robot, multi-demand assignment and planning in unknown environments. Our motivation is disaster response and search-and-rescue scenarios where ground vehicles must reach demand locations as soon as possible. We consider a setting where the terrain information is available only in the form of an aerial, georeferenced image. Deep learning techniques can be used for semantic segmentation of the aerial image to create a cost map for safe ground robot navigation. Such segmentation may still be noisy. Hence, we present a joint planning and perception frame…
Integrating Legal and Logical Specifications in Perception, Prediction, and Planning for Automated Driving: A Survey of Methods
Department of Computer Science and Mathematics, Freie Universität Berlin（自由大学柏林计算机科学与数学系）；Continental Automotive（大陆汽车）；Computation, Information and Technology, Technical University of Munich（慕尼黑技术大学计算、信息与技术系）
Lehigh University（利哈伊大学）；Eindhoven University of Technology（埃因霍温理工大学）
This work presents a step towards utilizing incrementally-improving symbolic perception knowledge of the robot's surroundings for provably correct reactive control synthesis applied to an autonomous driving problem. Combining abstract models of motion control and information gathering, we show that assume-guarantee specifications (a subclass of Linear Temporal Logic) can be used to define and resolve traffic rules for cautious planning. We propose a novel representation called symbolic refinement tree for perception that captures the incremental knowledge about the environment and embodies the…
Credibility-Aware Learning and Control for Safe USV Navigation under Perception Uncertainty
School of Information Engineering, Henan University of Science and Technology（河南科技大学信息工程学院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；School of Logistics Engineering, Shanghai Maritime University（上海 Maritime University物流工程学院）；Shenzhen Technology University（深圳科技大学）；School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Software Engineering, Tongji University（同济大学软件工程学院）
Montreal Robotics and Embodied AI Lab（蒙特利尔机器人与具身人工智能实验室）；Mila（米拉人工智能研究所）；Université de Montréal（蒙特利尔大学）；Denso（电装公司）
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainti…
This work focuses on improving uncertainty estimation in the field of object classification from RGB images and demonstrates its benefits in two robotic applications. We employ a (BNN), and evaluate two practical inference techniques to obtain better uncertainty estimates, namely Concrete Dropout (CDP) and Kronecker-factored Laplace Approximation (LAP). We show a performance increase using more reliable uncertainty estimates as unary potentials within a Conditional Random Field (CRF), which is able to incorporate contextual information as well. Furthermore, the obtained uncertainties are explo…
GE Research（通用电气研究院）；Motional（Motional公司）；GE Digital（通用电气数字部门）
Off-road autonomous unmanned ground vehicles (UGVs) are being developed for military and commercial use to deliver crucial supplies in remote locations, help with mapping and surveillance, and to assist war-fighters in contested environments. Due to complexity of the off-road environments and variability in terrain, lighting conditions, diurnal and seasonal changes, the models used to perceive the environment must handle a lot of input variability. Current datasets used to train perception models for off-road autonomous navigation lack of diversity in seasons, locations, semantic classes, as w…
California Institute of Technology（加州理工学院）；Iowa State University（爱荷华州立大学）
In many autonomy applications, performance of perception algorithms is important for effective planning and control. In this paper, we introduce a framework for computing the probability of satisfaction of formal system specifications given a confusion matrix, a statistical average performance measure for multi-class classification. We define the probability of satisfaction of a linear temporal logic formula given a specific initial state of the agent and true state of the environment. Then, we present an algorithm to construct a Markov chain that represents the system behavior under the compo…
Uncertainty Quantification of Set-Membership Estimation in Control and Perception: Revisiting the Minimum Enclosing Ellipsoid
Princeton University（普林斯顿大学）；LAAS-CNRS（法国国家科学研究中心LAAS实验室）；Toulouse School of Economics (TSE)（图卢兹经济学院）；School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）
We are witnessing significant progress on perception models, specifically those trained on large-scale internet images. However, efficiently generalizing these perception models to unseen embodied tasks is insufficiently studied, which will help various relevant applications (e.g., home robots). Unlike static perception methods trained on pre-collected images, the embodied agent can move around in the environment and obtain images of objects from any viewpoints. Therefore, efficiently learning the exploration policy and collection method to gather informative training samples is the key to thi…
Hyper-V2X: Hypernetworks for Estimating Epistemic and Aleatoric Uncertainty in Cooperative Bird's-Eye-View Semantic Segmentation
CARISSMA Institute for Electric, COnnected, and Secure Mobility (C-ECOS), Technische Hochschule Ingolstadt（CARISSMA电动、连接与安全移动研究所（C-ECOS）、因戈尔施塔特技术大学）；University of Applied Sciences Aschaffenburg（阿施发堡应用科学大学）
Uncertainties in Deep Neural Network (DNN)-based perception and vehicle's motion pose challenges to the development of safe autonomous driving vehicles. In this paper, we propose a safe motion planning framework featuring the quantification and propagation of DNN-based perception uncertainties and motion uncertainties. Contributions of this work are twofold: (1) A Bayesian Deep Neural network model which detects 3D objects and quantitatively captures the associated aleatoric and epistemic uncertainties of DNNs; (2) An uncertainty-aware motion planning algorithm (PU-RRT) that accounts for uncer…
Uppsala University（乌普萨拉大学）；Washington University（华盛顿大学）；KTH Royal Institute of Technology（KTH皇家理工学院）
We develop an algorithm for the motion and task planning of a system comprised of multiple robots and unactuated objects under tasks expressed as Linear Temporal Logic (LTL) constraints. The robots and objects evolve subject to uncertain dynamics in an obstacle-cluttered environment. The key part of the proposed solution is the intelligent construction of a coupled transition system that encodes the motion and tasks of the robots and the objects. We achieve such a construction by designing appropriate adaptive control protocols in the lower level, which guarantee the safe robot navigation/obje…
Most real-world 3D sensors such as LiDARs perform fixed scans of the entire environment, while being decoupled from the recognition system that processes the sensor data. In this work, we propose a method for 3D object recognition using light curtains, a resource-efficient controllable sensor that measures depth at user-specified locations in the environment. Crucially, we propose using prediction uncertainty of a deep learning based 3D point cloud detector to guide active perception. Given a neural network's uncertainty, we derive an optimization objective to place light curtains using the pr…
