This work presents an embodied agent that can adapt its semantic segmentation network to new indoor environments in a fully autonomous way. Because semantic segmentation networks fail to generalize well to unseen environments, the agent collects images of the new environment which are then used for self-supervised domain adaptation. We formulate this as an informative path planning problem, and present a novel information gain that leverages uncertainty extracted from the semantic model to safely collect relevant data. As domain adaptation progresses, these uncertainties change over time and the rapid learning feedback of our system drives the agent to collect different data. Experiments show that our method adapts to new environments faster and with higher final performance compared to an exploration objective, and can successfully be deployed to real-world environments on physical robots.
Delft University of Technology（代尔夫特理工大学）；ETH Zurich（苏黎世联邦理工学院）
This paper presents an online informative path planning approach for active information gathering on three-dimensional surfaces using aerial robots. Most existing works on surface inspection focus on planning a path offline that can provide full coverage of the surface, which inherently assumes the surface information is uniformly distributed hence ignoring potential spatial correlations of the information field. In this paper, we utilize manifold Gaussian processes (mGPs) with geodesic kernel functions for mapping surface information fields and plan informative paths online in a receding hori…
This technical report is an extended version of the paper 'A Receding Horizon Algorithm for Informative Path Planning with Temporal Logic Constraints' accepted to the 2013 IEEE International Conference on Robotics and Automation (ICRA). This paper considers the problem of finding the most informative path for a sensing robot under temporal logic constraints, a richer set of constraints than have previously been considered in information gathering. An algorithm for informative path planning is presented that leverages tools from information theory and formal control synthesis, and is proven to …
Autonomous Systems Lab. (ASL), ETH Zürich（苏黎世联邦理工学院自主系统实验室）；TU Delft（代尔夫特理工大学）
Target search with unmanned aerial vehicles (UAVs) is relevant problem to many scenarios, e.g., search and rescue (SaR). However, a key challenge is planning paths for maximal search efficiency given flight time constraints. To address this, we propose the Obstacle-aware Adaptive Informative Path Planning (OA-IPP) algorithm for target search in cluttered environments using UAVs. Our approach leverages a layered planning strategy using a Gaussian Process (GP)-based model of target occupancy to generate informative paths in continuous 3D space. Within this framework, we introduce an adaptive rep…
In this paper, we introduce an informative path planning (IPP) framework for active classification using unmanned aerial vehicles (UAVs). Our algorithm uses a combination of global viewpoint selection and evolutionary optimization to refine the planned trajectory in continuous 3D space while satisfying dynamic constraints. Our approach is evaluated on the application of weed detection for precision agriculture. We model the presence of weeds on farmland using an occupancy grid and generate adaptive plans according to information-theoretic objectives, enabling the UAV to gather data efficiently…
This paper reports on developing an integrated framework for safety-aware informative motion planning suitable for legged robots. The information-gathering planner takes a dense stochastic map of the environment into account, while safety constraints are enforced via Control Barrier Functions (CBFs). The planner is based on the Incrementally-exploring Information Gathering (IIG) algorithm and allows closed-loop kinodynamic node expansion using a Model Predictive Control (MPC) formalism. Robotic exploration and information gathering problems are inherently path-dependent problems. That is, the …
We propose an informative path planning (IPP) algorithm for active classification using an unmanned aerial vehicle (UAV), focusing on weed detection in precision agriculture. We model the presence of weeds on farmland using an occupancy grid and generate plans according to information-theoretic objectives, enabling the UAV to gather data efficiently. We use a combination of global viewpoint selection and evolutionary optimization to refine the UAV's trajectory in continuous space while satisfying dynamic constraints. We validate our approach in simulation by comparing against standard "lawnmow…
Informative path planning is an important and challenging problem in robotics that remains to be solved in a manner that allows for wide-spread implementation and real-world practical adoption. Among various reasons for this, one is the lack of approaches that allow for informative path planning in high-dimensional spaces and non-trivial sensor constraints. In this work we present a sampling-based approach that allows us to tackle the challenges of large and high-dimensional search spaces. This is done by performing informed sampling in the high-dimensional continuous space and incorporating p…
Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Semantic segmentation of aerial imagery is an important tool for mapping and earth observation. However, supervised deep learning models for segmentation rely on large amounts of high-quality labelled data, which is labour-intensive and time-consuming to generate. To address this, we propose a new approach for using unmanned aerial vehicles (UAVs) to autonomously collect useful data for model training. We exploit a Bayesian approach to estimate model uncertainty in semantic segmentation. During a mission, the semantic predictions and model uncertainty are used as input for terrain mapping. A k…
ETH Zürich（苏黎世联邦理工学院）；University of Technology Sydney（悉尼科技大学）
Information gathering algorithms play a key role in unlocking the potential of robots for efficient data collection in a wide range of applications. However, most existing strategies neglect the fundamental problem of the robot pose uncertainty, which is an implicit requirement for creating robust, high-quality maps. To address this issue, we introduce an informative planning framework for active mapping that explicitly accounts for the pose uncertainty in both the mapping and planning tasks. Our strategy exploits a Gaussian Process (GP) model to capture a target environmental field given the …
Fingerprint-based indoor localization methods are promising due to the high availability of deployed access points and compatibility with commercial-off-the-shelf user devices. However, to train regression models for localization, an extensive site survey is required to collect fingerprint data from the target areas. In this paper, we consider the problem of informative path planning (IPP) to find the optimal walk for site survey subject to a budget constraint. IPP for location fingerprint collection is related to the well-known orienteering problem (OP) but is more challenging due to edge-bas…
Informative path planning (IPP) is used to design paths for robotic sensor platforms to extract the best/maximum possible information about a quantity of interest while operating under a set of constraints, such as the dynamic feasibility of vehicles. The key challenges of IPP are the strong coupling in multiple layers of decisions: the selection of locations to visit, the allocation of sensor platforms to those locations; and the processing of the gathered information along the paths. This paper presents a systematic procedure for IPP and environmental mapping using multiple UAV sensor platfo…
Velocity Field: An Informative Traveling Cost Representation for Trajectory Planning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）
针对自动驾驶轨迹规划难以穷尽设计规则的问题，提出名为 Velocity Field 的局部地图表示方法，通过学习人类驾驶示范提供朝向与速度先验，在真实数据集上比栅格化代价地图更可靠高效。
Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
In this paper, we solve a multi-robot informative path planning (MIPP) task under the influence of uncertain communication and adversarial attackers. The goal is to create a multi-robot system that can learn and unify its knowledge of an unknown environment despite the presence of corrupted robots sharing malicious information. We use a Gaussian Process (GP) to model our unknown environment and define informativeness using the metric of mutual information. The objectives of our MIPP task is to maximize the amount of information collected by the team while maximizing the probability of resilien…
Scientists interested in studying natural phenomena often take physical specimens from locations in the environment for later analysis. These analysis locations are typically specified by expert heuristics. Instead, we propose to choose locations for scientific analysis by using a robot to perform an informative path planning survey. The survey results in a list of locations that correspond to the quantile values of the phenomenon of interest. We develop a robot planner using novel objective functions to improve the estimates of the quantile values over time and an approach to find locations w…
Division of Robotics, Perception and Learning（机器人、感知与学习分部）；Division of Naval Architecture（造船分部）；Digital Futures at KTH Royal Institute of Technology（瑞典皇家理工学院数字未来中心）；KTH Royal Institute of Technology（瑞典皇家理工学院）
IA-TIGRIS: An Incremental and Adaptive Sampling-Based Planner for Online Informative Path Planning
Robotics Institute, School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院机器人研究所）；Department of Mechanical Engineering, Brigham Young University（杨百翰大学机械工程系）
University of Technology, Sydney（悉尼科技大学）；ETH Zürich（苏黎世联邦理工学院）
Unmanned Aerial Vehicles (UAVs) represent a new frontier in a wide range of monitoring and research applications. To fully leverage their potential, a key challenge is planning missions for efficient data acquisition in complex environments. To address this issue, this article introduces a general Informative Path Planning (IPP) framework for monitoring scenarios using an aerial robot, focusing on problems in which the value of sensor information is unevenly distributed in a target area and unknown a priori . The approach is capable of learning and focusing on regions of interest via adaptatio…
ETH Zürich（苏黎世联邦理工学院）；University of Technology, Sydney（悉尼科技大学）
Unmanned aerial vehicles (UAVs) can offer timely and cost-effective delivery of high-quality sensing data. How- ever, deciding when and where to take measurements in complex environments remains an open challenge. To address this issue, we introduce a new multiresolution mapping approach for informative path planning in terrain monitoring using UAVs. Our strategy exploits the spatial correlation encoded in a Gaussian Process model as a prior for Bayesian data fusion with probabilistic sensors. This allows us to incorporate altitude-dependent sensor models for aerial imaging and perform constan…
Department of Computing and Software, McMaster University（麦克马斯特大学计算机与软件工程系）；McMaster University（麦克马斯特大学）；Department of Computing and Software（计算机与软件工程系）
Large-scale spatial data such as air quality, thermal conditions and location signatures play a vital role in a variety of applications. Collecting such data manually can be tedious and labour intensive. With the advancement of robotic technologies, it is feasible to automate such tasks using mobile robots with sensing and navigation capabilities. However, due to limited battery lifetime and scarcity of charging stations, it is important to plan paths for the robots that maximize the utility of data collection, also known as the informative path planning (IPP) problem. In this paper, we propos…
Aerial robots are increasingly being utilized for environmental monitoring and exploration. However, a key challenge is efficiently planning paths to maximize the information value of acquired data as an initially unknown environment is explored. To address this, we propose a new approach for informative path planning based on deep reinforcement learning (RL). Combining recent advances in RL and robotic applications, our method combines tree search with an offline-learned neural network predicting informative sensing actions. We introduce several components making our approach applicable for r…
Stanford University（斯坦福大学）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）
Adaptive Informative Path Planning with Multimodal Sensing (AIPPMS) considers the problem of an agent equipped with multiple sensors, each with different sensing accuracy and energy costs. The agent's goal is to explore the environment and gather information subject to its resource constraints in unknown, partially observable environments. Previous work has focused on the less general Adaptive Informative Path Planning (AIPP) problem, which considers only the effect of the agent's movement on received observations. The AIPPMS problem adds additional complexity by requiring that the agent reaso…
DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；Georgia Institute of Technology（佐治亚理工学院）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
IndustryNav: Exploring Spatial Reasoning of Embodied Agents in Dynamic Industrial Navigation
Michigan State University（密歇根州立大学）；Independent Researcher（独立研究者）；University of California, Irvine（加州大学尔湾分校）；Ohio State University（俄亥俄州立大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；University of Pittsburgh（匹兹堡大学）；University of Virginia（弗吉尼亚大学）；Arizona State University（亚利桑那州立大学）；Purdue University Northwest（普渡大学西北分校）
PM-Nav: Priori-Map Guided Embodied Navigation in Functional Buildings
Faculty of Robot Science and Engineering at Northeastern University（东北大学机器人科学与工程学院）；Foshan Graduate School of Innovation at Northeastern University（东北大学佛山创新研究生学院）；School of Aeronautic Science and Engineering at Beihang University（北航航空科学与工程学院）
Indian Institute of Technology Madras（马德拉斯印度理工学院）；Carnegie Mellon University（卡内基梅隆大学）；Georgia Institute of Technology（佐治亚理工学院）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
Lincoln Institute for Agri-food Technology, University of Lincoln（林肯大学农业食品技术林肯研究所）
In our previous work, we designed a systematic policy to prioritize sampling locations to lead significant accuracy improvement in spatial interpolation by using the prediction uncertainty of Gaussian Process Regression (GPR) as "attraction force" to deployed robots in path planning. Although the integration with Traveling Salesman Problem (TSP) solvers was also shown to produce relatively short travel distance, we here hypothesise several factors that could decrease the overall prediction precision as well because sub-optimal locations may eventually be included in their paths. To address thi…
We are witnessing significant progress on perception models, specifically those trained on large-scale internet images. However, efficiently generalizing these perception models to unseen embodied tasks is insufficiently studied, which will help various relevant applications (e.g., home robots). Unlike static perception methods trained on pre-collected images, the embodied agent can move around in the environment and obtain images of objects from any viewpoints. Therefore, efficiently learning the exploration policy and collection method to gather informative training samples is the key to thi…
Luddy School of Informatics, Computing, and Engineering, Indiana University（印第安纳大学勒迪信息学、计算与工程学院）
Informative planning seeks a sequence of actions that guide the robot to collect the most informative data to build a large-scale environmental model or learn a dynamical system. Existing work in informative planning mainly focuses on proposing new planners and applying them to various robotic applications such as environmental monitoring, autonomous exploration, and system identification. The informative planners optimize an objective given by a probabilistic model, e.g., Gaussian process regression (GPR). In practice, the ubiquitous sensing outliers can easily affect the model, resulting in …
PFEA: An LLM-based High-Level Natural Language Planning and Feedback Embodied Agent for Human-Centered AI
School of Electrical and Automation Engineering, Nanjing Normal University（南京师范大学电气与自动化工程学院）；Department of Mechanical Engineering, Temple University（Temple大学机械工程系）
