The optical scanning gauges mounted on the robots are commonly used in quality inspection, such as verifying the dimensional specification of sheet structures. Coverage path planning (CPP) significantly influences the accuracy and efficiency of robotic quality inspection. Traditional CPP strategies focus on minimizing the number of viewpoints or traveling distance of robots under the condition of full coverage inspection. The measurement uncertainty when collecting the scanning data is less considered in the free-form surface inspection. To address this problem, a novel CPP method with the optimal viewpoint sampling strategy is proposed to incorporate the measurement uncertainty of key measurement points (MPs) into free-form surface inspection. At first, the feasible ranges of measurement uncertainty are calculated based on the tolerance specifications of the MPs. The initial feasible viewpoint set is generated considering the measurement uncertainty and the visibility of MPs. Then, the inspection cost function is built to evaluate the number of selected viewpoints and the average measurement uncertainty in the field of views (FOVs) of all the selected viewpoints. Afterward, an enhanced rapidly-exploring random tree (RRT*) algorithm is proposed for viewpoint sampling using the inspection cost function and CPP optimization. Case studies, including simulation tests and inspection experiments, have been conducted to evaluate the effectiveness of the proposed method. Results show that the scanning precision of key MPs is significantly improved compared with the benchmark method.
This report overviews a set of recent contributions in the field of path planning that were developed to enable the realization of the autonomous aerial robotic worker for inspection operations. The specific algorithmic contributions address several fundamental challenges of robotic inspection and exploration, and specifically those of optimal coverage planning given an a priori known model of the structure to be inspected, full coverage, optimized and fast inspection path planning, as well as efficient exploration of completely unknown environments and structures. All of the developed path pl…
A*STAR Artificial Intelligence Initiative (A*AI)（新加坡科技研究局人工智能计划（A*AI））；Institute of High Performance Computing（高性能计算研究所）；Carnegie Mellon University（卡内基梅隆大学）
Planning the path to gather the surface information of the target objects is crucial to improve the efficiency of and reduce the overall cost, for visual inspection applications with Unmanned Aerial Vehicles (UAVs). Coverage Path Planning (CPP) problem is often formulated for these inspection applications because of the coverage requirement. Traditionally, researchers usually plan and optimize the viewpoints to capture the surface information first, and then optimize the path to visit the selected viewpoints. In this paper, we propose a novel planning method to directly sample and plan the ins…
This article addresses the problem of Cooperative Coverage Path Planning (C-CPP) for the inspection of complex infrastructures (offline 3D reconstruction) by utilizing multiple Unmanned Autonomous Vehicles (UAVs). The proposed scheme, based on a priori 3D model of the infrastructure under inspection, is able to generate multiple paths for UAVs in order to achieve a complete cooperative coverage in a short time. Initially the infrastructure under inspection is being sliced by horizontal planes, which has the capability of recognizing the branches of the structure and these branches will be hand…
In this paper, we consider the problem of generating inspection paths for robots. These paths should allow an attached measurement device to perform high-quality measurements. We formally show that generating robot paths, while maximizing the inspection quality, naturally corresponds to the submodular orienteering problem. Traditional methods that are able to generate solutions with mathematical guarantees do not scale to real-world problems. In this work, we propose a method that is able to generate near-optimal solutions for real-world complex problems. We experimentally test this method in …
A*STAR Institute of Information Research (IR)（新加坡科技研究局信息研究所）；A*STAR Institute of High Performance Computing (IHPC)（新加坡科技研究局高性能计算研究所）；A*STAR Initiative for Artificial Intelligence and Analytics (AI)（新加坡科技研究局人工智能与分析计划）；Department of Mechanical Engineering, Carnegie Mellon University（卡内基梅隆大学机械工程系）
We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicle Routing Problem (SC-VRP) is formulated and subsequently solved by a modified Biased Random Key Genetic Algorithm (BRKGA) with novel, efficient encoding strategies and local improvement heuristics. W…
Evaluation of a Smart Mobile Robotic System for Industrial Plant Inspection and Supervision
Fraunhofer Institute for Highspeed Dynamics, Ernst-Mach-Institute (EMI)（弗劳恩霍夫高速动力学研究所恩斯特·马赫研究所）；Endress+Hauser Digital Solutions GmbH（恩德斯豪斯数字解决方案有限公司）；University of Freiburg（弗赖堡大学）；dotscene GmbH（dotscene有限公司）；Evonik Operations GmbH（赢创运营有限公司）；Telocate GmbH（Telocate有限公司）；University of Technology Nuremberg（纽伦堡工业大学）
Brazilian Institute of Robotics（巴西机器人研究所）；SENAI CIMATEC（巴西国家工业培训服务西玛特克分校）；Department of Informatics, University of Oslo（奥斯陆大学信息学系）；SENAI Institute for Innovation in Automation（巴西国家工业培训服务自动化创新研究所）；Robotics Innovation Center, DFKI GmbH（德国人工智能研究中心机器人创新中心）
An important open problem in robotic planning is the autonomous generation of 3D inspection paths -- that is, planning the best path to move a robot along in order to inspect a target structure. We recently suggested a new method for planning paths allowing the inspection of complex 3D structures, given a triangular mesh model of the structure. The method differs from previous approaches in its emphasis on generating and considering also plans that result in imperfect coverage of the inspection target. In many practical tasks, one would accept imperfections in coverage if this results in a sub…
Coupled Routing and Configuration Optimization for Multi-Viewpoint Robotic Inspection
Center for Artificial Intelligence Research (CAIR), VinUniversity（VinUniversity人工智能研究中心（CAIR））；VinRobotics JSC（VinRobotics股份公司）；Division of Computer and Mathematical Sciences, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学计算机与数学科学系）
Preference-Driven Active 3D Scene Representation for Robotic Inspection in Nuclear Decommissioning
School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；University of Manchester（曼彻斯特大学）；James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）
Quantum-Assisted Automatic Path-Planning for Robotic Quality Inspection in Industry 4.0
TECNALIA, Basque Research and Technology Alliance (BRTA)（TECNALIA，巴斯克研究与技术联盟）；Department of Mechanical, Energy and Management Engineering, University of Calabria（机械、能源与管理工程系，卡拉布里亚大学）
PierGuard: A Planning Framework for Underwater Robotic Inspection of Coastal Piers
Shenzhen Key Laboratory of Robotics Perception and Intelligence（深圳机器人感知与智能重点实验室）；Department of Electronic and Electrical Engineering（电子与电气工程系）；Southern University of Science and Technology（南方科技大学）；Department of Electronic and Computer Engineering（电子与计算机工程系）；Hong Kong University of Science and Technology（香港科学与技术大学）；Jiaxing Research Institute（嘉兴研究院）；Department of Electronic Engineering（电子工程系）；The Chinese University of Hong Kong（香港中文大学）
This article presents a novel framework for performing visual inspection around 3D infrastructures, by establishing a team of fully autonomous Micro Aerial Vehicles (MAVs) with robust localization, planning and perception capabilities. The proposed aerial inspection system reaches high level of autonomy on a large scale, while pushing to the boundaries the real life deployment of aerial robotics. In the presented approach, the MAVs deployed for the inspection of the structure rely only on their onboard computer and sensory systems. The developed framework envisions a modular system, combining …
University of Shanghai for Science and Technology（上海理工大学）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
Autonomous multi-robot optical inspection systems are increasingly applied for obtaining inline measurements in process monitoring and quality control. Numerous methods for path planning and robotic coordination have been developed for static and dynamic environments and applied to different fields. However, these approaches may not work for the autonomous multi-robot optical inspection system due to fast computation requirements of inline optimization, unique characteristics on robotic end-effector orientations, and complex large-scale free-form product surfaces. This paper proposes a novel t…
Autonomous navigation of steel bridge inspection robots is essential for proper maintenance. The majority of existing robotic solutions for steel bridge inspection requires human intervention to assist in the control and navigation. In this paper, a control and navigation framework has been proposed for the steel bridge inspection robot developed by the Advanced Robotics and Automation (ARA)to facilitate autonomous real-time navigation and minimize human intervention. The ARA robot is designed to work in two modes: mobile and inch-worm. The robot uses mobile mode when moving on a plane surface…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Carnegie Mellon University（卡内基梅隆大学）
Inspection planning, the task of planning motions that allow a robot to inspect a set of points of interest, has applications in domains such as industrial, field, and medical robotics. Inspection planning can be computationally challenging, as the search space over motion plans that inspect the points of interest grows exponentially with the number of inspected points. We propose a novel method, Incremental Random Inspection-roadmap Search (IRIS), that computes inspection plans whose length and set of inspected points asymptotically converge to those of an optimal inspection plan. IRIS increm…
University of Rijeka（里耶卡大学）；University of Southern Denmark（南丹麦大学）
The application of autonomous UAVs to infrastructure inspection tasks provides benefits in terms of operation time reduction, safety, and cost-effectiveness. This paper presents trajectory planning for three-dimensional autonomous multi-UAV volume coverage and visual inspection of infrastructure based on the Heat Equation Driven Area Coverage (HEDAC) algorithm. The method generates trajectories using a potential field and implements distance fields to prevent collisions and to determine UAVs' camera orientation. It successfully achieves coverage during the visual inspection of complex structur…
Within this paper, the problem of 3D structural inspection path planning for distributed infrastructure using aerial robots that are subject to time constraints is addressed. The proposed algorithm handles varying spatial properties of the infrastructure facilities, accounts for their different importance and exploration function and computes an overall inspection path of high inspection reward while respecting the robot endurance or mission time constraints as well as the vehicle dynamics and sensor limitations. To achieve its goal, it employs an iterative, 3-step optimization strategy at eac…
Autonomous navigation is essential for steel bridge inspection robot to monitor and maintain the working condition of steel bridges. Majority of existing robotic solutions requires human support to navigate the robot doing the inspection. In this paper, a navigation framework is proposed for ARA robot [1], [2] to run on mobile mode. In this mode, the robot needs to cross and inspect all the available steel bars. The most significant contributions of this research are four algorithms, which can process the depth data, segment it into clusters, estimate the boundaries, construct a graph to repre…
REACT: Real-time Entanglement-Aware Coverage Path Planning for Tethered Underwater Vehicles
Artificial Intelligence in Robotics Laboratory (AiR Lab), Department of Electrical and Computer Engineering, Aarhus University（人工智能机器人实验室（AiR实验室），电气与计算机工程系，阿arhus大学）；NestAI；EIVA a/s（EIVA公司）；Deutsches Forschungszentrum für Künstliche Intelligenz GmbH (DFKI)（德国人工智能研究中心（DFKI））；Automatic Control Group, Department of Electrical Engineering and Information Technology, Paderborn University（自动控制组，电气工程与信息科技系，帕德博恩大学）
Multi-objective or multi-destination path planning is crucial for mobile robotics applications such as mobility as a service, robotics inspection, and electric vehicle charging for long trips. This work proposes an anytime iterative system to concurrently solve the multi-objective path planning problem and determine the visiting order of destinations. The system is comprised of an anytime informable multi-objective and multi-directional RRT* algorithm to form a simple connected graph, and a proposed solver that consists of an enhanced cheapest insertion algorithm and a genetic algorithm to sol…
Vietnam National University（越南国立大学）；University of Technology Sydney（悉尼科技大学）
In built infrastructure monitoring, an efficient path planning algorithm is essential for robotic inspection of large surfaces using computer vision. In this work, we first formulate the inspection path planning problem as an extended travelling salesman problem (TSP) in which both the coverage and obstacle avoidance were taken into account. An enhanced discrete particle swarm optimization (DPSO) algorithm is then proposed to solve the TSP, with performance improvement by using deterministic initialization, random mutation, and edge exchange. Finally, we take advantage of parallel computing to…
The Hong Kong University of Science and Technology（香港科技大学）；Beihang University（北京航空航天大学）
Autonomous exploration of unknown environments has been widely applied in inspection, surveillance, and search and rescue. In exploration task, the basic requirement for robots is to detect the unknown space as fast as possible. In this paper, we propose an autonomous collaborative system consists of an aerial robot and a ground vehicle to explore in unknown environments. We combine the frontier based method and the harmonic field to generate a path. Then, For the ground robot, a minimum jerk piecewise Bezier curve which can guarantee safety and dynamical feasibility is generated amid obstacle…
Developing Smart MAVs for Autonomous Inspection in GPS-denied Constructions
College of Engineering, South China Agriculture University（华南农业大学工程学院）；College of Civil Aviation, Nanjing University of Aeronautics and Astronautics（南京航空航天大学民航学院）；College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）
Ergodic Trajectory Optimization on Generalized Domains Using Maximum Mean Discrepancy
Department of Mechanical Engineering, Yale University（耶鲁大学机械工程系）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；School of Mathematics, The University of Edinburgh（爱丁堡大学数学学院）；NVIDIA
UAS-based Automated Structural Inspection Path Planning via Visual Data Analytics and Optimization
University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；Zhejiang University - University of Illinois at Urbana Champaign Institute（浙江大学伊利诺伊大学厄巴纳香槟校区联合学院）
Robotics Institute at Carnegie Mellon University（卡内基梅隆大学机器人研究所）
In search and surveillance applications in robotics, it is intuitive to spatially distribute robot trajectories with respect to the probability of locating targets in the domain. Ergodic coverage is one such approach to trajectory planning in which a robot is directed such that the percentage of time spent in a region is in proportion to the probability of locating targets in that region. In this work, we extend the ergodic coverage algorithm to robots operating in constrained environments and present a formulation that can capture sensor footprint and avoid obstacles and restricted areas in t…
PDE-Based Optimization for Stochastic Mapping and Coverage Strategies using Robotic Ensembles
School for Engineering of Matter, Transport and Energy, Arizona State University（物质、传输与能量工程学院，亚利桑那州立大学）；School of Mathematical and Statistical Sciences, Arizona State University（数学与统计科学学院，亚利桑那州立大学）
University of Nevada, Reno（内华达大学里诺分校）；Rutgers University（罗格斯大学）；Dong-A University（东亚大学）
The threat to safety of aging bridges has been recognized as a critical concern to the general public due to the poor condition of many bridges in the U.S. Currently, the bridge inspection is conducted manually, and it is not efficient to identify bridge condition deterioration in order to facilitate implementation of appropriate maintenance or rehabilitation procedures. In this paper, we report a new development of the autonomous mobile robotic system for bridge deck inspection and evaluation. The robot is integrated with several nondestructive evaluation (NDE) sensors and a navigation contro…
University of Nevada, Reno（内华达大学里诺分校）；University of Technology Sydney（悉尼科技大学）
This paper presents visual and 3D structure inspection for steel structures and bridges using a developed climbing robot. The robot can move freely on a steel surface, carry sensors, collect data and then send to the ground station in real time for monitoring as well as further processing. Steel surface image stitching and 3D map building are conducted to provide a current condition of the structure. Also, a computer vision-based method is implemented to detect surface defects on stitched images. The effectiveness of the climbing robot's inspection is tested in multiple circumstances to ensure…
Bridge condition assessment is important to maintain the quality of highway roads for public transport. Bridge deterioration with time is inevitable due to aging material, environmental wear and in some cases, inadequate maintenance. Non-destructive evaluation (NDE) methods are preferred for condition assessment for bridges, concrete buildings, and other civil structures. Some examples of NDE methods are ground penetrating radar (GPR), acoustic emission, and electrical resistivity (ER). NDE methods provide the ability to inspect a structure without causing any damage to the structure in the pr…
Solar-powered aircraft promise significantly increased flight endurance over conventional aircraft. While this makes them promising candidates for large-scale aerial inspection missions, their structural fragility necessitates that adverse weather is avoided using appropriate path planning methods. This paper therefore presents MetPASS, the Meteorology-aware Path Planning and Analysis Software for Solar-powered UAVs. MetPASS is the first path planning framework in the literature that considers all aspects that influence the safety or performance of solar-powered flight: It avoids environmental…
The Hong Kong Polytechnic University（香港理工大学）；Hong Kong Applied Science and Technology Research Institute（香港应用科技研究院）
In this paper, we present a new robotic system to perform defect inspection tasks over free-form specular surfaces. The autonomous procedure is achieved by a six-DOF manipulator, equipped with a line scan camera and a high-intensity lighting system. Our method first uses the object's CAD mesh model to implement a K-means unsupervised learning algorithm that segments the object's surface into areas with similar curvature. Then, the scanning path is computed by using an adaptive algorithm that adjusts the camera's ROI to observe regions with irregular shapes properly. A novel iterative closest p…
Explainable Deep Anomaly Detection with Sequential Hypothesis Testing for Robotic Sewer Inspection
School of Electrical and Electronic Engineering（电气与电子工程学院）；University of Sheffield（谢菲尔德大学）；School of Mechanical, Aerospace and Civil Engineering（机械、航空航天与土木工程学院）
Mapping-Guided Task Discovery and Allocation for Robotic Inspection of Underwater Structures
Department of Mechanical Engineering, University of Washington（机械工程系，华盛顿大学）；Department of Industrial & Systems Engineering and Department of Mechanical Engineering, University of Washington（工业与系统工程系和机械工程系，华盛顿大学）
