A robotic solution for the unmanned ground vehicles (UGVs) to execute the highly complex task of object manipulation in an autonomous mode is presented. This paper primarily focuses on developing an autonomous robotic system capable of assembling elementary blocks to build the large 3D structures in GPS-denied environments. The key contributions of this system paper are i) Designing of a deep learning-based unified multi-task visual perception system for object detection, part-detection, instance segmentation, and tracking, ii) an electromagnetic gripper design for robust grasping, and iii) system integration in which multiple system components are integrated to develop an optimized software stack. The entire mechatronic and algorithmic design of UGV for the above application is detailed in this work. The performance and efficacy of the overall system are reported through several rigorous experiments.
Virginia Tech（弗吉尼亚理工大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
Localization in a global map is critical to success in many autonomous robot missions. This is particularly challenging for multi-robot operations in unknown and adverse environments. Here, we are concerned with providing a small unmanned ground vehicle (UGV) the ability to localize itself within a 2.5D aerial map generated from imagery captured by a low-flying unmanned aerial vehicle (UAV). We consider the scenario where GPS is unavailable and appearance-based scene changes may have occurred between the UAV's flight and the start of the UGV's mission. We present a GPS-free solution to this lo…
Laboratory for Robotics and Intelligent Control systems（机器人与智能控制系统实验室）；University of Zagreb Faculty of Electrical Engineering and Computing（萨格勒布大学电气工程与计算学院）；Unska 3, Zagreb, Croatia（克罗地亚萨格勒布Unska 3号）
In this paper we present our hardware design and control approaches for a mobile manipulation platform used in Challenge 2 of the MBZIRC 2020 competition. In this challenge, a team of UAVs and a single UGV collaborate in an autonomous, wall-building scenario, motivated by construction automation and large-scale robotic 3D printing. The robots must be able, autonomously, to detect, manipulate, and transport bricks in an unstructured, outdoor environment. Our control approach is based on a state machine that dictates which controllers are active at each stage of the Challenge. In the first stage…
Institute for Systems Analysis of Russian Academy of Sciences（俄罗斯科学院系统分析研究所）；Soloviev Rybinsk State Aviation Technical University（索洛维约夫雷宾斯克国立航空技术大学）；NPP SATEK plus（SATEK plus公司）
The problem of developing distributed control and navigation system for quadrotor UAVs operating in GPS-denied environments is addressed in the paper. Cooperative navigation, marker detection and mapping task solved by a team of multiple unmanned aerial vehicles is chosen as demo example. Developed intelligent control system complies with on 4D\RCS reference model and its implementation is based on ROS framework. Custom implementation of EKF-based map building algorithm is used to solve marker detection and map building task.
Center for Non-Linear Studies, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室非线性研究中心）；Dept. of Aerospace Engg., University of Cincinnati（辛辛那提大学航空航天工程系）；Dept. of Mechanical Engg., Texas A&M University（德克萨斯A&M大学机械工程系）
Most of the routing algorithms for unmanned vehicles, that arise in data gathering and monitoring applications in the literature, rely on the Global Positioning System (GPS) information for localization. However, disruption of GPS signals either intentionally or unintentionally could potentially render these algorithms not applicable. In this article, we present a novel method to address this difficulty by combining methods from cooperative localization and routing. In particular, the article formulates a fundamental combinatorial optimization problem to plan routes for an unmanned vehicle in …
University of Pennsylvania（宾夕法尼亚大学）；University of Zurich（苏黎世大学）
One of the most challenging tasks for a flying robot is to autonomously navigate between target locations quickly and reliably while avoiding obstacles in its path, and with little to no a-priori knowledge of the operating environment. This challenge is addressed in the present paper. We describe the system design and software architecture of our proposed solution, and showcase how all the distinct components can be integrated to enable smooth robot operation. We provide critical insight on hardware and software component selection and development, and present results from extensive experiment…
UniPilot: Enabling GPS-Denied Autonomy Across Embodiments
Department of Engineering Cybernetics, O. S. Bragstads Plass 2D, Norwegian University of Science and Technology (NTNU), Trondheim, Norway（工程 cybernetics 系，挪威科学技术大学（NTNU），特隆赫姆，挪威）
Threading the Needle: Test and Evaluation of Early Stage UAS Capabilities to Autonomously Navigate GPS-Denied Environments in the DARPA Fast Lightweight Autonomy (FLA) Program
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；New England Robotics Validation and Experimentation (NERVE) Center（新英格兰机器人验证与实验中心）
This paper addresses the problem of position estimation in UAVs operating in a cluttered environment where GPS information is unavailable. A model learning-based approach is proposed that takes in the rotor RPMs and past state as input and predicts the one-step-ahead position of the UAV using a novel spectral-normalized memory neural network (SN-MNN). The spectral normalization guarantees stable and reliable prediction performance. The predicted position is transformed to global coordinate frame which is then fused along with the odometry of other peripheral sensors like IMU, barometer, compas…
This paper addresses the issues of unmanned aerial vehicle (UAV) indoor navigation, specifically in areas where GPS and magnetometer sensor measurements are unavailable or unreliable. The proposed solution is to use an error state extended Kalman filter (ES -EKF) in the context of multi-sensor fusion. Its implementation is adapted to fuse measurements from multiple sensor sources and the state model is extended to account for sensor drift and possible calibration inaccuracies. Experimental validation is performed by fusing IMU data obtained from the PixHawk 2.1 flight controller with pose meas…
High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to f…
This paper reviews vision-based localization methods in GPS-denied environments and classifies the mainstream methods into Relative Vision Localization (RVL) and Absolute Vision Localization (AVL). For RVL, we discuss the broad application of optical flow in feature extraction-based Visual Odometry (VO) solutions and introduce advanced optical flow estimation methods. For AVL, we review recent advances in Visual Simultaneous Localization and Mapping (VSLAM) techniques, from optimization-based methods to Extended Kalman Filter (EKF) based methods. We also introduce the application of offline ma…
Cross-Session 3D LiDAR and Camera Fusion for Robust Localization of Unmanned Aerial Vehicles in GPS-Denied Environments
School of Electrical and Data Engineering, University of Technology Sydney（悉尼大学电气与数据工程学院）；Institute of Aerospace Engineering and Technology, Duy Tan University（杜益大学航空航天工程与技术学院）；College of Engineering and Computer Science, VinUniversity（Vin大学工程与计算机科学学院）；Smart Green Transformation Center (GREEN-X), VinUniversity（Vin大学智能绿色转型中心 (GREEN-X)）；Department of Artificial Intelligence and Robotics, Sejong University（成均馆大学人工智能与机器人系）
Developing Smart MAVs for Autonomous Inspection in GPS-denied Constructions
College of Engineering, South China Agriculture University（华南农业大学工程学院）；College of Civil Aviation, Nanjing University of Aeronautics and Astronautics（南京航空航天大学民航学院）；College of Mechatronics and Control Engineering, Shenzhen University（深圳大学机电与控制工程学院）
University of Cincinnati（辛辛那提大学）；Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Air Force Research Laboratory（空军研究实验室）
This paper presents a novel data-driven navigation system to navigate an Unmanned Vehicle (UV) in GPS-denied, feature-deficient environments such as tunnels, or mines. The method utilizes landmarks that vehicle can deploy and measure range from to enable localization as the vehicle traverses its pre-defined path through the tunnel. A key question that arises in such scenario is to estimate and reduce the number of landmarks that needs to be deployed for localization before the start of the mission, given some information about the environment. The main focus is to keep the maximum position unc…
UAV-assisted Visual SLAM Generating Reconstructed 3D Scene Graphs in GPS-denied Environments
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability, and Trust (SnT)（跨学科安全、可靠性与信任中心（SnT））；Faculty of Science, Technology, and Medicine, University of Luxembourg（卢森堡大学科学、技术与医学学院）
In this paper, we address the problem of autonomous multi-robot mapping, exploration and navigation in unknown, GPS-denied indoor or urban environments using a swarm of robots equipped with directional sensors with limited sensing capabilities and limited computational resources. The robots have no a priori knowledge of the environment and need to rapidly explore and construct a map in a distributed manner using existing landmarks, the presence of which can be detected using onboard senors, although little to no metric information (distance or bearing to the landmarks) is available. In order t…
UAVs have been widely used in visual inspections of buildings, bridges and other structures. In either outdoor autonomous or semi-autonomous flights missions strong GPS signal is vital for UAV to locate its own positions. However, strong GPS signal is not always available, and it can degrade or fully loss underneath large structures or close to power lines, which can cause serious control issues or even UAV crashes. Such limitations highly restricted the applications of UAV as a routine inspection tool in various domains. In this paper a vision-model-based real-time self-positioning method is …
Service Robotics LaboratoryUniversidad Pablo de Olavide（塞维利亚大学服务机器人实验室）；Computer Vision and Aerial Robotics GroupCentre for Automation and Robotics (C.A.R.)Universidad Politécnica de Madrid (UPM-CSIC)（马德里理工大学（UPM-CSIC）自动化与机器人中心计算机视觉与空中机器人小组）；Universidad Pablo de Olavide（塞维利亚大学）；Universidad Politécnica de Madrid (UPM-CSIC)（马德里理工大学（UPM-CSIC））；Centre for Automation and Robotics (C.A.R.)（自动化与机器人中心（C.A.R.））；Computer Vision and Aerial Robotics Group（计算机视觉与空中机器人小组）；Service Robotics Laboratory（服务机器人实验室）
The paper presents a framework for fire extinguishing in an urban scenario by a team of aerial and ground robots. The system was developed to address Challenge 3 of the 2020Mohamed Bin Zayed International Robotics Challenge (MBZIRC). The challenge required to autonomously detect, locate and extinguish fires on different floors of a building, as well as in its surroundings. The multi-robot system developed consists of a heterogeneous robot team of up to three Unmanned Aerial Vehicles (UAV) and one Unmanned Ground Vehicle (UGV). We describe the main hardware and software components for UAV and U…
Cooperative Localisation of a GPS-Denied UAV in 3-Dimensional Space Using Direction of Arrival Measurements
Australian National University（澳大利亚国立大学）；Hangzhou Dianzi University（杭州电子科技大学）；Data61-CSIRO（Data61联邦科学与工业研究组织）；Australian Defence Science and Technology Group（澳大利亚国防科学与技术集团）
Cooperative Localisation of a GPS-Denied UAV using Direction of Arrival Measurements
Research School of Engineering, Australian National University（澳大利亚国立大学工程研究学院）；Hangzhou Dianzi University（杭州电子科技大学）；Data61-CSIRO (formerly NICTA Ltd.)（Data61-CSIRO（原NICTA有限公司））；Australian Defence Science and Technology Group (DST Group)（澳大利亚国防科学与技术集团（DST集团））
Indian Institute of Technology, Kanpur（印度理工学院坎普尔分校）
In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) …
Queensland University of Technology（昆士兰科技大学）；Data61 / CSIRO（Data61 / 联邦科学与工业研究组织）；Griffith University（格里菲斯大学）
The use of multi-rotor Unmanned Aerial Vehicles (UAVs) for search and rescue as well as remote sensing is rapidly increasing. Multi-rotor UAVs, however, have limited endurance. The range of UAV applications can be widened if teams of multiple UAVs are used. We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Examples of such environments include indoor scenarios, urban or natural canyons…
Marker-Constrained Pose-Graph Correction for Cross-Platform Georeferencing in GNSS-Denied Environments
University of Luxembourg（卢森堡大学）；Interdisciplinary Centre for Security, Reliability and Trust (SnT)（安全、可靠性与信任跨学科中心（SnT））；Faculty of Science, Technology and Medicine, University of Luxembourg（卢森堡大学科学、技术与医学院）
Autonomous missions of drones require continuous and reliable estimates for the drone's attitude, velocity, and position. Traditionally, these states are estimated by applying Extended Kalman Filter (EKF) to Accelerometer, Gyroscope, Barometer, Magnetometer, and GPS measurements. When the GPS signal is lost, position and velocity estimates deteriorate quickly, especially when using low-cost inertial sensors. This paper proposes an estimation method that uses a Recurrent Neural Network (RNN) to allow reliable estimation of a drone's position and velocity in the absence of GPS signal. The RNN is…
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）
Mobile robots rely on odometry to navigate through areas where localization fails. Visual odometry (VO) is a common solution for obtaining robust and consistent relative motion estimates of the vehicle frame. Contrarily, Global Positioning System (GPS) measurements are typically used for absolute positioning and localization. However, when the constraint on absolute accuracy is relaxed, time-differenced carrier phase (TDCP) measurements can be used to find accurate relative position estimates with one single-frequency GPS receiver. This suggests practitioners may want to consider GPS odometry …
A Distributed Multi-UGV Exploration Framework With Loop-Aware Planning and Descriptor-Aided Localization in Resource-Limited Environments
School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械与车辆学院）；China North Artificial Intelligence & Innovation Research Institute, Collective Intelligence & Collaboration Laboratory (CIC)（中国北方人工智能与创新研究院集体智能与协作实验室）；Zhengzhou Intelligent Technology Research Institute, Beijing Institute of Technology（北京理工大学郑州智能科技研究院）
Hybrid Aerial-Ground Vehicle Autonomy in GPS-denied Environments
School of Aerospace, Mechanical and Mechatronic Engineering（航空航天、机械与机电工程学院）；The University of Sydney（悉尼大学）；Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
Control and State Estimation of Vehicle-Mounted Aerial Systems in GPS-Denied, Non-Inertial Environments
Robotics, Intelligent Systems, and Control Lab, King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia（机器人、智能系统与控制实验室，国王阿卜杜勒-阿齐兹大学科学与技术学院（KAUST），沙特阿拉伯图瓦尔）
UAV See, UGV Do: Aerial Imagery and Virtual Teach Enabling Zero-Shot Ground Vehicle Repeat
University of Toronto Robotics Institute（多伦多大学机器人研究所）；University of Toronto Institute of Aerospace Studies（多伦多大学航空航天研究学院）；Ingenuity Labs（Ingenuity实验室）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Multi-robot Systems Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克技术大学电气工程学院控制学系多机器人系统组）；Turku Intelligent Embedded and Robotic Systems (TIERS) Lab, University of Turku（图尔库大学智能嵌入式与机器人系统实验室）
Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping in building inspection. Yet, the number of studies is limited considering the post processing of the data and its integration with autonomous UAVs. These will enable huge steps onward into full automation of building inspection. In this regard, this work presents a decision making tool for revisiting tasks in visual building inspection by autonomous UAVs. The tool is an implementation of fine-tuning a pretrained Convolutional Neural Network (CNN) for surface c…
