In the modern era of automation and robotics, autonomous vehicles are currently the focus of academic and industrial research. With the ever increasing number of unmanned aerial vehicles getting involved in activities in the civilian and commercial domain, there is an increased need for autonomy in these systems too. Due to guidelines set by the governments regarding the operation ceiling of civil drones, road-tracking based navigation is garnering interest . In an attempt to achieve the above mentioned tasks, we propose an imitation learning based, data-driven solution to UAV autonomy for navigating through city streets by learning to fly by imitating an expert pilot. Derived from the classic image classification algorithms, our classifier has been constructed in the form of a fast 39-layered Inception model, that evaluates the presence of roads using the tomographic reconstructions of the input frames. Based on the Inception-v3 architecture, our system performs better in terms of processing complexity and accuracy than many existing models for imitation learning. The data used for training the system has been captured from the drone, by flying it in and around urban and semi-urban streets, by experts having at least 6-8 years of flying experience. Permissions were taken from required authorities who made sure that minimal risk (to pedestrians) is involved in the data collection process. With the extensive amount of drone data that we collected, we have been able to navigate successfully through roads without crashing or overshooting, with an accuracy of 98.44%. The computational efficiency of MAVNet enables the drone to fly at high speeds of upto 6m/sec. We present the same results in this research and compare them with other state-of-the-art methods of vision and learning based navigation.
King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）；University of Maryland（马里兰大学）；Polytechnique Montreal（蒙特利尔理工学院）；Australian College of Kuwait（科威特澳大利亚学院）
We present the Robotics Intelligent Systems & Control (RISC) Lab multiagent testbed for reliable search and rescue and aerial transport in outdoor environments. The system consists of a team of three multirotor unmanned aerial vehicles (UAVs), which are capable of autonomously searching, picking up, and transporting randomly distributed objects in an outdoor field. The method involves vision based object detection and localization, passive aerial grasping with our novel design, GPS based UAV navigation, and safe release of the objects at the drop zone. Our cooperative strategy ensures safe spa…
We present a system to capture video footage of human subjects in the real world. Our system leverages a quadrotor camera to automatically capture well-composed video of two subjects. Subjects are tracked in a large-scale outdoor environment using RTK GPS and IMU sensors. Then, given the tracked state of our subjects, our system automatically computes static shots based on well-established visual composition principles and canonical shots from cinematography literature. To transition between these static shots, we calculate feasible, safe, and visually pleasing transitions using a novel real-t…
Zhejiang University（浙江大学）；University of Pennsylvania（宾夕法尼亚大学）
Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficia…
Berkeley AI Research, UC Berkeley（加州大学伯克利分校伯克利人工智能研究院）；Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）
Legged robots are physically capable of traversing a wide range of challenging environments, but designing controllers that are sufficiently robust to handle this diversity has been a long-standing challenge in robotics. Reinforcement learning presents an appealing approach for automating the controller design process and has been able to produce remarkably robust controllers when trained in a suitable range of environments. However, it is difficult to predict all likely conditions the robot will encounter during deployment and enumerate them at training-time. What if instead of training contr…
Threading the Needle: Test and Evaluation of Early Stage UAS Capabilities to Autonomously Navigate GPS-Denied Environments in the DARPA Fast Lightweight Autonomy (FLA) Program
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；New England Robotics Validation and Experimentation (NERVE) Center（新英格兰机器人验证与实验中心）
Decentralized Heterogeneous Multi-Robot Collaborative Exploration for Indoor and Outdoor 3D Environments
College of Artificial Intelligence, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）人工智能学院）；Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong（香港中文大学机械与自动化工程系）
High-Speed Interception Multicopter Control by Image-based Visual Servoing
School of Automation Science and Electrical Engineering, Beihang University（北京航空航天大学自动化科学与电气工程学院）；School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）
An Aerial Transport System in Marine GNSS-Denied Environment
School of Aerospace Engineering（航空航天学院）；Beijing Institute of Technology（北京理工大学）；Centre for Autonomous Robotic Systems（自主机器人系统中心）；Khalifa University（哈利法大学）
Amazon（亚马逊公司）；West Virginia University（西弗吉尼亚大学）；Microsoft（微软公司）；Gather AI（Gather AI公司）；Mowito（Mowito公司）；Carnegie Mellon University（卡内基梅隆大学）
Many aerial robotic applications require the ability to land on moving platforms, such as delivery trucks and marine research boats. We present a method to autonomously land an Unmanned Aerial Vehicle on a moving vehicle. A visual servoing controller approaches the ground vehicle using velocity commands calculated directly in image space. The control laws generate velocity commands in all three dimensions, eliminating the need for a separate height controller. The method has shown the ability to approach and land on the moving deck in simulation, indoor and outdoor environments, and compared t…
Indian Institute of Technology, Kanpur（印度理工学院坎普尔分校）
In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) …
This paper proposes RUR53: an Unmanned Ground Vehicle able to autonomously navigate through, identify, and reach areas of interest; and there recognize, localize, and manipulate work tools to perform complex manipulation tasks. The proposed contribution includes a modular software architecture where each module solves specific sub-tasks and that can be easily enlarged to satisfy new requirements. Included indoor and outdoor tests demonstrate the capability of the proposed system to autonomously detect a target object (a panel) and precisely dock in front of it while avoiding obstacles. They sh…
This paper presents our method for enabling a UAV quadrotor, equipped with a monocular camera, to autonomously avoid collisions with obstacles in unstructured and unknown indoor environments. When compared to obstacle avoidance in ground vehicular robots, UAV navigation brings in additional challenges because the UAV motion is no more constrained to a well-defined indoor ground or street environment. Horizontal structures in indoor and outdoor environments like decorative items, furnishings, ceiling fans, sign-boards, tree branches etc., also become relevant obstacles unlike those for ground v…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；EarthSense Inc.（EarthSense公司）；DEVCOM Army Research Lab（DEVCOM陆军研究实验室）
We present a self-supervised approach for learning to predict traversable paths for wheeled mobile robots that require good traction to navigate. Our algorithm, termed WayFAST (Waypoint Free Autonomous Systems for Traversability), uses RGB and depth data, along with navigation experience, to autonomously generate traversable paths in outdoor unstructured environments. Our key inspiration is that traction can be estimated for rolling robots using kinodynamic models. Using traction estimates provided by an online receding horizon estimator, we are able to train a traversability prediction neural…
The paper presents a receding horizon planning and control strategy for quadrotor-type \ac{mav}s to navigate reactively and intercept a moving target in a cluttered unknown and dynamic environment. Leveraging a lightweight short-range sensor that generates a point-cloud within a relatively narrow and short \ac{fov}, and an \acs{ssd}-MobileNet based Deep neural network running on board the \ac{mav}, the proposed motion planning and control strategy produces safe and dynamically feasible \ac{mav} trajectories within the sensor \acs{fov}, which the vehicle uses to autonomously navigate, pursue, a…
Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（布拉格捷克理工大学信息学、机器人学与控制论研究所）
Exploration of an unknown environment by a mobile robot is a complex task involving solution of many fundamental problems from data processing, localization to high-level planning and decision making. The exploration framework we developed is based on processing of RGBD data provided by a MS Kinect2 sensor, which allows to take advantage of state-of-the-art SLAM (Simultaneous Localization and Mapping) algorithms and to autonomously build a realistic 3D map of the environment with projected visual information about the scene. In this paper, we describe practical issues that appeared during depl…
Huazhong University of Science and Technology（华中科技大学）
The advances in compact and agile micro aerial vehicles (MAVs) have shown great potential in replacing human for labor-intensive or dangerous indoor investigation, such as warehouse management and fire rescue. However, the design of a state estimation system that enables autonomous flight in such dim or smoky environments presents a conundrum: conventional GPS or computer vision based solutions only work in outdoors or well-lighted texture-rich environments. This paper takes the first step to overcome this hurdle by proposing Marvel, a lightweight RF backscatter-based state estimation system f…
Long-term autonomy in service robotics is a current research topic, especially for dynamic, large-scale environments that change over time. We present Sobi, a mobile service robot developed as an interactive guide for open environments, such as public places with indoor and outdoor areas. The robot will serve as a platform for environmental modeling and human-robot interaction. Its main hardware and software components, which we freely license as a documented open source project, are presented. Another key focus is Sobi's monitoring system for long-term autonomy, which restores system componen…
Topological mapping for traversability-aware long-range navigation in off-road terrain
Center for Intelligent Machines, McGill University（麦吉尔大学智能机器中心）；Gault Nature Reserve（戈尔特自然保护区）；CMLabs（CMLabs公司）；National Canadian Robotics Network（加拿大国家机器人网络）
Precision agriculture is rapidly attracting research to efficiently introduce automation and robotics solutions to support agricultural activities. Robotic navigation in vineyards and orchards offers competitive advantages in autonomously monitoring and easily accessing crops for harvesting, spraying and performing time-consuming necessary tasks. Nowadays, autonomous navigation algorithms exploit expensive sensors which also require heavy computational cost for data processing. Nonetheless, vineyard rows represent a challenging outdoor scenario where GPS and Visual Odometry techniques often st…
Advances in planetary robotics have led to wheeled robots that have beamed back invaluable science data from the surface of the Moon and Mars. However, these large wheeled robots are unable to access rugged environments such as cliffs, canyons and crater walls that contain exposed rock-faces and are geological time-capsules into the early Moon and Mars. We have proposed the SphereX robot with a mass of 3 kg, 30 cm diameter that can hop, roll and fly short distances. A single robot may slip and fall, however, a multirobot system can work cooperatively by being interlinked using spring-tethers a…
Contextual information in human environments, such as signs, symbols, and objects provide important information for robots to use for exploration and navigation. To identify and segment contextual information from complex images obtained in these environments, data-driven methods such as Convolutional Neural Networks (CNNs) are used. However, these methods require large amounts of human labeled data which are slow and time-consuming to obtain. Weakly supervised methods address this limitation by generating pseudo segmentation labels (PSLs). In this paper, we present the novel Weakly Supervised…
AutoFly: Vision-Language-Action Model for UAV Autonomous Navigation in the Wild
Southeast University（东南大学）；Purple Mountain Labs（紫金山实验室）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Fudan University（复旦大学）；Shanghai Jiao Tong University（上海交通大学）
