This paper presents the ShanghaiTech Mapping Robot, a state-of-the-art unmanned ground vehicle (UGV) designed for collecting comprehensive multi-sensor datasets to support research in robotics, Simultaneous Localization and Mapping (SLAM), computer vision, and autonomous driving. The robot is equipped with a wide array of sensors including RGB cameras, RGB-D cameras, event-based cameras, IR cameras, LiDARs, mmWave radars, IMUs, ultrasonic range finders, and a GNSS RTK receiver. The sensor suite is integrated onto a specially designed mechanical structure with a centralized power system and a synchronization mechanism to ensure spatial and temporal alignment of the sensor data. A 16-node on-board computing cluster handles sensor control, data collection, and storage. We describe the hardware and software architecture of the robot in detail and discuss the calibration procedures for the various sensors and investigate the interference for LiDAR and RGB-D sensors. The capabilities of the platform are demonstrated through an extensive outdoor dataset collected in a diverse campus environment. Experiments with two LiDAR-based and two RGB-based SLAM approaches showcase the potential of the dataset to support development and benchmarking for robotics. To facilitate research, we make the dataset publicly available along with the associated robot sensor calibration data: https://slam-hive.net/wiki/ShanghaiTech_Datasets
University of Technology Sydney（悉尼科技大学）；Australian Centre for Field Robotics, University of Sydney（悉尼大学澳大利亚现场机器人中心）
Marine vehicles have been used for various scientific missions where information over features of interest is collected. In order to maximise efficiency in collecting information over a large search space, we should be able to deploy a large number of autonomous vehicles that make a decision based on the latest understanding of the target feature in the environment. In our previous work, we have presented a hierarchical framework for the multi-vessel multi-float (MVMF) problem where surface vessels drop and pick up underactuated floats in a time-minimal way. In this paper, we present the field…
McGill University（麦吉尔大学）；University of Pennsylvania（宾夕法尼亚大学）
This paper presents the portable autonomous probing system (APS), a low-cost robotic design for collecting water quality measurements at targeted depths from an autonomous surface vehicle (ASV). This system fills an important but often overlooked niche in marine sampling by enabling mobile sensor observations throughout the near-surface water column without the need for advanced underwater equipment. We present a probe delivery mechanism built with commercially available components and describe the corresponding open-source simulator and winch controller. Finally, we demonstrate the system in …
Norwegian University of Life Sciences（挪威生命科学大学）；Pontifical Catholic University of Rio de Janeiro（里约热内卢天主教大学）
In recent years, the use of robots in agriculture has been increasing mainly due to the high demand of productivity, precision and efficiency, which follow the climate change effects and world population growth. Unlike conventional agriculture, sugarcane farms are usually regions with dense vegetation, gigantic areas, and subjected to extreme weather conditions, such as intense heat, moisture and rain. TIBA - Tankette for Intelligent BioEnergy Agriculture - is the first result of an R&D project which strives to develop an autonomous mobile robotic system for carrying out a number of agricultur…
Accurate extrinsic sensor calibration is essential for both autonomous vehicles and robots. Traditionally this is an involved process requiring calibration targets, known fiducial markers and is generally performed in a lab. Moreover, even a small change in the sensor layout requires recalibration. With the anticipated arrival of consumer autonomous vehicles, there is demand for a system which can do this automatically, after deployment and without specialist human expertise. To solve these limitations, we propose a flexible framework which can estimate extrinsic parameters without an explicit…
NATO STO Centre for Maritime Research and Experimentation (CMRE)（北约科技组织海事研究与实验中心）
In this paper we address the mine countermeasures (MCM) search problem for an autonomous underwater vehicle (AUV) surveying the seabed using a side-looking sonar. We propose a coverage path planning method that adapts the AUV track spacing with the objective of collecting better data. We achieve this by shifting the coverage overlap at the tail of the sensor range where the lowest data quality is expected. To assess the algorithm, we collected data from three at-sea experiments. The adaptive survey allowed the AUV to recover from a situation where the sensor range was overestimated and resulte…
In this paper, we present a system to train driving policies from experiences collected not just from the ego-vehicle, but all vehicles that it observes. This system uses the behaviors of other agents to create more diverse driving scenarios without collecting additional data. The main difficulty in learning from other vehicles is that there is no sensor information. We use a set of supervisory tasks to learn an intermediate representation that is invariant to the viewpoint of the controlling vehicle. This not only provides a richer signal at training time but also allows more complex reasonin…
Michigan State University（密歇根州立大学）；National Technical University of Athens（雅典国立技术大学）；Massachusetts Institute of Technology（麻省理工学院）
We identify the need for a gamified self-driving simulator where game mechanics encourage high-quality data capture, and design and apply such a simulator to collecting lane-following training data. The resulting synthetic data enables a Convolutional Neural Network (CNN) to drive an in-game vehicle. We simultaneously develop a physical test platform based on a radio-controlled vehicle and the Robotic Operating System (ROS) and successfully transfer the simulation-trained model to the physical domain without modification. The cross-platform simulator facilitates unsupervised crowdsourcing, hel…
University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）
We introduce the Precise Synthetic Image and LiDAR (PreSIL) dataset for autonomous vehicle perception. Grand Theft Auto V (GTA V), a commercial video game, has a large detailed world with realistic graphics, which provides a diverse data collection environment. Existing works creating synthetic LiDAR data for autonomous driving with GTA V have not released their datasets, rely on an in-game raycasting function which represents people as cylinders, and can fail to capture vehicles past 30 metres. Our work creates a precise LiDAR simulator within GTA V which collides with detailed models for all…
Shenzhen Institute of Artificial Intelligence and Robotics for Society (AIRS)（深圳人工智能与机器人研究院（AIRS））；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）理工学院）
Implementing fully automatic unmanned surface vehicles (USVs) monitoring water quality is challenging since effectively collecting environmental data while keeping the platform stable and environmental-friendly is hard to approach. To address this problem, we construct a USV that can automatically navigate an efficient path to sample water quality parameters in order to monitor the aquatic environment. The detection device needs to be stable enough to resist a hostile environment or climates while enormous volumes will disturb the aquaculture environment. Meanwhile, planning an efficient path …
Pontificial Catholic University of Rio de Janeiro (PUC-Rio)（里约热内卢天主教大学（PUC-Rio））
In many kinds of research, collecting data is tailored to individual research. It is usual to use dedicated and not reusable software to collect data. GrADyS Ground Station framework (GrADyS-GS) aims to collect data in a reusable manner with dynamic background tools. This technical report describes GrADyS-GS, a ground station software designed to connect with various technologies to control, monitor, and store results of Mobile Internet of Things field experiments with Autonomous Vehicles (UAV) and Sensor Networks (WSN). In the GrADyS project GrADyS-GS is used with ESP32-based IoT devices on t…
An Open-source Hardware/Software Architecture and Supporting Simulation Environment to Perform Human FPV Flight Demonstrations for Unmanned Aerial Vehicle Autonomy
Autonomous underwater vehicles (AUV) perform various applications such as seafloor mapping and underwater structure health monitoring. Commonly, an inertial navigation system aided by a Doppler velocity log (DVL) is used to provide the vehicle's navigation solution. In such fusion, the DVL provides the velocity vector of the AUV, which determines the navigation solution's accuracy and helps estimate the navigation states. This paper proposes BeamsNet, an end-to-end deep learning framework to regress the estimated DVL velocity vector that improves the accuracy of the velocity vector estimate, a…
Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping in building inspection. Yet, the number of studies is limited considering the post processing of the data and its integration with autonomous UAVs. These will enable huge steps onward into full automation of building inspection. In this regard, this work presents a decision making tool for revisiting tasks in visual building inspection by autonomous UAVs. The tool is an implementation of fine-tuning a pretrained Convolutional Neural Network (CNN) for surface c…
SSIL: Self-Supervised Imitation Learning for End-to-End Driving
R & D Center, SL Corporation（SL公司研发中心）；Department of Electrical and Computer Engineering, Sungkyunkwan University（成均馆大学电子与计算机工程系）；Department of Information and Communication Engineering, Handong Global University（翰林全球大学信息与通信工程系）；College of Engineering, Texas A & M University-Corpus Christi（德克萨斯A&M大学科珀斯克里斯蒂分校工程学院）；School of Computer Science and Electrical Engineering, Handong Global University（翰林全球大学计算机科学与电子工程学院）
Australian Centre for Field Robotics（澳大利亚现场机器人中心）；University of Sydney（悉尼大学）
Autonomous Underwater Vehicles (AUVs) are increasingly being used to support scientific research and monitoring studies. One such application is in benthic habitat mapping where these vehicles collect seafloor imagery that complements broadscale bathymetric data collected using sonar. Using these two data sources, the relationship between remotely-sensed acoustic data and the sampled imagery can be learned, creating a habitat model. As the areas to be mapped are often very large and AUV systems collecting seafloor imagery can only sample from a small portion of the survey area, the information…
University of Toronto（多伦多大学）；McGill University（麦吉尔大学）
We present Nav2Goal, a data-efficient and end-to-end learning method for goal-conditioned visual navigation. Our technique is used to train a navigation policy that enables a robot to navigate close to sparse geographic waypoints provided by a user without any prior map, all while avoiding obstacles and choosing paths that cover user-informed regions of interest. Our approach is based on recent advances in conditional imitation learning. General-purpose, safe and informative actions are demonstrated by a human expert. The learned policy is subsequently extended to be goal-conditioned by traini…
BMW of North America（北美宝马公司）；Graz University of Technology（格拉茨工业大学）
Collecting realistic driving trajectories is crucial for training machine learning models that imitate human driving behavior. Most of today's autonomous driving datasets contain only a few trajectories per location and are recorded with test vehicles that are cautiously driven by trained drivers. In particular in interactive scenarios such as highway merges, the test driver's behavior significantly influences other vehicles. This influence prevents recording the whole traffic space of human driving behavior. In this work, we present a novel methodology to extract trajectories of traffic objec…
Autonomous Systems Lab, ETH Zürich（苏黎世联邦理工学院自主系统实验室）
Model-based controllers on real robots require accurate knowledge of the system dynamics to perform optimally. For complex dynamics, first-principles modeling is not sufficiently precise, and data-driven approaches can be leveraged to learn a statistical model from real experiments. However, the efficient and effective data collection for such a data-driven system on real robots is still an open challenge. This paper introduces an optimization problem formulation to find an informative trajectory that allows for efficient data collection and model learning. We present a sampling-based method t…
Woven Planet United Kingdom Limited（Woven Planet英国有限公司）
With the Autonomous Vehicle (AV) industry shifting towards machine-learned approaches for motion planning, the performance of self-driving systems is starting to rely heavily on large quantities of expert driving demonstrations. However, collecting this demonstration data typically involves expensive HD sensor suites (LiDAR + RADAR + cameras), which quickly becomes financially infeasible at the scales required. This motivates the use of commodity sensors like cameras for data collection, which are an order of magnitude cheaper than HD sensor suites, but offer lower fidelity. Leveraging these s…
University of Science and Technology of China（中国科学技术大学）；University of Michigan（密歇根大学）；University of Michigan Transportation Research Institute（密歇根大学交通研究所）
Traffic conflicts have been studied by the transportation research community as a surrogate safety measure for decades. However, due to the rarity of traffic conflicts, collecting large-scale real-world traffic conflict data becomes extremely challenging. In this paper, we introduce and analyze ROCO - a real-world roundabout traffic conflict dataset. The data is collected at a two-lane roundabout at the intersection of State St. and W. Ellsworth Rd. in Ann Arbor, Michigan. We use raw video dataflow captured from four fisheye cameras installed at the roundabout as our input data source. We adop…
