Motivated by lunar exploration, we consider deploying a network of mobile robots to explore an unknown environment while acting as a cooperative positioning system. Robots measure and communicate position-related data in order to perform localization in the absence of infrastructure-based solutions (e.g. stationary beacons or GPS). We present Trilateration for Exploration and Mapping (TEAM), a novel algorithm for low-complexity localization and mapping with robotic networks. TEAM is designed to leverage the capability of commercially-available ultra-wideband (UWB) radios on board the robots to provide range estimates with centimeter accuracy and perform anchorless localization in a shared, stationary frame. It is well-suited for feature-deprived environments, where feature-based localization approaches suffer. We provide experimental results in varied Gazebo simulation environments as well as on a testbed of Turtlebot3 Burgers with Pozyx UWB radios. We compare TEAM to the popular Rao-Blackwellized Particle Filter for Simultaneous Localization and Mapping (SLAM). We demonstrate that TEAM requires an order of magnitude less computational complexity and reduces the necessary sample rate of LiDAR measurements by an order of magnitude. These advantages do not require sacrificing performance, as TEAM reduces the maximum localization error by 50% and achieves up to a 28% increase in map accuracy in feature-deprived environments and comparable map accuracy in other settings.
Polytechnique Montréal（蒙特利尔综合理工学院）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Massachusetts Institute of Technology（麻省理工学院）
To achieve collaborative tasks, robots in a team need to have a shared understanding of the environment and their location within it. Distributed Simultaneous Localization and Mapping (SLAM) offers a practical solution to localize the robots without relying on an external positioning system (e.g. GPS) and with minimal information exchange. Unfortunately, current distributed SLAM systems are vulnerable to perception outliers and therefore tend to use very conservative parameters for inter-robot place recognition. However, being too conservative comes at the cost of rejecting many valid loop clo…
CSAR: Containerized System Architecture for Robotics
Machine Perception and Intelligent Robotics group (MAPIR), Dept. of System Engineering and Automation, Málaga Institute for Mechatronics Engineering and Cyber-Physical Systems (IMECH.UMA), University of Málaga（机器感知与智能机器人组（MAPIR），系统工程与自动化系，马拉加机电一体化工程与信息物理系统研究所（IMECH.UMA），马拉加大学）
University of West Attica（西阿提卡大学）；University of Manchester（曼彻斯特大学）；University of Birmingham（伯明翰大学）
This paper presents a Mixed-Initiative (MI) framework for addressing the problem of control authority transfer between a remote human operator and an AI agent when cooperatively controlling a mobile robot. Our Hierarchical Expert-guided Mixed-Initiative Control Switcher (HierEMICS) leverages information on the human operator's state and intent. The control switching policies are based on a criticality hierarchy. An experimental evaluation was conducted in a high-fidelity simulated disaster response and remote inspection scenario, comparing HierEMICS with a state-of-the-art Expert-guided Mixed-…
Due to the highly complex environment present during the DARPA Subterranean Challenge, all six funded teams relied on legged robots as part of their robotic team. Their unique locomotion skills of being able to step over obstacles require special considerations for navigation planning. In this work, we present and examine ArtPlanner, the navigation planner used by team CERBERUS during the Finals. It is based on a sampling-based method that determines valid poses with a reachability abstraction and uses learned foothold scores to restrict areas considered safe for stepping. The resulting planni…
ETH Zurich（苏黎世联邦理工学院）；Norwegian University of Science and Technology（挪威科技大学）；University of Nevada, Reno（内华达大学雷诺分校）；University of Oxford（牛津大学）
This article presents the CERBERUS robotic system-of-systems, which won the DARPA Subterranean Challenge Final Event in 2021. The Subterranean Challenge was organized by DARPA with the vision to facilitate the novel technologies necessary to reliably explore diverse underground environments despite the grueling challenges they present for robotic autonomy. Due to their geometric complexity, degraded perceptual conditions combined with lack of GPS support, austere navigation conditions, and denied communications, subterranean settings render autonomous operations particularly demanding. In resp…
Team Mountaineers launched efforts on the NASA Space Robotics Challenge Phase-2 (SRC2). The challenge will be held on the lunar terrain with virtual robotic platforms to establish an in-situ resource utilization process. In this report, we provide an overview of a simulation environment, a virtual mobile robot, and a software architecture that was created by Team Mountaineers in order to prepare for the competition's qualification round before the competition environment was released.
This paper describes the vision based robotic picking system that was developed by our team, Team Applied Robotics, for the Amazon Picking Challenge 2016. This competition challenged teams to develop a robotic system that is able to pick a large variety of products from a shelve or a tote. We discuss the design considerations and our strategy, the high resolution 3D vision system, the use of a combination of texture and shape-based object detection algorithms, the robot path planning and object manipulators that were developed.
This paper describes an approach to the design of a population of cooperative robots based on concepts borrowed from Systems Theory and Artificial Intelligence. The research has been developed under the SocRob project, carried out by the Intelligent Systems Laboratory at the Institute for Systems and Robotics - Instituto Superior Tecnico (ISR/IST) in Lisbon. The acronym of the project stands both for "Society of Robots" and "Soccer Robots", the case study where we are testing our population of robots. Designing soccer robots is a very challenging problem, where the robots must act not only to …
Autonomous Systems Lab (ASL)（自主系统实验室）；ETH Zurich - Swiss Federal Institute of Technology（苏黎世联邦理工学院）
This article describes the hardware and software systems of the Micro Aerial Vehicle (MAV) platforms used by the ETH Zurich team in the 2017 Mohamed Bin Zayed International Robotics Challenge (MBZIRC). The aim was to develop robust outdoor platforms with the autonomous capabilities required for the competition, by applying and integrating knowledge from various fields, including computer vision, sensor fusion, optimal control, and probabilistic robotics. This paper presents the major components and structures of the system architectures, and reports on experimental findings for the MAV-based c…
Faculty of Electrical Engineering, Czech Technical University（捷克技术大学电气工程学院）
We present a field report of CTU-CRAS-NORLAB team from the Subterranean Challenge (SubT) organised by the Defense Advanced Research Projects Agency (DARPA). The contest seeks to advance technologies that would improve the safety and efficiency of search-and-rescue operations in GPS-denied environments. During the contest rounds, teams of mobile robots have to find specific objects while operating in environments with limited radio communication, e.g. mining tunnels, underground stations or natural caverns. We present a heterogeneous exploration robotic system of the CTU-CRAS-NORLAB team, which…
School of Earth and Space Exploration（地球与空间探索学院）；Vanderbilt University（范德堡大学）；National Institute of Technology Karnataka（卡纳塔克邦国家理工学院）
Remote sample recovery is a rapidly evolving application of Small Unmanned Aircraft Systems (sUAS) for planetary sciences and space exploration. Development of cyber-physical systems (CPS) for autonomous deployment and recovery of sensor probes for sample caching is already in progress with NASA's MARS 2020 mission. To challenge student teams to develop autonomy for sample recovery settings, the 2020 NSF CPS Challenge was positioned around the launch of the MARS 2020 rover and sUAS duo. This paper discusses perception and trajectory planning for sample recovery by sUAS in a simulation environm…
University of Technology of Belfort-Montbéliard (UTBM)（贝尔福-蒙贝利亚尔技术大学（UTBM））；University of Sheffield（谢菲尔德大学）
This paper presents the research focus and ideas incorporated in the EPANer robotics team, entering the World Robot Challenge 2020 - Partner Robot Challenge (Real Space).
This technical report addresses the problem of optimal surveillance of the route followed by a dynamic parade using a team of aerial robots. The dynamic parade is considered to take place within an urban environment, it is discretized and at every iteration, the algorithm computes the best possible placing of the aerial robotic team members, subject to their camera model and the occlusions arising from the environment. As the parade route is only as well covered as its least-covered point, the optimization objective is to place the aerial robots such that they maximize the minimum coverage ove…
Plans for establishing a long-term human presence on the Moon will require substantial increases in robot autonomy and multi-robot coordination to support establishing a lunar outpost. To achieve these objectives, algorithm design choices for the software developments need to be tested and validated for expected scenarios such as autonomous in-situ resource utilization (ISRU), localization in challenging environments, and multi-robot coordination. However, real-world experiments are extremely challenging and limited for extraterrestrial environment. Also, realistic simulation demonstrations in…
HARMONIC: Cognitive and Control Collaboration in Human-Robotic Teams
Cognitive Science Department（认知科学系）；Department of Computer Science（计算机科学系）；Rensselaer Polytechnic Institute（拉特格斯理工学院）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
MOSAIC: Modular Scalable Autonomy for Intelligent Coordination of Heterogeneous Robotic Teams
FZI Forschungszentrum für Informatik, Karlsruhe, Germany（弗赖堡信息研究中心，卡尔斯鲁厄，德国）；Robotic Systems Lab (RSL), ETH Zürich, Zürich, Switzerland（机器人系统实验室（RSL），苏黎世联邦理工学院，苏黎世，瑞士）；Machine Intelligence and Robotics Lab (MaiRo), Karlsruhe Institute for Technology (KIT), Karlsruhe, Germany（机器智能与机器人实验室（MaiRo），卡尔斯鲁厄技术学院（KIT），卡尔斯鲁厄，德国）；European Space Agency (ESA), Noordwijk, Netherlands（欧洲航天局（ESA），诺德威克，荷兰）
University of Oklahoma（俄克拉荷马大学）；University of Nevada, Reno（内华达大学雷诺分校）
There is a lot of waste in an industrial environment that could cause harmful effects to both the products and the workers resulting in product defects, itchy eyes or chronic obstructive pulmonary disease, etc. While automative cleaning robots could be used, the environment is often too big for one robot to clean alone in addition to the fact that it does not have adequate stored dirt capacity. We present a multi-robotic dirt cleaning system algorithm for multiple automatic iRobot Creates teaming to efficiently clean an environment. Moreover, since some spaces in the environment are clean whil…
This paper address the challenges encountered by developers when deploying a distributed decision-making behavior on heterogeneous robotic systems. Many applications benefit from the use of multiple robots, but their scalability and applicability are fundamentally limited if relying on a central control station. Getting beyond the centralized approach can increase the complexity of the embedded intelligence, the sensitivity to the network topology, and render the deployment on physical robots tedious and error-prone. By integrating the swarm-oriented programming language Buzz with the standard…
Australian National University (ANU)（澳大利亚国立大学）；Monash University（莫纳什大学）；Queensland University of Technology (QUT)（昆士兰科技大学）
Open Cloud Robot Table Organization Challenge (OCRTOC) is one of the most comprehensive cloud-based robotic manipulation competitions. It focuses on rearranging tabletop objects using vision as its primary sensing modality. In this extended abstract, we present our entry to the OCRTOC2020 and the key challenges the team has experienced.
Autonomous robotic systems for various applications including transport, mobile manipulation, and disaster response are becoming more and more complex. Evaluating and analyzing such systems is challenging. Robotic competitions are designed to benchmark complete robotic systems on complex state-of-the-art tasks. Participants compete in defined scenarios under equal conditions. We present our UGV solution developed for the Mohamed Bin Zayed International Robotics Challenge 2020. Our hard- and software components to address the challenge tasks of wall building and fire fighting are integrated int…
IR2: Implicit Rendezvous for Robotic Exploration Teams under Sparse Intermittent Connectivity
Department of Mechanical Engineering, College of Design and Engineering, National University of Singapore（机械工程系，设计与工程学院，新加坡国立大学）；Singapore Technologies Engineering Ltd（新加坡科技工程有限公司）
OMRON SINIC X Corp.（欧姆龙SINIC X公司）；OMRON Corp.（欧姆龙公司）；Osaka University（大阪大学）；SenseTime Japan Ltd.（商汤日本有限公司）；National Inst. of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））；Laboro.AI Inc.（Laboro.AI公司）
We propose a versatile robotic system for kitting and assembly tasks which uses no jigs or commercial tool changers. Instead of specialized end effectors, it uses its two-finger grippers to grasp and hold tools to perform subtasks such as screwing and suctioning. A third gripper is used as a precision picking and centering tool, and uses in-built passive compliance to compensate for small position errors and uncertainty. A novel grasp point detection for bin picking is described for the kitting task, using a single depth map. Using the proposed system we competed in the Assembly Challenge of t…
We present Scratchy---a modular, lightweight robot built for low budget competition attendances. Its base is mainly built with standard 4040 aluminium profiles and the robot is driven by four mecanum wheels on brushless DC motors. In combination with a laser range finder we use estimated odometry -- which is calculated by encoders -- for creating maps using a particle filter. A RGB-D camera is utilized for object detection and pose estimation. Additionally, there is the option to use a 6-DOF arm to grip objects from an estimated pose or generally for manipulation tasks. The robot can be assemb…
Every day, burning buildings threaten the lives of occupants and first responders trying to save them. Quick action is of essence, but some areas might not be accessible or too dangerous to enter. Robotic systems have become a promising addition to firefighting, but at this stage, they are mostly manually controlled, which is error-prone and requires specially trained personal. We present two systems for autonomous firefighting from air and ground we developed for the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2020. The systems use LiDAR for reliable localization within narrow…
For robotic systems to succeed in high risk, real-world situations, they have to be quickly deployable and robust to environmental changes, under-performing hardware, and mission subtask failures. These robots are often designed to consider a single sequence of mission events, with complex algorithms lowering individual subtask failure rates under some critical constraints. Our approach utilizes common techniques in vision and control, and encodes robustness into mission structure through outcome monitoring and recovery strategies. In addition, our system infrastructure enables rapid deploymen…
Heidelberg University Hospital（海德堡大学医院）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Cognitive cooperative assistance in robot-assisted surgery holds the potential to increase quality of care in minimally invasive interventions. Automation of surgical tasks promises to reduce the mental exertion and fatigue of surgeons. In this work, multi-agent reinforcement learning is demonstrated to be robust to the distribution shift introduced by pairing a learned policy with a human team member. Multi-agent policies are trained directly from images in simulation to control multiple instruments in a sub task of the minimally invasive removal of the gallbladder. These agents are evaluated…
University of Pennsylvania（宾夕法尼亚大学）；University of New Mexico（新墨西哥大学）
In this paper we consider the problem of coordinating robotic systems with different kinematics, sensing and vision capabilities to achieve certain mission goals. An approach that makes use of a heterogeneous team of agents has several advantages when cost, integration of capabilities, or large search areas need to be considered. A heterogeneous team allows for the robots to become "specialized", accomplish sub-goals more effectively, and thus increase the overall mission efficiency. Two main scenarios are considered in this work. In the first case study we exploit mobility to implement a powe…
Scientific competitions are important in robotics because they foster knowledge exchange and allow teams to test their research in unstandardized scenarios and compare result. In the field of service robotics its role becomes crucial. Competitions like RoboCup@Home bring robots to people, a fundamental step to integrate them into society. In this paper we summarize and discuss the differences between the achievements claimed by teams in their team description papers, and the results observed during the competition^1 from a qualitative perspective. We conclude with a set of important challenges…
At present, the research on robot team cooperation is still in qualitative analysis phase and lacks the description model that can quantitatively describe the dynamical evolution of team cooperative relationships with constantly changeable task demand in Multi-robot field. First this paper whole and static describes organization model HWROM of robot team, then uses Markov course and Bayesian theorem for reference, dynamical describes the team cooperative relationships building. Finally from cooperative entity layer, ability layer and relative layer we research team formation and cooperative me…
Vision Foundation Models for Domain Generalisable Cross-View Localisation in Planetary Ground-Aerial Robotic Teams
AI for Space Group and 3 Andy Thomas Centre for Space Resources, The University of Adelaide（AI空间组和安迪·托马斯太空资源中心，阿德莱德大学）；Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA（喷气推进实验室，加州理工学院，帕萨迪纳，CA 91109，美国）；California Institute of Technology（加州理工学院）
This paper presents the WiFi-Sensor-for-Robotics (WSR) toolbox, an open source C++ framework. It enables robots in a team to obtain relative bearing to each other, even in non-line-of-sight (NLOS) settings which is a very challenging problem in robotics. It does so by analyzing the phase of their communicated WiFi signals as the robots traverse the environment. This capability, based on the theory developed in our prior works, is made available for the first time as an opensource tool. It is motivated by the lack of easily deployable solutions that use robots' local resources (e.g WiFi) for se…
While a number of excellent review articles on military robots have appeared in existing literature, this paper focuses on a distinct sub-space of related problems: small military robots organized into moderately sized squads, operating in a ground combat environment. Specifically, we consider the following: - Command of practical small robots, comparable to current generation, small unmanned ground vehicles (e.g., PackBots) with limited computing and sensor payload, as opposed to larger vehicle-sized robots or micro-scale robots; - Utilization of moderately sized practical forces of 3-10 robo…
Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arm…
Using a deformable sheet to handle objects is convenient and found in many practical applications. For object manipulation through a deformable sheet that is held by multiple mobile robots, it is a challenging task to model the object-sheet interactions. We present a computational model and algorithm to capture the object position on the deformable sheet with changing robotic team formations. A virtual variable cables model (VVCM) is proposed to simplify the modeling of the robot-sheet-object system. With the VVCM, we further present a motion planner for the robotic team to transport the objec…
Academy for Engineering and Technology, Fudan University（复旦大学工程与技术研究院）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；Zhejiang University（浙江大学）
Designing autonomous aerial robot team systems remains a grand challenge in robotics. Existing works in this field can be categorized as centralized and decentralized. Centralized methods suffer from scale dilemmas, while decentralized ones often lead to poor planning quality. In this paper, we propose an enhanced decentralized autonomous aerial robot team system with group planning. According to the spatial distribution of agents, the system dynamically divides the team into several groups and isolated agents. For conflicts within each group, we propose a novel coordination mechanism named gr…
The recent availability of commercial-off-the-shelf (COTS) legged robot platforms have opened up new opportunities in deploying legged systems into different scenarios. While the main advantage of legged robots is their ability to traverse unstructured terrain, there are still large gaps between what robot platforms can achieve and their animal counterparts. Therefore, when deploying as part of a heterogeneous robot team of different platforms, it is beneficial to understand the different scenarios where a legged platform would perform better than a wheeled, tracked or aerial platform. Two COT…
The University of New Mexico（新墨西哥大学）；Florida Institute of Technology（佛罗里达理工学院）；NASA Kennedy Space Center（美国国家航空航天局肯尼迪航天中心）；Santa Fe Institute（圣达菲研究所）
The Swarmathon is a swarm robotics programming challenge that engages college students from minority-serving institutions in NASA's Journey to Mars. Teams compete by programming a group of robots to search for, pick up, and drop off resources in a collection zone. The Swarmathon produces prototypes for robot swarms that would collect resources on the surface of Mars. Robots operate completely autonomously with no global map, and each team's algorithm must be sufficiently flexible to effectively find resources from a variety of unknown distributions. The Swarmathon includes Physical and Virtual…
Autonomous Intelligent Systems Group（自主智能系统小组）；University of Bonn（波恩大学）
The Mohamed Bin Zayed International Robotics Challenge (MBZIRC) 2017 has defined ambitious new benchmarks to advance the state-of-the-art in autonomous operation of ground-based and flying robots. This article covers our approaches to solve the two challenges that involved micro aerial vehicles (MAV). Challenge 1 required reliable target perception, fast trajectory planning, and stable control of an MAV in order to land on a moving vehicle. Challenge 3 demanded a team of MAVs to perform a search and transportation task, coined "Treasure Hunt", which required mission planning and multi-robot co…
The First WARA Robotics Mobile Manipulation Challenge -- Lessons Learned
Center for Applied Autonomous Sensor Systems (AASS)（应用自主传感系统中心）；Politecnico di Milano（米兰理工学院）；Division of Robotics, Perception and Learning, Royal Institute of Technology (KTH)（机器人、感知与学习 division，皇家理工学院）；Lund Technical University (LTH)（卢恩技术大学）；University of Basilicata（巴塞拉塔大学）；Pharmaceutical Technology & Development, R&D, AstraZeneca（制药技术与开发，研发，阿斯利康）；ABB Corporate Research（ABB企业研究）
本文总结首届 WARA Robotics 移动操作挑战赛的设置、产业动机、四类参赛方案与经验教训，并提出 2025 年第二届赛事的改进方向，以促进真实实验室中的自主移动操作机器人研发和产学协作。
In the context of collaborative robotics, distributed situation awareness is essential for supporting collective intelligence in teams of robots and human agents where it can be used for both individual and collective decision support. This is particularly important in applications pertaining to emergency rescue and crisis management. During operational missions, data and knowledge is gathered incrementally and in different ways by heterogeneous robots and humans. We describe this as the creation of \emph{Hastily Formed Knowledge Networks} (HFKNs). The focus of this paper is the specification …
University of California San Diego（加州大学圣地亚哥分校）；University of Minnesota（明尼苏达大学）；University of Michigan（密歇根大学）
Robotics has the potential to magnify the skilled workforce of the nation by complementing our workforce with automation: teams of people and robots will be able to do more than either could alone. The economic engine of the U.S. runs on the productivity of our people. The rise of automation offers new opportunities to enhance the work of our citizens and drive the innovation and prosperity of our industries. Most critically, we need research to understand how future robot technologies can best complement our workforce to get the best of both human and automated labor in a collaborative team. …
Alibaba AI Labs（阿里巴巴人工智能实验室）；UC San Diego（加州大学圣迭戈分校）；University of Edinburgh（爱丁堡大学）；German Aerospace Center（德国航空航天中心）；University of South Florida（南佛罗里达大学）；Shanghai Jiaotong University（上海交通大学）；Simon Fraser University（西蒙弗雷泽大学）
In this paper, we propose a cloud-based benchmark for robotic grasping and manipulation, called the OCRTOC benchmark. The benchmark focuses on the object rearrangement problem, specifically table organization tasks. We provide a set of identical real robot setups and facilitate remote experiments of standardized table organization scenarios in varying difficulties. In this workflow, users upload their solutions to our remote server and their code is executed on the real robot setups and scored automatically. After each execution, the OCRTOC team resets the experimental setup manually. We also …
