In this work, we present preliminary work on a novel method for Human-Swarm Interaction (HSI) that can be used to change the macroscopic behavior of a swarm of robots with decentralized sensing and control. By integrating a small yet capable hand gesture recognition convolutional neural network (CNN) with the next-generation Robot Operating System \emph{ros2}, which enables decentralized implementation of robot software for multi-robot applications, we demonstrate the feasibility of programming a swarm of robots to recognize and respond to a sequence of hand gestures that capable of correspond to different types of swarm behaviors. We test our approach using a sequence of gestures that modifies the target inter-robot distance in a group of three Turtlebot3 Burger robots in order to prevent robot collisions with obstacles. The approach is validated in three different Gazebo simulation environments and in a physical testbed that reproduces one of the simulated environments.
A Gesture-Based Visual Learning Model for Acoustophoretic Interactions using a Swarm of AcoustoBots
Department of Computer Science University College London London United Kingdom（计算机科学系伦敦大学学院伦敦英国）；Department of Artificial Intelligence University of Malaya Kuala Lumpur Malaysia（人工智能系马来大学吉隆坡马来西亚）；University College London（伦敦大学学院）；University of Malaya（马来大学）
Swarm robotic search is concerned with searching targets in unknown environments (e.g., for search and rescue or hazard localization), using a large number of collaborating simple mobile robots. In such applications, decentralized swarm systems are touted for their task/coverage scalability, time efficiency, and fault tolerance. To guide the behavior of such swarm systems, two broad classes of approaches are available, namely nature-inspired swarm heuristics and multi-robotic search methods. However, simultaneously offering computationally-efficient scalability and fundamental insights into th…
In densely-packed robot swarms operating in confined regions, spatial interference -- which manifests itself as a competition for physical space -- forces robots to spend more time navigating around each other rather than performing the primary task. This paper develops a decentralized algorithm that enables individual robots to decide whether to stay in the region and contribute to the overall mission, or vacate the region so as to reduce the negative effects that interference has on the overall efficiency of the swarm. We develop this algorithm in the context of a distributed collection task…
We present a decentralized and scalable approach for deployment of a robot swarm. Our approach tackles scenarios in which the swarm must reach multiple spatially distributed targets, and enforce the constraint that the robot network cannot be split. The basic idea behind our work is to construct a logical tree topology over the physical network formed by the robots. The logical tree acts as a backbone used by robots to enforce connectivity constraints. We study and compare two algorithms to form the logical tree: outwards and inwards. These algorithms differ in the order in which the robots jo…
Decentralized swarm robotic solutions to searching for targets that emit a spatially varying signal promise task parallelism, time efficiency, and fault tolerance. It is, however, challenging for swarm algorithms to offer scalability and efficiency, while preserving mathematical insights into the exhibited behavior. A new decentralized search method (called Bayes-Swarm), founded on batch Bayesian Optimization (BO) principles, is presented here to address these challenges. Unlike swarm heuristics approaches, Bayes-Swarm decouples the knowledge generation and task planning process, thus preservi…
Singapore University of Technology and Design（新加坡科技设计大学）
In this paper, we consider the problem of collectively exploring unknown and dynamic environments with a decentralized heterogeneous multi-robot system consisting of multiple units of two variants of a miniature robot. The first variant-a wheeled ground unit-is at the core of a swarm of floor-mapping robots exhibiting scalability, robustness and flexibility. These properties are systematically tested and quantitatively evaluated in unstructured and dynamic environments, in the absence of any supporting infrastructure. The results of repeated sets of experiments show a consistent performance fo…
Multiple robotic systems, working together, can provide important solutions to different real-world applications (e.g., disaster response), among which task allocation problems feature prominently. Very few existing decentralized multi-robotic task allocation (MRTA) methods simultaneously offer the following capabilities: consideration of task deadlines, consideration of robot range and task completion capacity limitations, and allowing asynchronous decision-making under dynamic task spaces. To provision these capabilities, this paper presents a computationally efficient algorithm that involve…
Boston University（波士顿大学）；Northeastern University（东北大学）
The Weighted-Mean Subsequence Reduced (W-MSR) algorithm, the state-of-the-art method for Byzantine-resilient design of decentralized multi-robot systems, is based on discarding outliers received over Linear Consensus Protocol (LCP). Although W-MSR provides well-understood theoretical guarantees relating robust network connectivity to the convergence of the underlying consensus, the method comes with several limitations preventing its use at scale: (1) the number of Byzantine robots, F, to tolerate should be known a priori, (2) the requirement that each robot maintains 2F+1 neighbors is impract…
Auto-HSI: Personalized human control of a robot swarm on demand by using LLMs for online automatic code generation
IRIDIA, Université Libre de Bruxelles（布鲁塞尔自由大学IRIDIA研究所）；Department of Aerospace Science and Technology, Politecnico di Milano（米兰理工大学航空航天科学与技术系）；Télécom Physique Strasbourg, Université de Strasbourg（斯特拉斯堡大学斯特拉斯堡电信物理学院）；Polytech Lyon, Lyon 1 Université Claude Bernard（里昂第一大学里昂理工学院）；University of Pisa（比萨大学）；Scuola Superiore Sant’Anna（圣安娜高等研究学院）
We consider the problem of finding distributed controllers for large networks of mobile robots with interacting dynamics and sparsely available communications. Our approach is to learn local controllers that require only local information and communications at test time by imitating the policy of centralized controllers using global information at training time. By extending aggregation graph neural networks to time varying signals and time varying network support, we learn a single common local controller which exploits information from distant teammates using only local communication interch…
Singapore University of Technology and Design（新加坡科技设计大学）；Thales Solutions Asia（泰雷兹亚洲解决方案公司）；University of Ottawa（渥太华大学）
The task of searching for and tracking of multiple targets is a challenging one. However, most works in this area do not consider evasive targets that move faster than the agents comprising the multi-robot system. This is due to the assumption that the movement patterns of such targets, combined with their excessive speed, would make the task nearly impossible to accomplish. In this work, we show that this is not the case and we propose a decentralized search and tracking strategy in which the level of exploration and exploitation carried out by the swarm is adjustable. By tuning a swarm's exp…
Singapore University of Technology and Design（新加坡科技设计大学）
Collective animal behaviors are paradigmatic examples of fully decentralized operations involving complex collective computations such as collective turns in flocks of birds or collective harvesting by ants. These systems offer a unique source of inspiration for the development of fault-tolerant and self-healing multi-robot systems capable of operating in dynamic environments. Specifically, swarm robotics emerged and is significantly growing on these premises. However, to date, most swarm robotics systems reported in the literature involve basic computational tasks---averages and other algebra…
Efficient exploration strategies are vital in tasks such as search-and-rescue missions and disaster surveying. Unmanned Aerial Vehicles (UAVs) have become particularly popular in such applications, promising to cover large areas at high speeds. Moreover, with the increasing maturity of onboard UAV perception, research focus has been shifting toward higher-level reasoning for single- and multi-robot missions. However, autonomous navigation and exploration of previously unknown large spaces still constitutes an open challenge, especially when the environment is cluttered and exhibits large and f…
A Unified Stochastic Mechanism Underlying Collective Behavior in Ants, Physical Systems, and Robotic Swarms
Computer Science and Artificial Intelligence Laboratory, MIT, MA, US（麻省理工学院计算机科学与人工智能实验室）；The International Institute for Industrial Environmental Economics, Lund University, Sweden（隆德大学国际工业环境经济研究所）
In this paper, we propose an approach to the distributed storage and fusion of data for collective perception in resource-limited robot swarms. We demonstrate our approach in a distributed semantic classification scenario. We consider a team of mobile robots, in which each robot runs a pre-trained classifier of known accuracy to annotate objects in the environment. We provide two main contributions: (i) a decentralized, shared data structure for efficient storage and retrieval of the semantic annotations, specifically designed for low-resource mobile robots; and (ii) a voting-based, decentrali…
Swarming Without an Anchor (SWA): Robot Swarms Adapt Better to Localization Dropouts Then a Single Robot
Czech Technical University in Prague（捷克技术大学）；University of Klagenfurt（克雷格福特大学）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；New York University Abu Dhabi（纽约大学阿布扎克分校）；Dubai Future Labs（迪拜未来实验室）
AVOID-JACK: Avoidance of Jackknifing for Swarms of Long Heavy Articulated Vehicles
Chair for Computational Intelligence, Otto-von-Guericke-University（计算智能系，奥托·冯·格里克大学）；Fraunhofer Institute for Transportation and Infrastructure Systems IVI（交通与基础设施系统研究所）
École Polytechnique de Montréal（蒙特利尔综合理工学院）；Royal Melbourne Institute of Technology（皇家墨尔本理工大学）
We present Buzz, a novel programming language for heterogeneous robot swarms. Buzz advocates a compositional approach, offering primitives to define swarm behaviors both from the perspective of the single robot and of the overall swarm. Single-robot primitives include robot-specific instructions and manipulation of neighborhood data. Swarm-based primitives allow for the dynamic management of robot teams, and for sharing information globally across the swarm. Self-organization stems from the completely decentralized mechanisms upon which the Buzz run-time platform is based. The language can be …
Science of Intelligence Excellence Cluster（科学智能卓越集群）；Technical University Berlin（柏林工业大学）；Institute for Theoretical Biology, Humboldt University Berlin（柏林洪堡大学理论生物学研究所）；Centre de Recherches sur la Cognition Animale, Centre de Biologie Intégrative, Université de Toulouse（法国图卢兹大学动物认知研究中心、整合生物学中心；法国国家科学研究中心、图卢兹第三大学）；CNRS, UPS, France（德国康斯坦茨大学计算机与信息科学系）；Department of Computer and Information Science, University of Konstanz, Konstanz, Germany（柏林计算神经科学伯恩斯坦中心）；Bernstein Center for Computational Neuroscience Berlin, Berlin, Germany
TCS Research（塔塔咨询服务公司研究院）；Indian Institute of Science（印度科学学院）；Jadavpur University（贾达普大学）
In this paper, we introduce a semi-decentralized control technique for a swarm of robots transporting a fragile object to a destination in an uncertain occluded environment.The proposed approach has been split into two parts. The initial part (Phase 1) includes a centralized control strategy for creating a specific formation among the agents so that the object to be transported, can be positioned properly on the top of the system. We present a novel triangle packing scheme fused with a circular region-based shape control method for creating a rigid configuration among the robots. In the later …
Swarms of robots will revolutionize many industrial applications, from targeted material delivery to precision farming. However, several of the heterogeneous characteristics that make them ideal for certain future applications --- robot autonomy, decentralized control, collective emergent behavior, etc. --- hinder the evolution of the technology from academic institutions to real-world problems. Blockchain, an emerging technology originated in the Bitcoin field, demonstrates that by combining peer-to-peer networks with cryptographic algorithms a group of agents can reach an agreement on a part…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
We propose a novel human-swarm interaction system, allowing the user to directly control a swarm of drones in a complex environment through trajectory drawing with a hand gesture interface based on the DNN-based gesture recognition. The developed CV-based system allows the user to control the swarm behavior without additional devices through human gestures and motions in real-time, providing convenient tools to change the swarm's shape and formation. The two types of interaction were proposed and implemented to adjust the swarm hierarchy: trajectory drawing and free-form trajectory generation …
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Teleoperation tasks with multi-agent systems have a high potential in supporting human-swarm collaborative teams in exploration and rescue operations. However, it requires an intuitive and adaptive control approach to ensure swarm stability in a cluttered and dynamically shifting environment. We propose a novel human-swarm interaction system, allowing the user to control swarm position and formation by either direct hand motion or by trajectory drawing with a hand gesture interface based on the DNN gesture recognition. The key technology of the SwarmPaint is the user's ability to perform vario…
Gesture based Human-Swarm Interactions for Formation Control using interpreters
Department of Mechanical and Aerospace Engineering, University of California at San Diego, La Jolla, CA 92093, USA（机械与航空航天工程系，加州大学圣地亚哥分校，拉古拉，CA 92093，美国）
Anthropomorphic robot avatars present a conceptually novel approach to remote affective communication, allowing people across the world a wider specter of emotional and social exchanges over traditional 2D and 3D image data. However, there are several limitations of current telepresence robots, such as the high weight, complexity of the system that prevents its fast deployment, and the limited workspace of the avatars mounted on either static or wheeled mobile platforms. In this paper, we present a novel concept of telecommunication through a robot avatar based on an anthropomorphic swarm of d…
We present a decentralized algorithm to achieve segregation into an arbitrary number of groups with swarms of autonomous robots. The distinguishing feature of our approach is in the minimalistic assumptions on which it is based. Specifically, we assume that (i) Each robot is equipped with a ternary sensor capable of detecting the presence of a single nearby robot, and, if that robot is present, whether or not it belongs to the same group as the sensing robot; (ii) The robots move according to a differential drive model; and (iii) The structure of the control system is purely reactive, and it m…
Collective Behavior Clone with Visual Attention via Neural Interaction Graph Prediction
College of Computer Science and Technology at Zhejiang University（浙江大学计算机科学与技术学院）；Westlake University（西湖大学）；Max Planck Institute of Animal Behavior（马克斯·普朗克动物行为研究所）；University of Konstanz（康斯坦茨大学）
University of Namur（那慕尔大学）；Université Libre de Bruxelles（布鲁塞尔自由大学）
Self-organized aggregation is a well studied behavior in swarm robotics as it is the pre-condition for the development of more advanced group-level responses. In this paper, we investigate the design of decentralized algorithms for a swarm of heterogeneous robots that self-aggregate over distinct target sites. A previous study has shown that including as part of the swarm a number of informed robots can steer the dynamic of the aggregation process to a desirable distribution of the swarm between the available aggregation sites. We have replicated the results of the previous study using a simpl…
Connectivity maintenance plays a key role in achieving a desired global behavior among a swarm of robots. However, connectivity maintenance in realistic environments is hampered by lack of computation resources, low communication bandwidth, robot failures, and unstable links. In this paper, we propose a novel decentralized connectivity-preserving algorithm that can be deployed on top of other behaviors to enforce connectivity constraints. The algorithm takes a set of targets to be reached while keeping a minimum number of redundant links between robots, with the goal of guaranteeing bandwidth …
Robotics Institute, School of Computer Science, Carnegie Mellon University（卡内基梅隆大学计算机科学学院机器人研究所）
Robotic swarms are decentralized multi-robot systems whose members use local information from proximal neighbors to execute simple reactive control laws that result in emergent collective behaviors. In contrast, members of a general multi-robot system may have access to global information, all- to-all communication or sophisticated deliberative collabora- tion. Some algorithms in the literature are applicable to robotic swarms. Others require the extra complexity of general multi- robot systems. Given an application domain, a system designer or supervisory operator must choose an appropriate s…
University of Calabria（卡拉布里亚大学）；Middlesex University London（伦敦密德萨斯大学）
The mine detection in an unexplored area is an optimization problem where multiple mines, randomly distributed throughout an area, need to be discovered and disarmed in a minimum amount of time. We propose a strategy to explore an unknown area, using a stigmergy approach based on ants behavior, and a novel swarm based protocol to recruit and coordinate robots for disarming the mines cooperatively. Simulation tests are presented to show the effectiveness of our proposed Ant-based Task Robot Coordination (ATRC) with only the exploration task and with both exploration and recruiting strategies. M…
Hong Kong University of Science and Technology（香港科技大学）
Decentralized state estimation is one of the most fundamental components of autonomous aerial swarm systems in GPS-denied areas yet it still remains a highly challenging research topic. Omni-swarm, a decentralized omnidirectional visual-inertial-UWB state estimation system for aerial swarms, is proposed in this paper to address this research niche. To solve the issues of observability, complicated initialization, insufficient accuracy, and lack of global consistency, we introduce an omnidirectional perception front-end in Omni-swarm. It consists of stereo wide-FoV cameras and ultra-wideband se…
Swarms of robots will revolutionize many industrial applications, from targeted material delivery to precision farming. Controlling the motion and behavior of these swarms presents unique challenges for human operators, who cannot yet effectively convey their high-level intentions to a group of robots in application. This work proposes a new human-swarm interface based on novel wearable gesture-control and haptic-feedback devices. This work seeks to combine a wearable gesture recognition device that can detect high-level intentions, a portable device that can detect Cartesian information and f…
Gesture-Controlled Aerial Robot Formation for Human-Swarm Interaction in Safety Monitoring Applications
Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague（电子工程系控制学系，捷克技术大学）；Department of Power Generation Technologies and Materials, Ricerca sul Sistema Energetico（发电技术与材料系，能源系统研究）
Decentralized Swarm Control via SO(3) Embeddings for 3D Trajectories
Department of Electrical and Computer Engineering, and the Ingenuity Labs Research Institute（电气与计算机工程系，以及创新实验室研究机构）；School of Computing and the Ingenuity Labs Research Institute（计算学院，以及创新实验室研究机构）
We propose a decentralized control algorithm for a minimalistic robotic swarm with limited capabilities such that the desired global behavior emerges. We consider the problem of searching for and encapsulating various targets present in the environment while avoiding collisions with both static and dynamic obstacles. The novelty of this work is the guaranteed generation of desired complex swarm behavior with constrained individual robots which have no memory, no localization, and no knowledge of the exact relative locations of their neighbors. Moreover, we analyze how the emergent behavior cha…
