We present the Light Augmented Reality System LARS as an open-source and cost-effective tool. LARS leverages light-projected visual scenes for indirect robot-robot and human-robot interaction through the real environment. It operates in real-time and is compatible with a range of robotic platforms, from miniature to middle-sized robots. LARS can support researchers in conducting experiments with increased freedom, reliability, and reproducibility. This XR tool makes it possible to enrich the environment with full control by adding complex and dynamic objects while keeping the properties of robots as realistic as they are.
We present a method to register individual members of a robotic swarm in an augmented reality display while showing relevant information about swarm dynamics to the user that would be otherwise hidden. Individual swarm members and clusters of the same group are identified by their color, and by blinking at a specific time interval that is distinct from the time interval at which their neighbors blink. We show that this problem is an instance of the graph coloring problem, which can be solved in a distributed manner in O(log(n)) time. We demonstrate our approach using a swarm chemistry simulati…
Human-guided Swarms: Impedance Control-inspired Influence in Virtual Reality Environments
School of Mechanical and Materials Engineering（机械与材料工程学院）；Washington State University（华盛顿州立大学）；Department of Mechanical and Industrial Engineering（机械与工业工程系）；University of Massachusetts（马萨诸塞大学）
Mixed Reality Environment and High-Dimensional Continuification Control for Swarm Robotics
Scuola Superiore Meridionale（南方高等学院）；University of Rome La Sapienza（罗马第一大学）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；University of Naples Federico II（那不勒斯费德里科二世大学）
This paper proposes an intuitive human-swarm interaction framework inspired by our childhood memory in which we interacted with living ants by changing their positions and environments as if we were omnipotent relative to the ants. In virtual reality, analogously, we can be a super-powered virtual giant who can supervise a swarm of mobile robots in a vast and remote environment by flying over or resizing the world and coordinate them by picking and placing a robot or creating virtual walls. This work implements this idea by using Virtual Reality along with Leap Motion, which is then validated …
Skolkovo Institute of Science and Technology（斯科尔科沃科技学院）
We propose a novel concept of augmented reality (AR) human-drone interaction driven by RL-based swarm behavior to achieve intuitive and immersive control of a swarm formation of unmanned aerial vehicles. The DroneARchery system developed by us allows the user to quickly deploy a swarm of drones, generating flight paths simulating archery. The haptic interface LinkGlide delivers a tactile stimulus of the bowstring tension to the forearm to increase the precision of aiming. The swarm of released drones dynamically avoids collisions between each other, the drone following the user, and external o…
RoomShift is a room-scale dynamic haptic environment for virtual reality, using a small swarm of robots that can move furniture. RoomShift consists of nine shape-changing robots: Roombas with mechanical scissor lifts. These robots drive beneath a piece of furniture to lift, move and place it. By augmenting virtual scenes with physical objects, users can sit on, lean against, place and otherwise interact with furniture with their whole body; just as in the real world. When the virtual scene changes or users navigate within it, the swarm of robots dynamically reconfigures the physical environmen…
We present an augmented reality human-swarm interface that combines two modalities of interaction: environment-oriented and robot-oriented. The environment-oriented modality allows the user to modify the environment (either virtual or physical) to indicate a goal to attain for the robot swarm. The robot-oriented modality makes it possible to select individual robots to reassign them to other tasks to increase performance or remedy failures. Previous research has concluded that environment-oriented interaction might prove more difficult to grasp for untrained users. In this paper, we report a u…
Although fiducial markers give an accurate pose estimation in laboratory conditions, where the noisy factors are controlled, using them in field robotic applications remains a challenge. This is constrained to the fiducial maker systems, since they only work within the RGB image space. As a result, noises in the image produce large pose estimation errors. In robotic applications, fiducial markers have been mainly used in its original and simple form, as a plane in a printed paper sheet. This setup is sufficient for basic visual servoing and augmented reality applications, but not for complex s…
This paper proposes a novel methodology for addressing the simulation-reality gap for multi-robot swarm systems. Rather than immediately try to shrink or `bridge the gap' anytime a real-world experiment failed that worked in simulation, we characterize conditions under which this is actually necessary. When these conditions are not satisfied, we show how very simple simulators can still be used to both (i) design new multi-robot systems, and (ii) guide real-world swarming experiments towards certain emergent behaviors when the gap is very large. The key ideas are an iterative simulator-in-the-…
BotDirector: Robot Storytelling Across the Symmetrical Reality with Multi-modal Interactions
State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China（国家一般人工智能重点实验室，BIGAI，北京，中国）；Peking University, Beijing, China（北京大学，北京，中国）
