The projection mapping systems on the human face is limited by the latency and the movement of the users. The area of the projection is restricted by the position of the projectors and the cameras. We are introducing MaskBot, a real-time projection mapping system operated by a 6 Degrees of Freedom (DoF) collaborative robot. The collaborative robot locates the projector and camera in normal position to the face of the user to increase the projection area and to reduce the latency of the system. A webcam is used to detect the face and to sense the robot-user distance to modify the projection size and orientation. MaskBot projects different images on the face of the user, such as face modifications, make-up, and logos. In contrast to the existing methods, the presented system is the first that introduces a robotic projection mapping. One of the prospective applications is to acquire a dataset of adversarial images to challenge face detection DNN systems, such as Face ID.
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学）
Having a precise and robust transformation between the robot coordinate system and the AR-device coordinate system is paramount during human-robot interaction (HRI) based on augmented reality using Head mounted displays (HMD), both for intuitive information display and for the tracking of human motions. Most current solutions in this area rely either on the tracking of visual markers, e.g. QR codes, or on manual referencing, both of which provide unsatisfying results. Meanwhile a plethora of object detection and referencing methods exist in the wider robotic and machine vision communities. The…
Patients suffering from quadriplegia have limited body motion which prevents them from performing daily activities. We have developed an assistive robotic system with an intuitive free-view gaze interface. The user's point of regard is estimated in 3D space while allowing free head movement and is combined with object recognition and trajectory planning. This framework allows the user to interact with objects using fixations. Two operational modes have been implemented to cater for different eventualities. The automatic mode performs a pre-defined task associated with a gaze-selected object, w…
In this paper, we consider the detection of a decrease of engagement by users spontaneously interacting with a socially assistive robot in a public space. We first describe the UE-HRI dataset that collects spontaneous Human-Robot Interactions following the guidelines provided by the Affective Computing research community to collect data "in-the-wild". We then analyze the users' behaviors, focusing on proxemics, gaze, head motion, facial expressions and speech during interactions with the robot. Finally, we investigate the use of deep learning techniques (Recurrent and Deep Neural Networks) to …
University of Denver（丹佛大学）；Purdue University（普渡大学）；Boulder Language Technologies（博尔德语言技术公司）
This article proposes an emotive lifelike robotic face, called ExpressionBot, that is designed to support verbal and non-verbal communication between the robot and humans, with the goal of closely modeling the dynamics of natural face-to-face communication. The proposed robotic head consists of two major components: 1) a hardware component that contains a small projector, a fish-eye lens, a custom-designed mask and a neck system with 3 degrees of freedom; 2) a facial animation system, projected onto the robotic mask, that is capable of presenting facial expressions, realistic eye movement, and…
University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
We present an initial examination of a novel approach to accurately position a patient during head and neck intensity modulated radiotherapy (IMRT). Position-based visual-servoing of a radio-transparent soft robot is used to control the flexion/extension cranial motion of a manikin head. A Kinect RGB-D camera is used to measure head position and the error between the sensed and desired position is used to control a pneumatic system which regulates pressure within an inflatable air bladder (IAB). Results show that the system is capable of controlling head motion to within 2mm with respect to a …
STREAMS: An Assistive Multimodal AI Framework for Empowering Biosignal Based Robotic Controls
Dept. of Electrical Engineering University of Rhode Island Kingston, RI, USA（电气工程系 罗德岛大学 罗得岛州金斯顿）；Computer Engineering Northeastern University Boston, Massachusetts, USA（计算机工程 美国东北大学 波士顿）
Accelerating Audio Research with Robotic Dummy Heads
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Stony Brook University（纽约州立大学石溪分校）；University of Illinois Chicago（伊利诺伊大学芝加哥分校）；Discovery Partners Institute（发现伙伴研究所）；Amazon Web Services（亚马逊云服务）
Monash University（莫纳什大学）；University of British Columbia（不列颠哥伦比亚大学）
We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provided a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 en…
Advantages of Multimodal versus Verbal-Only Robot-to-Human Communication with an Anthropomorphic Robotic Mock Driver
Centre for Applied Autonomous Sensor Systems (AASS)（应用自主传感器系统中心（AASS））；Örebro University（厄勒布鲁大学）；Robert Bosch GmbH（罗伯特·博世有限公司）；University of Skövde（舍夫德大学）；TU Munich（慕尼黑工业大学）
Robotic vision introduces requirements for real-time processing of fast-varying, noisy information in a continuously changing environment. In a real-world environment, convenient assumptions, such as static camera systems and deep learning algorithms devouring high volumes of ideally slightly-varying data are hard to survive. Leveraging on recent studies on the neural connectome associated with eye movements, we designed a neuromorphic oculomotor controller and placed it at the heart of our in-house biomimetic robotic head prototype. The controller is unique in the sense that (1) all data are …
Web-based Augmented Reality with Auto-Scaling and Real-Time Head Tracking towards Markerless Neurointerventional Preoperative Planning and Training of Head-mounted Robotic Needle Insertion
The Chinese University of Hong Kong（香港中文大学）；CUHK Shenzhen Research Institute（香港中文大学（深圳）研究院）
本文提出一种无标记Web增强现实应用，利用MediaPipe和React Three Fiber实现头部自动缩放与实时跟踪，用于神经介入术前规划和头戴式机器人针插入训练，成本低且支持协作。
Chittagong University of Engineering and Technology（吉大港工程技术大学）
Meeting eye contact is the essential prerequisite skill of a human to initiate any conversation with others. However, it is not an easy task for a robot to meet eye contact with a human if they are not facing each other initially or the human is intensely engaged his or her task. If the robot would like to start communication with a particular person, it should turn its gaze to that person first. However, only such a turning action alone is not always enough to set up eye contact. Sometimes, the robot should perform some strong actions so that it can capture the human's attention toward it. In…
Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modeling
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（信息处理国家重点实验室，北京大学计算机学院）；AI 2 Robotics（AI与机器人）；Hong Kong University of Science and Technology（香港科学与技术大学）
ForzaETH Race Stack -- Scaled Autonomous Head-to-Head Racing on Fully Commercial off-the-Shelf Hardware
ETH Zürich（苏黎世联邦理工学院）；Center for Project-Based Learning (PBL), ETH Zürich（苏黎世联邦理工学院项目式学习中心）；Institute for Dynamic Systems and Control (IDSC), ETH Zürich（苏黎世联邦理工学院动态系统与控制研究所）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
This paper considers the problem of audio source separation where the goal is to isolate a target audio signal (say Alice's speech) from a mixture of multiple interfering signals (e.g., when many people are talking). This problem has gained renewed interest mainly due to the significant growth in voice controlled devices, including robots in homes, offices, and other public facilities. Although a rich body of work exists on the core topic of source separation, we find that robotic motion of the microphone -- say the robot's head -- is a complementary opportunity to past approaches. Briefly, we…
Dept. of Mechanical Engineering（机械工程系）；Dept. of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patient, attached to consoles where their heads must be fixed and their arms can only move in a limited space. This creates a barrier between physicians and patients that does not exist in normal surgery. To address this issue, we propose a mobile telesurgery solution where the surgeons are no longer mech…
Autonomous Intelligent Systems group of University of Bonn（波恩大学自主智能系统小组）
We present a method for real-time stereo scene capture and remote VR visualization that allows a human operator to freely move their head and thus intuitively control their perspective during teleoperation. The stereo camera is mounted on a 6D robotic arm, which follows the operator's head pose. Existing VR teleoperation systems either induce high latencies on head movements, leading to motion sickness, or use scene reconstruction methods to allow re-rendering of the scene from different perspectives, which cannot handle dynamic scenes effectively. Instead, we present a decoupled approach whic…
X-DiffVLA: X-Embodied Diffusion Action Heads for Vision-Language-Action Models
SKL-MAIS, Institute of Automation, Chinese Academy of Sciences（SKL-MAIS，自动化研究所，中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院，中国科学院大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；BeingBeyond；School of Computer Science, Peking University（北京大学计算机学院）
This paper presents a soft-robotic platform for exploring the ecological relevance of non-locomotory movements via animal-robot interactions. Coral snakes (genus Micrurus) and their mimics use vigorous, non-locomotory, and arrhythmic thrashing to deter predation. There is variation across snake species in the duration and curvature of anti-predator thrashes, and it is unclear how these aspects of motion interact to contribute to snake survival. In this work, soft robots composed of fiber-reinforced elastomeric enclosures (FREEs) are developed to emulate the anti-predator behaviors of three gen…
Boston University（波士顿大学）；Stanford University（斯坦福大学）
This work presents a novel marriage of Swarm Robotics and Brain Computer Interface technology to produce an interface which connects a user to a swarm of robots. The proposed interface enables the user to control the swarm's size and motion employing just thoughts and eye movements. The thoughts and eye movements are recorded as electrical signals from the scalp by an off-the-shelf Electroencephalogram (EEG) headset. Signal processing techniques are used to filter out noise and decode the user's eye movements from raw signals, while a Hidden Markov Model technique is employed to decipher the u…
The demonstrations of micro-robotic systems in minimally invasive medicine include an individual or a swarm of microswimmer of various origin, artificial or biohybrid, often with an external computer-controlled electromagnetic field. There are several in vivo and in vitro control performances with artificial microswimmers but control of a bio-hybrid microswimmer using an open kinematic chain remains untouched. In this work, non-contact maneuvering control of a single magnetotactic bacterium cell is simulated. The results show that the proposed system is capable of adjusting the heading of the …
TeleHairing: A Teleoperation Baseline for Robotic Haircutting
University of Oulu（奥卢大学）；Shaoyang University（邵阳学院）；Jishou University（吉首大学）；University of Exeter（埃克塞特大学）；University of Naples Federico II（那不勒斯腓特烈二世大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）
Coverage Planning for Robotic Tooth Preparation in Densely Constrained Environments
Tsinghua University（清华大学）；ETH Zürich（苏黎世联邦理工学院）；Tsingscribe Medical Ltd.（清scribe医疗有限公司）；Peking University School and Hospital of Stomatology（北京大学口腔医学院）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
