Shared control in teleoperation leverages both human and robot's strengths and has demonstrated great advantages of reducing the difficulties in teleoperating a robot and increasing the task performance. One fundamental question in shared control is how to effectively allocate the control power to the human and robot. Researchers have been subjectively defining the arbitrate policies following conflicting principles, which resulted in great inconsistency in the policies. We attribute this inconsistency to the inconsiderateness of the multi-resource uncertainty in the human-robot system. To fill the gap, we developed a multi-source uncertainty model that was applicable to various types of uncertainty in real world, and then a general arbitration model was developed to comprehensively fuse the uncertainty and regulate the arbitration weight assigned to the robotic agent. Beside traditional macro performance metrics, we introduced objective and quantitative metrics of robotic helpfulness and friendliness that evaluated the assistive robot's cooperation at micro and macro levels. Results from simulations and experiments showed the new arbitration model was more effective and friendly over the existing policies and was robust to coping with multi-source uncertainty. With this new arbitration model, we expect the increased adoption of human-robot shared control in practical and complex teleoperation tasks.
In the field of Human-Robot Interaction (HRI), many researchers study shared control systems. Shared control is when a person and agent both contribute to the performance of a task in a collaborative way, often by providing control inputs for a robot. One of the most important things in shared control is the nature of the communication between the person and robot, which could help the human-robot team handle a variety of challenging situations. In this paper, we identify key challenges in shared control, in which better communication design could be useful, including when encountering novel s…
This paper develops a novel framework called Providers-Clients-Robots (PCR), applicable to socially assistive robots that support research on shared understanding in human-robot interactions. Providers, Clients, and Robots share an actionable and intuitive representation of the environment to create plans that best satisfy the combined needs of all parties. The plans are formed via interaction between the Client and the Robot based on a previously built multi-modal navigation graph. The explainable environmental representation in the form of a navigation graph is constructed collaboratively be…
When humans control robot arms these robots often need to infer the human's desired task. Prior research on assistive teleoperation and shared autonomy explores how robots can determine the desired task based on the human's joystick inputs. In order to perform this inference the robot relies on an internal mapping between joystick inputs and discrete tasks: e.g., pressing the joystick left indicates that the human wants a plate, while pressing the joystick right indicates a cup. This approach works well after the human understands how the robot interprets their inputs -- but inexperienced user…
University of South Carolina（南卡罗来纳大学）；University of Cincinnati（辛辛那提大学）；University of Wisconsin--Madison（威斯康星大学麦迪逊分校）；University of South Florida（佛罗里达州立大学）
Social robots like Moxie are designed to form strong emotional bonds with children, but their abrupt discontinuation can cause significant struggles and distress to children. When these services end, the resulting harm raises complex questions of who bears responsibility when children's emotional bonds are broken. Using the Moxie shutdown as a case study through a qualitative survey of 72 U.S. participants, our findings show that the responsibility is viewed as a shared duty across the robot company, parents, developers, and government. However, these attributions varied by political ideology …
The growth of scale and complexity of interactions between humans and robots highlights the need for new computational methods to automatically evaluate novel algorithms and applications. Exploring diverse scenarios of humans and robots interacting in simulation can improve understanding of the robotic system and avoid potentially costly failures in real-world settings. We formulate this problem as a quality diversity (QD) problem, where the goal is to discover diverse failure scenarios by simultaneously exploring both environments and human actions. We focus on the shared autonomy domain, whe…
Trust-Preserved Human-Robot Shared Autonomy enabled by Bayesian Relational Event Modeling
School of Electrical and Computer Engineering, Georgia Institute of Technology（电子与计算机工程学院，佐治亚理工学院）；Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology（电子与计算机工程系，香港科技大学）
Monash University（莫纳什大学）；University of British Columbia（不列颠哥伦比亚大学）
We provide an experimental evaluation of a wearable augmented reality (AR) system we have developed for human-robot teams working on tasks requiring collaboration in shared physical workspace. Recent advances in AR technology have facilitated the development of more intuitive user interfaces for many human-robot interaction applications. While it has been anticipated that AR can provided a more intuitive interface to robot assistants helping human workers in various manufacturing scenarios, existing studies in robotics have been largely limited to teleoperation and programming. Industry 5.0 en…
Machine Learning and Robotics Lab, University of Stuttgart（斯图加特大学机器学习与机器人实验室）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
We propose a formalism for shared control, which is the problem of defining a policy that blends user control and autonomous control. The challenge posed by the shared autonomy system is to maintain user control authority while allowing the robot to support the user. This can be done by enforcing constraints or acting optimally when the intent is clear. Our proposed solution relies on natural gradients emerging from the divergence constraint between the robot and the shared policy. We approximate the Fisher information by sampling a learned robot policy and computing the local gradient to augm…
Shared-Control Teleoperation Paradigms on a Soft Growing Robot Manipulator
Kadir Has University（卡迪尔哈斯大学）；University of Naples Federico II（那不勒斯费德里科二世大学）；Stanford University（斯坦福大学）；Washington State University（华盛顿州立大学）；Purdue University（普渡大学）；UC Santa Barbara（加州大学圣巴巴拉分校）
Wheelchair-mounted robotic arms (and other assistive robots) should help their users perform everyday tasks. One way robots can provide this assistance is shared autonomy. Within shared autonomy, both the human and robot maintain control over the robot's motion: as the robot becomes confident it understands what the human wants, it increasingly intervenes to automate the task. But how does the robot know what tasks the human may want to perform in the first place? Today's shared autonomy approaches often rely on prior knowledge: for example, the robot must know the set of possible human goals …
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；National Institute of Standards and Technology（国家标准技术研究院）；Department of Rehabilitation Science and Technology, School of Health and Rehabilitation Sciences, University of Pittsburgh（匹兹堡大学健康与康复科学学院康复科学与技术系）
Northwestern University（西北大学）；Shirley Ryan AbilityLab（雪莉·瑞安能力实验室）
Assistive robotic devices can increase the independence of individuals with motor impairments. However, each person is unique in their level of injury, preferences, and skills, which moreover can change over time. Further, the amount of assistance required can vary throughout the day due to pain or fatigue, or over longer periods due to rehabilitation, debilitating conditions, or aging. Therefore, in order to become an effective team member, the assistive machine should be able to learn from and adapt to the human user. To do so, we need to be able to characterize the user's control commands t…
UCLA Center for Vision, Cognition, Learning, and Autonomy (VCLA)（加州大学洛杉矶分校视觉、认知、学习与自主中心（VCLA））
We design and develop a new shared Augmented Reality (AR) workspace for Human-Robot Interaction (HRI), which establishes a bi-directional communication between human agents and robots. In a prototype system, the shared AR workspace enables a shared perception, so that a physical robot not only perceives the virtual elements in its own view but also infers the utility of the human agent--the cost needed to perceive and interact in AR--by sensing the human agent's gaze and pose. Such a new HRI design also affords a shared manipulation, wherein the physical robot can control and alter virtual obj…
Istituto Italiano di Tecnologia（意大利技术研究院）；Università degli Studi di Genova（热那亚大学）
Human perception is based on unconscious inference, where sensory input integrates with prior information. This phenomenon, known as context dependency, helps in facing the uncertainty of the external world with predictions built upon previous experience. On the other hand, human perceptual processes are inherently shaped by social interactions. However, how the mechanisms of context dependency are affected is to date unknown. If using previous experience - priors - is beneficial in individual settings, it could represent a problem in social scenarios where other agents might not have the same…
Istituto Italiano di Tecnologia（意大利理工学院）；Universita di Pisa（比萨大学）
Teleoperation is a widely adopted strategy to control robotic manipulators executing complex tasks that require highly dexterous movements and critical high-level intelligence. Classical teleoperation schemes are based on either joystick control, or on more intuitive interfaces which map directly the user arm motions into one robot arm's motions. These approaches have limits when the execution of a given task requires reconfigurable multiple robotic arm systems. Indeed, the simultaneous teleoperation of two or more robot arms could extend the workspace of the manipulation cell, or increase its…
Many tasks, particularly those involving interaction with the environment, are characterized by high variability, making robotic autonomy difficult. One flexible solution is to introduce the input of a human with superior experience and cognitive abilities as part of a shared autonomy policy. However, current methods for shared autonomy are not designed to address the wide range of necessary corrections (e.g., positions, forces, execution rate, etc.) that the user may need to provide to address task variability. In this paper, we present corrective shared autonomy, where users provide correcti…
Shared autonomy provides a framework where a human and an automated system, such as a robot, jointly control the system's behavior, enabling an effective solution for various applications, including human-robot interaction. However, a challenging problem in shared autonomy is safety because the human input may be unknown and unpredictable, which affects the robot's safety constraints. If the human input is a force applied through physical contact with the robot, it also alters the robot's behavior to maintain safety. We address the safety issue of shared autonomy in real-time applications by p…
Legible Shared Autonomy: Implicit Communication of Robot Belief through Motion
Department of Automation, University of Science and Technology of China（自动化系，中国科学技术大学）；Department of Automation, Hefei University of Technology（自动化系，合肥工业大学）；National Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）
Westphalian University of Applied Sciences（威斯特伐利亚应用科学大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
Living a self-determined life independent of human caregivers or fully autonomous robots is a crucial factor for human dignity and the preservation of self-worth for people with motor impairments. Assistive robotic solutions - particularly robotic arms - are frequently deployed in domestic care, empowering people with motor impairments in performing ADLs independently. However, while assistive robotic arms can help them perform ADLs, currently available controls are highly complex and time-consuming due to the need to control multiple DoFs at once and necessary mode-switches. This work provide…
University of Birmingham（伯明翰大学）；National Center for Nuclear Robotics（国家核机器人中心）
This paper investigates how trust, shared understanding between a human operator and a robot, and the Locus of Control (LoC) personality trait, evolve and affect Human-Robot Interaction (HRI) in mixed-initiative robotic systems. As such systems become more advanced and able to instigate actions alongside human operators, there is a shift from robots being perceived as a tool to being a team-mate. Hence, the team-oriented human factors investigated in this paper (i.e. trust, shared understanding, and LoC) can play a crucial role in efficient HRI. Here, we present the results from an experiment …
Shared autonomy teleoperation can guarantee safety, but does so by reducing the human operator's control authority, which can lead to reduced levels of human-robot agreement and user satisfaction. This paper presents a novel haptic shared autonomy teleoperation paradigm that uses haptic feedback to inform the user about the inner state of a shared autonomy paradigm, while still guaranteeing safety. This differs from haptic shared control, which uses haptic feedback to inform the user's actions, but gives the human operator full control over the robot's actions. We conducted a user study in whi…
Investigating the Effect of LED Signals and Emotional Displays in Human-Robot Shared Workspaces
German International University（德国国际大学）；Intelligent Autonomous Systems Lab（智能自主系统实验室）；Department of Computer Science, TU Darmstadt（计算机科学系，图恩-达姆施塔特技术大学）；German Research Center for AI (DFKI)（德国人工智能研究中心）；Centre for Cognitive Science, TU Darmstadt（认知科学中心，图恩-达姆施塔特技术大学）；Hessian Center for Artificial Intelligence (Hessian.AI)（黑森人工智能中心）
Beyond Omakase: Designing Shared Control for Navigation Robots with Blind People
University of Maryland, College Park（马里兰大学学院公园分校）；IBM Research - Tokyo（IBM东京研究院）；Waseda University（早稻田大学）；Miraikan - The National Museum of Emerging Science and Innovation（未来馆（日本未来科学馆））
Mobile telepresence robots (MTRs) have become increasingly popular in the expanding world of remote work, providing new avenues for people to actively participate in activities at a distance. However, humans operating MTRs often have difficulty navigating in densely populated environments due to limited situation awareness and narrow field-of-view, which reduces user acceptance and satisfaction. Shared autonomy in navigation has been studied primarily in static environments or in situations where only one pedestrian interacts with the robot. We present a multimodal shared autonomy approach, le…
