Task-Space Control Interface for SoftBank Humanoid Robots and its Human-Robot Interaction Applications
SoftBank Robotics Europe（软银欧洲机器人公司）；University of Montpellier–CNRS LIRMM（蒙彼利埃大学-法国国家科学研究中心蒙彼利埃信息与机器人实验室）；Beijing Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology（北京理工大学智能机器人与系统高精尖创新中心）；CNRS-AIST Joint Robotics Laboratory（法国国家科学研究中心-日本产业技术综合研究所联合机器人实验室）
Anastasia Bolotnikova, Pierre Gergondet, Arnaud Tanguy, Sébastien Courtois, Abderrahmane Kheddar
备注：
IEEE/SICE 13th International Symposium on System Integration (SII 2021)
We present an open-source software interface, called mc_naoqi, that allows to perform whole-body task-space Quadratic Programming based control, implemented in mc_rtc framework, on the SoftBank Robotics Europe humanoid robots. We describe the control interface, associated robot description packages, robot modules and sample whole-body controllers. We demonstrate the use of these tools in simulation for a robot interacting with a human model. Finally, we showcase and discuss the use of the developed open-source tools for running the human-robot close contact interaction experiments with real human subjects inspired from assistance scenarios.
In this paper, we illustrate our work on improving the accessibility of Cyber-Physical Systems (CPS), presenting a study on human-robot interaction where the end-users are either deaf or hearing-impaired people. Current trends in robotic designs include devices with robotic arms and hands capable of performing manipulation and grasping tasks. This paper focuses on how these devices can be used for a different purpose, which is that of enabling robotic communication via sign language. For the study, several tests and questionnaires are run to check and measure how end-users feel about interpret…
Istituto Italiano di Tecnologia（意大利理工学院）；Florida Institute for Human and Machine Cognition（佛罗里达人机认知研究所）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所）；Inria（法国国家信息与自动化研究所）；Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Tokyo University of Science（东京理科大学）
Teleoperation of humanoid robots enables the integration of the cognitive skills and domain expertise of humans with the physical capabilities of humanoid robots. The operational versatility of humanoid robots makes them the ideal platform for a wide range of applications when teleoperating in a remote environment. However, the complexity of humanoid robots imposes challenges for teleoperation, particularly in unstructured dynamic environments with limited communication. Many advancements have been achieved in the last decades in this area, but a comprehensive overview is still missing. This s…
VividFace: Real-Time and Realistic Facial Expression Shadowing for Humanoid Robots
School of Computing, Macquarie University（麦考瑞大学计算机学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；Anuradha and Vikas Sinha Department of Data Science, University of North Texas（北卡罗来纳州立大学数据科学系）
Istituto Italiano di Tecnologia（意大利理工学院）；The University of Manchester（曼彻斯特大学）
When a human and a humanoid robot collaborate physically, ergonomics is a key factor to consider. Assuming a given humanoid robot, several control architectures exist nowadays to address ergonomic physical human-robot collaboration. This paper takes one step further by considering robot hardware parameters as optimization variables in the problem of collaborative payload lifting. The variables that parametrize robot's kinematics and dynamics ensure their physical consistency, and the human model is considered in the optimization problem. By leveraging the proposed modelling framework, the ergo…
Institute for Media Innovation, Nanyang Technological University（南洋理工大学媒体创新研究所）；MIRALab, University of Geneva（日内瓦大学MIRALab实验室）
There are considerable advancements in medical health care in recent years, resulting in rising older population. As the workforce for such a population is not keeping pace, there is an urgent need to address this problem. Having robots to stimulating recreational activities for older adults can reduce the workload for caretakers and give them time to address the emotional needs of the elderly. In this paper, we investigate the effects of the humanoid social robot Nadine as an activity host for the elderly. This study aims to analyse if the elderly feels comfortable and enjoy playing game/acti…
We perform a Systematic Literature Review to discover how Humanoid robots are being applied in Socially Assistive Robotics experiments. Our search returned 24 papers, from which 16 were included for closer analysis. To do this analysis we used a conceptual framework inspired by Behavior-based Robotics. We were interested in finding out which robot was used (most use the robot NAO), what the goals of the application were (teaching, assisting, playing, instructing), how the robot was controlled (manually in most of the experiments), what kind of behaviors the robot exhibited (reacting to touch, …
Istituto Italiano di Tecnologia（意大利理工学院）；IRI CSIC-UPC（西班牙国家研究委员会-加泰罗尼亚理工大学机器人研究所）；AIST（日本产业技术综合研究所）；LAAS-CNRS（法国国家科学研究中心系统分析与架构实验室）；DIBRIS-UNIGE（热那亚大学计算机、生物工程、机器人与系统工程系）
The future where the industrial shop-floors witness humans and robots working in unison and the domestic households becoming a shared space for both these agents is not very far. The scientific community has been accelerating towards that future by extending their research efforts in human-robot interaction towards human-robot collaboration. It is possible that the anthropomorphic nature of the humanoid robots could deem the most suitable for such collaborations in semi-structured, human-centered environments. Wearable sensing technologies for human agents and efficient human-aware control str…
Istituto Italiano di Tecnologia（意大利技术研究院）；University of Genova（热那亚大学）；University of Manchester（曼彻斯特大学）；DeepMind（深度思维）
The topic of physical human-robot interaction received a lot of attention from the robotics community because of many promising application domains. However, studying physical interaction between a robot and an external agent, like a human or another robot, without considering the dynamics of both the systems may lead to many short-comings in fully exploiting the interaction. In this paper, we present a coupled-dynamics formalism followed by a sound approach in exploiting helpful interaction with a humanoid robot. In particular, we propose the first attempt to define and exploit the human help…
With an increasing need for elderly and disability care, there is an increasing opportunity for intelligent and mobile devices such as robots to provide care and support solutions. In order to naturally assist and interact with humans, a robot must possess effective conversational capabilities. Gestures accompanying spoken sentences are an important factor in human-to-human conversational communication. Humanoid robots must also use gestures if they are to be capable of the rich interactions implied and afforded by their humanlike appearance. However, present systems for gesture generation do …
University of the West of England（西英格兰大学）；Bristol Robotics Laboratory（布里斯托机器人实验室）；University of Florida（佛罗里达大学）；European Commission, Joint Research Centre（欧盟委员会联合研究中心）
For a period of three weeks in June 2021, we embedded a social robot (Softbank Pepper) in a Special Educational Needs (SEN) school, with a focus on supporting the well-being of autistic children. Our methodology to design and embed the robot among this vulnerable population follows a comprehensive participatory approach. We used the research project as a test-bed to demonstrate in a complex real-world environment the importance and suitability of the nine UNICEF guidelines on AI for Children. The UNICEF guidelines on AI for Children closely align with several of the UN goals for sustainable de…
We present data from two online human-robot interaction experiments where 227 participants viewed videos of a humanoid robot exhibiting faulty or non-faulty behaviours while either remaining mute or speaking. The participants were asked to evaluate their perception of the robot's trustworthiness, as well as its likeability, animacy, and perceived intelligence. The results show that, while a non-faulty robot achieves the highest trust, an apparently faulty robot that can speak manages to almost completely mitigate the loss of trust that is otherwise seen with faulty behaviour. We theorize that …
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Technical University of Munich（慕尼黑工业大学）
Humanoid robots are designed to operate in human centered environments where they execute a multitude of challenging tasks, each differing in complexity, resource requirements, and execution time. In such highly dynamic surroundings it is desirable to anticipate upcoming situations in order to predict future resource requirements such as CPU or memory usage. Resource prediction information is essential for detecting upcoming resource bottlenecks or conflicts and can be used enhance resource negotiation processes or to perform speculative resource allocation. In this paper we present a predicti…
Examining the legibility of humanoid robot arm movements in a pointing task
Faculty of Mathematics, Physics；Informatics, Comenius University Bratislava, Slovakia DIBRIS, University of Genoa, Genoa, Italy CONTACT, Italian Institute of Technology, Genoa, Italy Digital Health Dept., Nvision Systems \& Technologies, S.L., Barcelona, Spain Faculty of Education, University of Ljubljana, Slovenia
University of Pennsylvania（宾夕法尼亚大学）；Semio AI, Inc.（Semio AI公司）；USC（南加州大学）
Hardware platforms for socially interactive robotics can be limited by cost or lack of functionality. This paper presents the overall system -- design, hardware, and software -- for Quori, a novel, affordable, socially interactive humanoid robot platform for facilitating non-contact human-robot interaction (HRI) research. The design of the system is motivated by feedback sampled from the HRI research community. The overall design maintains a balance of affordability and functionality. Initial Quori testing and a six-month deployment are presented. Ten Quori platforms have been awarded to a div…
For several years, high development and production costs of humanoid robots restricted researchers interested in working in the field. To overcome this problem, several research groups have opted to work with simulated or smaller robots, whose acquisition costs are significantly lower. However, due to scale differences and imperfect simulation replicability, results may not be directly reproducible on real, adult-sized robots. In this paper, we present the NimbRo-OP2X, a capable and affordable adult-sized humanoid platform aiming to significantly lower the entry barrier for humanoid robot rese…
In order to detect and correct physical exercises, a Grow-When-Required Network (GWR) with recurrent connections, episodic memory and a novel subnode mechanism is developed in order to learn spatiotemporal relationships of body movements and poses. Once an exercise is performed, the information of pose and movement per frame is stored in the GWR. For every frame, the current pose and motion pair is compared against a predicted output of the GWR, allowing for feedback not only on the pose but also on the velocity of the motion. In a practical scenario, a physical exercise is performed by an exp…
Syracuse University（雪城大学）；UnityPoint Health（UnityPoint健康）；Stevens Institute of Technology（史蒂文斯理工学院）；Binghamton University（宾汉姆顿大学）；Texas Tech University（得克萨斯理工大学）
Application of humanoid robots has been common in the field of healthcare and education. It has been recurrently used to improve social behavior and mollify distress level among children with autism, cancer and cerebral palsy. This article discusses the same from a human factors perspective. It shows how people of different age and gender have a different opinion towards the application and acceptance of humanoid robots. Additionally, this article highlights the influence of cerebral condition and social interaction on a user behavior and attitude towards humanoid robots. Our study performed a…
E2H: A Two-Stage Non-Invasive Neural Signal Driven Humanoid Robotic Whole-Body Control Framework
University of Technology Sydney（悉尼科技大学）；The Hong Kong University of Science and Technology（香港科技大学）；Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）
The Cloud-based Advanced Robotics Laboratory (CARL) integrates a whole body controller and web-based teleoperation to enable any device with a web browser to access and control a humanoid robot. By integrating humanoid robots with the cloud, they are accessible from any Internet-connected device. Increased accessibility is important because few people have access to state-of-the-art humanoid robots limiting their rate of development. CARL's implementation is based on modern software libraries, frameworks, and middleware including Node.js, Socket.IO, ZMQ, ROS, Robot Web Tools, and ControlIt! Fe…
A common architecture for torque controlled humanoid robots consists in two nested loops. The outer loop generates desired joint/motor torques, and the inner loop stabilises these desired values. In doing so, the inner loop usually compensates for joint friction phenomena, thus removing their inherent stabilising property that may be also beneficial for high level control objectives. This paper shows how to exploit friction for joint and task space control of humanoid robots. Experiments are carried out using the humanoid robot iCub.
In this paper, a combination of ankle and hip strategy is used for push recovery of a position-controlled humanoid robot. Ankle strategy and hip strategy are equivalent to Center of Pressure (CoP) and Centroidal Moment Pivot (CMP) regulation respectively. For controlling the CMP and CoP we need a torque-controlled robot, however most of the conventional humanoid robots are position controlled. In this regard, we present an efficient way for implementation of the hip and ankle strategies on a position controlled humanoid robot. We employ a feedback controller to compensate the capture point err…
Our research explores the potential of a humanoid robot for work in unpredictable environments, but controlling a humanoid robot remains a very difficult problem. In our previous work, we designed a prototype virtual reality (VR) interface to allow an operator to command a humanoid robot. However, while usable, the initial interface was not sufficient for commanding the robot to perform the tasks; for example, in some cases, there was a lack of precision available for robot control. The interface was overly cumbersome in some areas as well. In this paper, we discuss numerous additions, inspire…
This position paper introduces the results of an initial card sorting experiment based on the reactions and questions of a group of children with autism working with a humanoid robot in a therapeutic laboratory on autonomy.
This paper describes the development of a humanoid robot called ARDOP. The goal of the project is to provide a modular, open-source, and inexpensive humanoid robot that would enable researchers to answer various problems related to robotic manipulation and perception. ARDOP primarily comprises of two functional units namely the perception and manipulation system, here we discuss the conceptualization and design methodology of these systems and proceed to present their performance results on simulation and various custom-designed experiments.
Forthcoming applications concerning humanoid robots may involve physical interaction between the robot and a dynamic environment. In such scenario, classical balancing and walking controllers that neglect the environment dynamics may not be sufficient for achieving a stable robot behavior. This paper presents a modeling and control framework for balancing humanoid robots in contact with a dynamic environment. We first model the robot and environment dynamics, together with the contact constraints. Then, a control strategy for stabilizing the full system is proposed. Theoretical results are ver…
Istituto Italiano di Tecnologia（意大利理工学院）；University of Genova（热那亚大学）；The University of Manchester（曼彻斯特大学）
Humanoid robot teleoperation allows humans to integrate their cognitive capabilities with the apparatus to perform tasks that need high strength, manoeuvrability and dexterity. This paper presents a framework for teleoperation of humanoid robots using a novel approach for motion retargeting through inverse kinematics over the robot model. The proposed method enhances scalability for retargeting, i.e., it allows teleoperating different robots by different human users with minimal changes to the proposed system. Our framework enables an intuitive and natural interaction between the human operato…
Most humanoid robots have highly complicated structure and design of robots that are very similar to human is extremely difficult. In this paper, modelling of a general and comprehensive algorithm for control of humanoid robots is presented using Colored Petri Nets. For keeping dynamic balance of the robot, combination of Gyroscope and Accelerometer sensors are used in algorithm. Image processing is used to identify two fundamental issues: first, detection of target or an object which robot must follow; second, detecting surface of the ground so that walking robot could maintain its balance ju…
Planning balanced and collision-free motion for humanoid robots is non-trivial, especially when they are operated in complex environments, such as reaching targets behind obstacles or through narrow passages. We propose a method that allows us to apply existing sampling--based algorithms to plan trajectories for humanoids by utilizing a customized state space representation, biased sampling strategies, and a steering function based on a robust inverse kinematics solver. Our approach requires no prior offline computation, thus one can easily transfer the work to new robot platforms. We tested t…
The use of standard robotic platforms can accelerate research and lower the entry barrier for new research groups. There exist many affordable humanoid standard platforms in the lower size ranges of up to 60cm, but larger humanoid robots quickly become less affordable and more difficult to operate, maintain and modify. The igus Humanoid Open Platform is a new and affordable, fully open-source humanoid platform. At 92cm in height, the robot is capable of interacting in an environment meant for humans, and is equipped with enough sensors, actuators and computing power to support researchers in m…
DFKI GmbH（德国人工智能研究中心）；University of Edinburgh（爱丁堡大学）；Alan Turing Institute（艾伦·图灵研究所）；LAAS-CNRS（法国国家科学研究中心系统分析与架构实验室）
Last decades of humanoid research has shown that humanoids developed for high dynamic performance require a stiff structure and optimal distribution of mass--inertial properties. Humanoid robots built with a purely tree type architecture tend to be bulky and usually suffer from velocity and force/torque limitations. This paper presents a novel series-parallel hybrid humanoid called RH5 which is 2 m tall and weighs only 62.5 kg capable of performing heavy-duty dynamic tasks with 5 kg payloads in each hand. The analysis and control of this humanoid is performed with whole-body trajectory optimiz…
National Institute of Applied Sciences and Technology（国家应用科学与技术学院）
In this paper, the dynamic modeling of a seven linked humanoid robot, is accurately developed, in the three dimensional space using the Newton-Euler formalism. The aim of this study is to provide a clear and a systematic approach so that starting from generalized motion equations of all rigid bodies of the humanoid robot one can establish a reduced dynamical model. The resulting model can be expended either for simulation propositions or implemented for any given control law. In addition, transformations and developments, proposed here, can be exploited for modeling any other three-dimensional…
Artificial and Mechanical Intelligence, Istituto Italiano di Tecnologia（人工智能与机械智能，意大利技术研究院）；School of Computer Science, University of Manchester（计算机科学学院，曼彻斯特大学）；Department of Industrial Engineering, University of Naples Federico II（工业工程系，那不勒斯费德里克二世大学）
