While humans are aware of their body and capabilities, robots are not. To address this, we present in this paper a neural network architecture that enables a dual-arm robot to get a sense of itself in an environment. Our approach is inspired by human self-awareness developmental levels and serves as the underlying building block for a robot to achieve awareness of itself while carrying out tasks in an environment. We assume that a robot has to know itself before interacting with the environment in order to be able to support different robotic tasks. Hence, we implemented a neural network architecture to enable a robot to differentiate its limbs from the environment using visual and proprioception sensory inputs. We demonstrate experimentally that a robot can distinguish itself with an accuracy of 88.7% on average in cluttered environmental settings and under confounding input signals.
Hanson Robotics Ltd（汉森机器人有限公司）；Indian Institute of Science（印度科学学院）；ARTPARK（ARTPARK（人工智能与机器人技术园区））
Open Arms is a novel open-source platform of realistic human-like robotic hands and arms hardware with 28 Degree-of-Freedom (DoF), designed to extend the capabilities and accessibility of humanoid robotic grasping and manipulation. The Open Arms framework includes an open SDK and development environment, simulation tools, and application development tools to build and operate Open Arms. This paper describes these hands controls, sensing, mechanisms, aesthetic design, and manufacturing and their real-world applications with a teleoperated nursing robot. From 2015 to 2022, the authors have desig…
University of California, Berkeley（加州大学伯克利分校）；Google（谷歌）
Autonomous fabric manipulation is a longstanding challenge in robotics, but evaluating progress is difficult due to the cost and diversity of robot hardware. Using Reach, a cloud robotics platform that enables low-latency remote execution of control policies on physical robots, we present the first systematic benchmarking of fabric manipulation algorithms on physical hardware. We develop 4 novel learning-based algorithms that model expert actions, keypoints, reward functions, and dynamic motions, and we compare these against 4 learning-free and inverse dynamics algorithms on the task of foldin…
Accenture Labs（埃森哲实验室）；National Research Council Canada（加拿大国家研究委员会）
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic environments, we show that the robot arm can learn the operational space and complete tasks faster over time. We also demonstrate that the adaptive robot control algorithm based on SNNs enables a fast res…
RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)
The University of Hong Kong（香港大学）；AgileX Robotics（极目机器人）；Shanghai AI Laboratory（上海人工智能实验室）；Shenzhen University（深圳大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model a…
Collaborative robotics requires effective communication between a robot and a human partner. This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature. These principles are evaluated through studies where English-speaking human subjects view animations of simulated robots instructing pick-and-place tasks. The evaluation distinguishes two classes of pointing actions that arise in pick-and-place tasks: referential pointing (identifying objects) and locating …
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Beckman Institute for Advanced Science and Technology（贝克曼高等科学技术研究所）；University of Michigan（密歇根大学）
Many soft-body organisms found in nature flourish underwater. Similarly, soft robots are potentially well-suited for underwater environments partly because the problematic effects of gravity, friction, and harmonic oscillations are less severe underwater. However, it remains a challenge to design, fabricate, waterproof, model, and control underwater soft robotic systems. Furthermore, submersible robots usually do not have configurable components because of the need for sealed electronics and mechanical elements. This work presents the development of a modular and submersible soft robotic arm d…
We outline the design and construction of novel robotic arms using machine perception, convolutional neural networks, and symbolic AI for logical control and affordance indexing. We describe our robotic arms built with a humanlike mechanical configuration and aesthetic, with 28 degrees of freedom, touch sensors, and series elastic actuators. The arms were modelled in Roodle and Gazebo with URDF models, as well as Unity, and implement motion control solutions for solving live games of Baccarat (the casino card game), rock paper scissors, handshaking, and drawing. This includes live interactions…
Generating and Customizing Robotic Arm Trajectories using Neural Networks
Faculty of Mathematics, Physics and Informatics（数学、物理与信息学系）；Comenius University Bratislava（布拉迪斯拉发康门尼大学）；DIBRIS（迪布里斯）；University of Genoa（热那亚大学）；Italian Institute of Technology（意大利理工学院）
Osaka University（大阪大学）；National Inst. of AIST（国立先进工业科学技术研究所）
In this paper, we present a planner for a master dual-arm robot to manipulate tethered tools with an assistant dual-arm robot's help. The assistant robot provides assistance to the master robot by manipulating the tool cable and avoiding collisions. The provided assistance allows the master robot to perform tool placements on the robot workspace table to regrasp the tool, which would typically fail since the tool cable tension may change the tool positions. It also allows the master robot to perform tool handovers, which would normally cause entanglements or collisions with the cable and the e…
We present an interactive, computational design system for creating custom robotic arms given high-level task descriptions and environmental constraints. Various task requirements can be encoded as desired motion trajectories for the robot arm's end-effector. Given such end-effector trajectories, our system enables on-demand design of custom robot arms using a library of modular and reconfigurable parts such as actuators and connecting links. By searching through the combinatorial set of possible arrangements of these parts, our method generates a functional, as-simple-as-possible robot arm th…
Osaka University（大阪大学）；National Inst. of AIST（日本产业技术综合研究所）
This paper develops a planner that plans the action sequences and motion for a dual-arm robot to lift up and flip heavy plates using crane pulley blocks. The problem is motivated by the low payload of modern collaborative robots. Instead of directly manipulating heavy plates that collaborative robots cannot afford, the paper develops a planner for collaborative robots to operate crane pulley blocks. The planner assumes a target plate is pre-attached to the crane hook. It optimizes dual-arm action sequences and plans the robot's dual-arm motion that pulls the rope of the crane pulley blocks to …
Ubtech Robotics Corp.（优必选机器人公司）；Tsinghua University（清华大学）；Beijing Innovation Center for Future Chips, Tsinghua University（清华大学北京未来芯片创新中心）
When a dual-arm robot clamps a rigid object in an environment for human beings, the environment or the collaborating human will impose incidental disturbance on the operated object or the robot arm, leading to clamping failure, damaging the robot even hurting the human. This research proposes a prioritized hierarchical compliance control to simultaneously deal with the two types of disturbances in the dual-arm robot clamping. First, we use hierarchical quadratic programming (HQP) to solve the robot inverse kinematics under the joint constraints and prioritize the compliance for the disturbance…
The purpose of this research is to design, fabricate and test a tendon driven a continuum soft robot with three modular segments, each of which has a tunable stiffness enabled by layer jamming technology. Compared with previous studies, the robotic arm design of this project has a modular structure, which means the length of the robotic arm can be adjusted by addition of extra arm modules/segments to the existing robotic prototype. Furthermore, the new arm prototype supports motion within a 3-dimensional space. To achieve the goals, the design and fabrication for the variable stiffness robotic…
The compliance of soft robotic arms renders the development of accurate kinematic & dynamical models especially challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC fails to effectively handle external forces, or even the influence of gravity, since the robot does not deform with a constant curvature under these conditions. In this paper, we establish three-dimensional (3D) modeling of a multi-segment soft robotic arm under the less restrictive assumption that each segment of the arm is deformed on a plane without twisting.…
Tecnalia France（法国泰克纳利亚）；LAAS-CNRS（法国国家科学研究中心自动化系统与机器人实验室）；University of Toulouse（图卢兹大学）；LIRMM（蒙彼利埃大学与法国国家科学研究中心联合机器人与微电子机械系统实验室）；Université de Montpellier（蒙彼利埃大学）；CNRS（法国国家科学研究中心）
A continuous collision checking method for a cable-driven parallel robot with an embarked robotic arm is proposed in this paper. The method aims at validating paths by checking for collisions between any pair of robot bodies (mobile platform, cables, and arm links). For a pair of bodies, an upper bound on their relative velocity and a lower bound on the distance between the bodies are computed and used to validate a portion of the path. These computations are done repeatedly until a collision is found or the path is validated. The method is integrated within the Humanoid Path Planner (HPP) sof…
In the scope of the "Chess-Bot" project, this study's goal is to choose the right model for the robotic arm that the "the Chess-Bot" will use to move the pawn from a cell to another. In this paper, there is the definition and the structure of a robot arm. Also, the different engineering and kinematics fundamentals of the robot and its components will be detailed. Furthermore, the different structures of robotic arms will be presented and compared based on different criteria. Finally, a model for "the Chess-Bot" arm will be synthesized based on accurate algorithms and equations.
Precision is a crucial performance indicator for robot arms, as high precision manipulation allows for a wider range of applications. Traditional methods for improving robot arm precision rely on error compensation. However, these methods are often not robust and lack adaptability. Learning-based methods offer greater flexibility and adaptability, while current researches show that they often fall short in achieving high precision and struggle to handle many scenarios requiring high precision. In this paper, we propose a novel high-precision robot arm manipulation framework based on online ite…
Imitation Learning (IL) is a powerful paradigm to teach robots to perform manipulation tasks by allowing them to learn from human demonstrations collected via teleoperation, but has mostly been limited to single-arm manipulation. However, many real-world tasks require multiple arms, such as lifting a heavy object or assembling a desk. Unfortunately, applying IL to multi-arm manipulation tasks has been challenging -- asking a human to control more than one robotic arm can impose significant cognitive burden and is often only possible for a maximum of two robot arms. To address these challenges,…
Chongqing University of Posts and Telecommunications（重庆邮电大学）；Swansea University（斯旺西大学）
Industrial robot arms are extensively important for intelligent manufacturing. An industrial robot arm commonly enjoys its high repetitive positioning accuracy while suffering from its low absolute positioning accuracy, which greatly restricts its application in high-precision manufacture, like automobile manufacture. Aiming at addressing this hot issue, this work proposes a novel robot arm calibration method based on cubic interpolated beetle antennae search (CIBAS). This study has three ideas: a) developing a novel CIBAS algorithm, which can effectively addresses the issue of local optimum i…
A large number of robotic and human-assisted missions to the Moon and Mars are forecast. NASA's efforts to learn about the geology and makeup of these celestial bodies rely heavily on the use of robotic arms. The safety and redundancy aspects will be crucial when humans will be working alongside the robotic explorers. Additionally, robotic arms are crucial to satellite servicing and planned orbit debris mitigation missions. The goal of this work is to create a custom Computer Vision (CV) based Artificial Neural Network (ANN) that would be able to rapidly identify the posture of a 7 Degree of F…
This paper investigates the problem of the dynamic balance control of multi-arm free-floating space robot during capturing an active object in close proximity. The position and orientation of space base will be affected during the operation of space manipulator because of the dynamics coupling between the manipulator and space base. This dynamics coupling is unique characteristics of space robot system. Such a disturbance will produce a serious impact between the manipulator hand and the object. To ensure reliable and precise operation, we propose to develop a space robot system consisting of …
University of Bordeaux（波尔多大学）；University Politehnica of Bucharest（布加勒斯特理工大学）
Attempts to install a rotating tool at the end of a robot arm poly-articulated date back twenty years, but these robots were not designed for that. Indeed, two essential features are necessary for machining: high rigidity and precision in a given workspace. The experimental results presented are the dynamic identification of a poly-articulated robot equipped with an integrated spindle. This study aims to highlight the influence of the geometric configuration of the robot arm on the overall stiffness of the system. The spindle is taken into account as an additional weight on board but also as a…
King’s College London（伦敦国王学院）；Xtronics Ltd.（Xtronics有限公司）；Guy’s & St Thomas’ NHS Foundation Trust（盖伊和圣托马斯NHS基金会信托）；Evelina Children’s Hospital（伊芙琳娜儿童医院）；Queen Mary University of London（伦敦玛丽女王大学）
The development of robotic-assisted extracorporeal ultrasound systems has a long history and a number of projects have been proposed since the 1990s focusing on different technical aspects. These aim to resolve the deficiencies of on-site manual manipulation of hand-held ultrasound probes. This paper presents the recent ongoing developments of a series of bespoke robotic systems, including both single-arm and dual-arm versions, for a project known as intelligent Fetal Imaging and Diagnosis (iFIND). After a brief review of the development history of the extracorporeal ultrasound robotic system …
Teleoperation provides a way for human operators to guide robots in situations where full autonomy is challenging or where direct human intervention is required. It can also be an important tool to teach robots in order to achieve autonomous behaviour later on. The increased availability of collaborative robot arms and Virtual Reality (VR) devices provides ample opportunity for development of novel teleoperation methods. Since robot arms are often kinematically different from human arms, mapping human motions to a robot in real-time is not trivial. Additionally, a human operator might steer th…
Examining the legibility of humanoid robot arm movements in a pointing task
Faculty of Mathematics, Physics；Informatics, Comenius University Bratislava, Slovakia DIBRIS, University of Genoa, Genoa, Italy CONTACT, Italian Institute of Technology, Genoa, Italy Digital Health Dept., Nvision Systems \& Technologies, S.L., Barcelona, Spain Faculty of Education, University of Ljubljana, Slovenia
Istituto Italiano di Tecnologia（意大利理工学院）；Universita di Pisa（比萨大学）
Teleoperation is a widely adopted strategy to control robotic manipulators executing complex tasks that require highly dexterous movements and critical high-level intelligence. Classical teleoperation schemes are based on either joystick control, or on more intuitive interfaces which map directly the user arm motions into one robot arm's motions. These approaches have limits when the execution of a given task requires reconfigurable multiple robotic arm systems. Indeed, the simultaneous teleoperation of two or more robot arms could extend the workspace of the manipulation cell, or increase its…
Closed kinematic chains are created whenever multiple robot arms concurrently manipulate a single object. The closed-chain constraint, when coupled with robot joint limits, dramatically changes the connectivity of the configuration space. We propose a regrasping move, termed "IK-switch", which allows efficiently bridging components of the configuration space that are otherwise mutually disconnected. This move, combined with several other developments, such as a method to stabilize the manipulated object using the environment, a new tree structure, and a compliant control scheme, enables us to …
This paper presents a planner that can automatically find an optimal assembly sequence for a dual-arm robot to assemble the soma blocks. The planner uses the mesh model of objects and the final state of the assembly to generate all possible assembly sequence and evaluate the optimal assembly sequence by considering the stability, graspability, assemblability, as well as the need for a second arm. Especially, the need for a second arm is considered when supports from worktables and other workpieces are not enough to produce a stable assembly. The planner will refer to an assisting grasp to addi…
Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax t…
University of Tübingen（蒂宾根大学）；Graz University of Technology（格拉茨技术大学）
In the paper, we show how scalable, low-cost trunk-like robotic arms can be constructed using only basic 3D-printing equipment and simple electronics. The design is based on uniform, stackable joint modules with three degrees of freedom each. Moreover, we present an approach for controlling these robots with recurrent spiking neural networks. At first, a spiking forward model learns motor-pose correlations from movement observations. After training, intentions can be projected back through unrolled spike trains of the forward model essentially routing the intention-driven motor gradients towar…
Robotic manipulation of tethered tools is widely seen in robotic work cells. They may cause excess strain on the tool's cable or undesired entanglements with the robot's arms. This paper presents a manipulation planner with cable orientation constraints for tethered tools suspended by tool balancers. The planner uses orientation constraints to limit the bending of the balancer's cable while the robot manipulates a tool and places it in a desired pose. The constraints reduce entanglements and decrease the torque induced by the cable on the robot joints. Simulation and real-world experiments sho…
KUKAloha: A General, Low-Cost, and Shared-Control based Teleoperation Framework for Construction Robot Arm
Department of Civil and Environmental Engineering, University of Michigan, United States of America（密歇根大学土木与环境工程系）；Department of Robotics, University of Michigan, United States of America（密歇根大学机器人系）
Robots are widely used in industry due to their efficiency and high accuracy in performance. One of the most intriguing issues in manufacturing stage of production line is to minimize significantly high percentage of energy consumed by these robot manipulators. The energy optimal control of robotic manipulators is a complex problem, as it requires a deep understanding of the robot's kinematics and dynamic behaviors. This paper propose a novel method of energy efficient path planning of an industrial robot arm in a workspace with multiple obstacles using differential evolution (DE) algorithm. T…
This paper presents an experimental adaptation of a non-collaborative robot arm to collaborate with the environment, as one step towards adapting legacy robotic machinery to fit in industry 4.0 requirements. A cloud-based internet of things (CIoT) service is employed to connect, supervise and control a robotic arm's motion using the added wireless sensing devices to the environment. A programmable automation controller (PAC) unit, connected to the robot arm receives the most recent changes and updates the motion of the robot arm. The experimental results show that the proposed non-expensive se…
Robotic deformable-object manipulation is a challenge in the robotic industry because deformable objects have complicated and various object states. Predicting those object states and updating manipulation planning is time-consuming and computationally expensive. In this paper, we propose learning known configurations of garments to allow a robot to recognise garment states and choose a pre-designed manipulation plan for garment flattening.
Most of industrial robots are still programmed using the typical teaching process, through the use of the robot teach pendant. In this paper is proposed an accelerometer-based system to control an industrial robot using two low-cost and small 3-axis wireless accelerometers. These accelerometers are attached to the human arms, capturing its behavior (gestures and postures). An Artificial Neural Network (ANN) trained with a back-propagation algorithm was used to recognize arm gestures and postures, which then will be used as input in the control of the robot. The aim is that the robot starts the…
Humans can leverage physical interaction to teach robot arms. As the human kinesthetically guides the robot through demonstrations, the robot learns the desired task. While prior works focus on how the robot learns, it is equally important for the human teacher to understand what their robot is learning. Visual displays can communicate this information; however, we hypothesize that visual feedback alone misses out on the physical connection between the human and robot. In this paper we introduce a novel class of soft haptic displays that wrap around the robot arm, adding signals without affect…
