This paper proposes a detailed and extensive comparison of the Trust Region Policy Optimization and DeepQ-Network with Normalized Advantage Functions with respect to other state of the art algorithms, namely Deep Deterministic Policy Gradient and Vanilla Policy Gradient. Comparisons demonstrate that the former have better performances then the latter when asking robotic arms to accomplish manipulation tasks such as reaching a random target pose and pick &placing an object. Both simulated and real-world experiments are provided. Simulation lets us show the procedures that we adopted to precisely estimate the algorithms hyper-parameters and to correctly design good policies. Real-world experiments let show that our polices, if correctly trained on simulation, can be transferred and executed in a real environment with almost no changes.
In this paper we present a Bayesian reinforcement learning framework that allows robotic manipulators to adaptively recover from random mechanical failures autonomously, hence being survivable. To this end, we formulate the framework of Bayesian Policy Morphing (BPM) that enables a robot agent to self-modify its learned policy after the diminution of its maneuvering dimensionality. We build upon existing actor-critic framework, and extend it to perform policy gradient updates as posterior learning, taking past policy updates as prior distributions. We show that policy search, in the direction …
Princeton University（普林斯顿大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Re-configurable robots have more utility and flexibility for many real-world tasks. Designing a learning agent to operate such robots requires adapting to different configurations. Here, we focus on robotic arms with multiple rigid links connected by joints. We propose a deep reinforcement learning agent with sequence neural networks embedded in the agent to adapt to robotic arms that have a varying number of links. Further, with the additional tool of domain randomization, this agent adapts to different configurations. We perform simulations on a 2D N-link arm to show the ability of our netwo…
Reinforcement learning is a promising method to accomplish robotic control tasks. The task of playing musical instruments is, however, largely unexplored because it involves the challenge of achieving sequential goals - melodies - that have a temporal dimension. In this paper, we address robotic musicianship by introducing a temporal extension to goal-conditioned reinforcement learning: Time-dependent goals. We demonstrate that these can be used to train a robotic musician to play the theremin instrument. We train the robotic agent in simulation and transfer the acquired policy to a real-world…
One of the biggest hurdles robotics faces is the facet of sophisticated and hard-to-engineer behaviors. Reinforcement learning offers a set of tools, and a framework to address this problem. In parallel, the misgivings of robotics offer a solid testing ground and evaluation metric for advancements in reinforcement learning. The two disciplines go hand-in-hand, much like the fields of Mathematics and Physics. By means of this survey paper, we aim to invigorate links between the research communities of the two disciplines by focusing on the work done in reinforcement learning for locomotive and …
University College Dublin（都柏林大学学院）；Resero Ltd（Resero有限公司）；Universidad de León（莱昂大学）
Reinforcement learning has shown great promise in robotics thanks to its ability to develop efficient robotic control procedures through self-training. In particular, reinforcement learning has been successfully applied to solving the reaching task with robotic arms. In this paper, we define a robust, reproducible and systematic experimental procedure to compare the performance of various model-free algorithms at solving this task. The policies are trained in simulation and are then transferred to a physical robotic manipulator. It is shown that augmenting the reward signal with the Hindsight …
Deep Reinforcement Learning is emerging as a promising approach for the continuous control task of robotic arm movement. However, the challenges of learning robust and versatile control capabilities are still far from being resolved for real-world applications, mainly because of two common issues of this learning paradigm: the exploration strategy and the slow learning speed, sometimes known as "the curse of dimensionality". This work aims at exploring and assessing the advantages of the application of Quantum Computing to one of the state-of-art Reinforcement Learning techniques for continuou…
Generating and Customizing Robotic Arm Trajectories using Neural Networks
Faculty of Mathematics, Physics and Informatics（数学、物理与信息学系）；Comenius University Bratislava（布拉迪斯拉发康门尼大学）；DIBRIS（迪布里斯）；University of Genoa（热那亚大学）；Italian Institute of Technology（意大利理工学院）
University of California, Berkeley（加州大学伯克利分校）；Google（谷歌）
Autonomous fabric manipulation is a longstanding challenge in robotics, but evaluating progress is difficult due to the cost and diversity of robot hardware. Using Reach, a cloud robotics platform that enables low-latency remote execution of control policies on physical robots, we present the first systematic benchmarking of fabric manipulation algorithms on physical hardware. We develop 4 novel learning-based algorithms that model expert actions, keypoints, reward functions, and dynamic motions, and we compare these against 4 learning-free and inverse dynamics algorithms on the task of foldin…
Hanson Robotics Ltd（汉森机器人有限公司）；Indian Institute of Science（印度科学学院）；ARTPARK（ARTPARK（人工智能与机器人技术园区））
Open Arms is a novel open-source platform of realistic human-like robotic hands and arms hardware with 28 Degree-of-Freedom (DoF), designed to extend the capabilities and accessibility of humanoid robotic grasping and manipulation. The Open Arms framework includes an open SDK and development environment, simulation tools, and application development tools to build and operate Open Arms. This paper describes these hands controls, sensing, mechanisms, aesthetic design, and manufacturing and their real-world applications with a teleoperated nursing robot. From 2015 to 2022, the authors have desig…
A large number of robotic and human-assisted missions to the Moon and Mars are forecast. NASA's efforts to learn about the geology and makeup of these celestial bodies rely heavily on the use of robotic arms. The safety and redundancy aspects will be crucial when humans will be working alongside the robotic explorers. Additionally, robotic arms are crucial to satellite servicing and planned orbit debris mitigation missions. The goal of this work is to create a custom Computer Vision (CV) based Artificial Neural Network (ANN) that would be able to rapidly identify the posture of a 7 Degree of F…
Inverse Kinematics (IK) solves the problem of mapping from the Cartesian space to the joint configuration space of a robotic arm. It has a wide range of applications in areas such as computer graphics, protein structure prediction, and robotics. With the vast advances of artificial neural networks (NNs), many researchers recently turned to data-driven approaches to solving the IK problem. Unfortunately, NNs become inadequate for robotic arms with redundant Degrees-of-Freedom (DoFs). This is because such arms may have multiple angle solutions to reach the same desired pose, while typical NNs on…
We present an interactive, computational design system for creating custom robotic arms given high-level task descriptions and environmental constraints. Various task requirements can be encoded as desired motion trajectories for the robot arm's end-effector. Given such end-effector trajectories, our system enables on-demand design of custom robot arms using a library of modular and reconfigurable parts such as actuators and connecting links. By searching through the combinatorial set of possible arrangements of these parts, our method generates a functional, as-simple-as-possible robot arm th…
In the scope of the "Chess-Bot" project, this study's goal is to choose the right model for the robotic arm that the "the Chess-Bot" will use to move the pawn from a cell to another. In this paper, there is the definition and the structure of a robot arm. Also, the different engineering and kinematics fundamentals of the robot and its components will be detailed. Furthermore, the different structures of robotic arms will be presented and compared based on different criteria. Finally, a model for "the Chess-Bot" arm will be synthesized based on accurate algorithms and equations.
D3-ARM: High-Dynamic, Dexterous and Fully Decoupled Cable-driven Robotic Arm
Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院人工智能与机器人中心）；School of Advanced Manufacturing, Sun Yat-sen University（中山大学先进制造学院）
The compliance of soft robotic arms renders the development of accurate kinematic & dynamical models especially challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC fails to effectively handle external forces, or even the influence of gravity, since the robot does not deform with a constant curvature under these conditions. In this paper, we establish three-dimensional (3D) modeling of a multi-segment soft robotic arm under the less restrictive assumption that each segment of the arm is deformed on a plane without twisting.…
The 'infinite' passive degrees of freedom of soft robotic arms render their control especially challenging. In this paper, we leverage a previously developed model, which drawing equivalence of the soft arm to a series of universal joints, to design two closed-loop controllers: a configuration space controller for trajectory tracking and a task space controller for position control of the end effector. Extensive experiments and simulations on a four-segment soft arm attest to substantial improvement in terms of: a) superior tracking accuracy of the configuration space controller and b) reduced…
Chittagong University of Engineering & Technology（吉大港工程与技术大学）
This paper describes a robotic arm with 5 degrees-of-freedom (DOF) which is controlled by human voice and has been developed in the Mechatronics Laboratory, CUET. This robotic arm is interfaced with a PC by serial communication (RS-232). Users' voice command is captured by a microphone, and this voice is processed by software which is made by Microsoft visual studio. Then the specific signal (obtained by signal processing) is sent to control unit. The main control unit that is used in the robotic arm is a microcontroller whose model no. is PIC18f452. Then Control unit drives the actuators, (Hi…
Western Macedonia Univ. of Applied Sciences（西马其顿应用科学大学）
The purpose of this paper is to present the construction and programming of a five degrees of freedom robotic arm which interacts with an infrared sensor for the identification and sorting of different sized objects. The main axis of the construction design will be up to the three main branches of science that make up the Mechatronics: Mechanical Engineering, Electronic-Electrical Engineering and Computer Engineering. The methods that have been used for the construction are presented as well as the methods for the programming of the arm in cooperation with the sensor. The aim is to present the…
Design And Control of A Robotic Arm For Industrial Applications
CMR Technical Campus（CMR技术学院）；JNTUH University College of Engineering, Science & Technology Hyderabad（贾瓦哈拉尔·尼赫鲁科技大学海得拉巴工程科学与技术学院）；Holy Mary Institute of Technology（圣玛丽技术学院）
We propose an approach based on probabilistic models, in particular POMDPs, to plan optimized search processes of known objects by intelligent eye in hand robotic arms. Searching and reaching for a known object (a pen, a book, or a hammer) in one's office is an operation that humans perform frequently in their daily activities. There is no reason why intelligent robotic arms would not encounter this problem frequently in the various applications in which they are expected to serve. The problem suffers from uncertainties coming both from the lack of information about the position of the object,…
Robotic deformable-object manipulation is a challenge in the robotic industry because deformable objects have complicated and various object states. Predicting those object states and updating manipulation planning is time-consuming and computationally expensive. In this paper, we propose learning known configurations of garments to allow a robot to recognise garment states and choose a pre-designed manipulation plan for garment flattening.
The purpose of this research is to design, fabricate and test a tendon driven a continuum soft robot with three modular segments, each of which has a tunable stiffness enabled by layer jamming technology. Compared with previous studies, the robotic arm design of this project has a modular structure, which means the length of the robotic arm can be adjusted by addition of extra arm modules/segments to the existing robotic prototype. Furthermore, the new arm prototype supports motion within a 3-dimensional space. To achieve the goals, the design and fabrication for the variable stiffness robotic…
Accenture Labs（埃森哲实验室）；National Research Council Canada（加拿大国家研究委员会）
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic environments, we show that the robot arm can learn the operational space and complete tasks faster over time. We also demonstrate that the adaptive robot control algorithm based on SNNs enables a fast res…
Recent advances in telecommunications have enabled surgeons to operate remotely on patients with the use of robotics. The investigation and testing of remote surgery using a robotic arm is presented. The robotic arm is designed to have four degrees of freedom that track the surgeon's x, y, z positions and the rotation angle of the forearm θ. The system comprises two main subsystems viz. the detecting and actuating systems. The detection system uses infrared light-emitting diodes, a retroreflective bracelet and two infrared cameras which as a whole determine the coordinates of the surgeon's for…
This study examines a biology-inspired approach of using reconfigurable articulation to reduce the control requirement for soft robotic arms. We construct a robotic arm by assembling Kresling origami modules that exhibit predictable bistability. Via switching between their two stable states, these origami modules can behave either like a flexible joint with low bending stiffness or like a stiff link with high stiffness, without requiring any continuous power supply. In this way, the robotic arm can exhibit pseudo-linkage kinematics with lower control requirements and improved motion accuracy. …
Westphalian University of Applied Sciences（威斯特伐利亚应用科学大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
Living a self-determined life independent of human caregivers or fully autonomous robots is a crucial factor for human dignity and the preservation of self-worth for people with motor impairments. Assistive robotic solutions - particularly robotic arms - are frequently deployed in domestic care, empowering people with motor impairments in performing ADLs independently. However, while assistive robotic arms can help them perform ADLs, currently available controls are highly complex and time-consuming due to the need to control multiple DoFs at once and necessary mode-switches. This work provide…
This paper presents a robust control scheme for flexible link robotic manipulators, which is based on considering the flexible mechanical structure as a system with slow (rigid) and fast (flexible) modes that can be controlled separately. The rigid dynamics is controlled by means of a robust sliding-mode approach with wellestablished stability properties while an LQR optimal design is adopted for the flexible dynamics. Experimental results show that this composite approach achieves good closed loop tracking properties both for the rigid and the flexible dynamics.
North Carolina State University（北卡罗来纳州立大学）；Indian Institute of Technology Patna（印度理工学院巴特那分校）；Zhejiang University（浙江大学）
Over the past few years, the use of camera-equipped robotic platforms for data collection and visually monitoring applications has exponentially grown. Cluttered construction sites with many objects (e.g., bricks, pipes, etc.) on the ground are challenging environments for a mobile unmanned ground vehicle (UGV) to navigate. To address this issue, this study presents a mobile UGV equipped with a stereo camera and a robotic arm that can remove obstacles along the UGV's path. To achieve this objective, the surrounding environment is captured by the stereo camera and obstacles are detected. The ob…
Controlling soft continuum robotic arms is challenging due to their hyper-redundancy and dexterity. In this paper we demonstrate, for the first time, closed-loop control of the configuration space variables of a soft robotic arm, composed of independently controllable segments, using a Cosserat rod model of the robot and the distributed sensing and actuation capabilities of the segments. Our controller solves the inverse dynamic problem by simulating the Cosserat rod model in MATLAB using a computationally efficient numerical solution scheme, and it applies the computed control output to the a…
King’s College London（伦敦国王学院）；Xtronics Ltd.（Xtronics有限公司）；Guy’s & St Thomas’ NHS Foundation Trust（盖伊和圣托马斯NHS基金会信托）；Evelina Children’s Hospital（伊芙琳娜儿童医院）；Queen Mary University of London（伦敦玛丽女王大学）
The development of robotic-assisted extracorporeal ultrasound systems has a long history and a number of projects have been proposed since the 1990s focusing on different technical aspects. These aim to resolve the deficiencies of on-site manual manipulation of hand-held ultrasound probes. This paper presents the recent ongoing developments of a series of bespoke robotic systems, including both single-arm and dual-arm versions, for a project known as intelligent Fetal Imaging and Diagnosis (iFIND). After a brief review of the development history of the extracorporeal ultrasound robotic system …
Design and Control of a Novel Six-Degree-of-Freedom Hybrid Robotic Arm
Intelligent Equipment Research Center, Beijing Academy of Agriculture and Forestry Sciences（北京农林科学院智能装备研究中心）；School of Mechanical Electrical Engineering and Automation, Shanghai University（上海大学机电工程与自动化学院）
Shanghai Jiao Tong University（上海交通大学）；University of Oxford（牛津大学）；Tencent（腾讯）
Collision avoidance is a widely investigated topic in robotic applications. When applying collision avoidance techniques to a mobile robot, how to deal with the spatial structure of the robot still remains a challenge. In this paper, we design a configuration-aware safe control law by solving a Quadratic Programming (QP) with designed Control Barrier Functions (CBFs) constraints, which can safely navigate a mobile robotic arm to a desired region while avoiding collision with environmental obstacles. The advantage of our approach is that it correctly and in an elegant way incorporates the spati…
Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model a…
Closed kinematic chains are created whenever multiple robot arms concurrently manipulate a single object. The closed-chain constraint, when coupled with robot joint limits, dramatically changes the connectivity of the configuration space. We propose a regrasping move, termed "IK-switch", which allows efficiently bridging components of the configuration space that are otherwise mutually disconnected. This move, combined with several other developments, such as a method to stabilize the manipulated object using the environment, a new tree structure, and a compliant control scheme, enables us to …
Tsinghua University（清华大学）；University of Alberta（阿尔伯塔大学）
One of the most efficient ways for a learning-based robotic arm to learn to process complex tasks as human, is to directly learn from observing how human complete those tasks, and then imitate. Our idea is based on success of Deep Q-Learning (DQN) algorithm according to reinforcement learning, and then extend to Deep Deterministic Policy Gradient (DDPG) algorithm. We developed a learning-based method, combining modified DDPG and visual imitation network. Our approach acquires frames only from a monocular camera, and no need to either construct a 3D environment or generate actual points. The re…
This paper focuses on the development of a ground-based test-bed to analyze the complexities of contact dynamics between multibody systems in space. The test-bed consists of an air-bearing platform equipped with a 7 degrees-of-freedom (one degree per revolute joint) robotic arm which acts as the servicing satellite. The dynamics of the manipulator on the platform is modeled as an aid for the analysis and design of stabilizing control algorithms suited for autonomous on-orbit servicing missions. The dynamics are represented analytically using a recursive Newton-Euler multibody method with D-H p…
Tecnalia France（法国泰克纳利亚）；LAAS-CNRS（法国国家科学研究中心自动化系统与机器人实验室）；University of Toulouse（图卢兹大学）；LIRMM（蒙彼利埃大学与法国国家科学研究中心联合机器人与微电子机械系统实验室）；Université de Montpellier（蒙彼利埃大学）；CNRS（法国国家科学研究中心）
A continuous collision checking method for a cable-driven parallel robot with an embarked robotic arm is proposed in this paper. The method aims at validating paths by checking for collisions between any pair of robot bodies (mobile platform, cables, and arm links). For a pair of bodies, an upper bound on their relative velocity and a lower bound on the distance between the bodies are computed and used to validate a portion of the path. These computations are done repeatedly until a collision is found or the path is validated. The method is integrated within the Humanoid Path Planner (HPP) sof…
This paper presents an experimental adaptation of a non-collaborative robot arm to collaborate with the environment, as one step towards adapting legacy robotic machinery to fit in industry 4.0 requirements. A cloud-based internet of things (CIoT) service is employed to connect, supervise and control a robotic arm's motion using the added wireless sensing devices to the environment. A programmable automation controller (PAC) unit, connected to the robot arm receives the most recent changes and updates the motion of the robot arm. The experimental results show that the proposed non-expensive se…
