This paper presents the application of a learning control approach for the realization of a fast and reliable pick-and-place application with a spherical soft robotic arm. The arm is characterized by a lightweight design and exhibits compliant behavior due to the soft materials deployed. A soft, continuum joint is employed, which allows for simultaneous control of one translational and two rotational degrees of freedom in a single joint. This allows us to axially approach and pick an object with the attached suction cup during the pick-and-place application. A control allocation based on pressure differences and the antagonistic actuator configuration is introduced, allowing decoupling of the system dynamics and simplifying the modeling and control. A linear parameter-varying model is identified, which is parametrized by the attached load mass and a parameter related to the joint stiffness. A gain-scheduled feedback controller is proposed, which asymptotically stabilizes the robotic system for aggressive tuning and over large variations of the parameters considered. The control architecture is augmented with an iterative learning control scheme enabling accurate tracking of aggressive trajectories involving set point transitions of 60 degrees within 0.3 seconds (no mass attached) to 0.6 seconds (load mass attached). The modeling and control approach proposed results in a reliable realization of a pick-and-place application and is experimentally demonstrated.
Swiss Federal Institute of Technology in Zürich (ETH Zürich)（苏黎世联邦理工学院）
Flying manipulators are aerial drones with attached rigid-bodied robotic arms and belong to the latest and most actively developed research areas in robotics. The rigid nature of these arms often lack compliance, flexibility, and smoothness in movement. This work proposes to use a soft-bodied robotic arm attached to an omnidirectional micro aerial vehicle (OMAV) to leverage the compliant and flexible behavior of the arm, while remaining maneuverable and dynamic thanks to the omnidirectional drone as the floating base. The unification of the arm with the drone poses challenges in the modeling a…
Soft robotic systems present a variety of new opportunities for solving complex problems. The use of soft robotic grippers, for example, can simplify the complexity in tasks such as the of grasping irregular and delicate objects. Adoption of soft robotics by academia and industry, however, has been slow and this is, in-part, due to the amount of hardware and software that must be developed from scratch for each use of soft system components. In this paper we detail the design, fabrication and validation of an open-source framework that we designed to lower the barrier to entry for integrating …
Compared with traditional rigid-body robots, soft robots not only exhibit unprecedented adaptation and flexibility but also present novel challenges in their modeling and control because of their infinite degrees of freedom. Most of the existing approaches have mainly relied on approximated models so that the well-developed finite-dimensional control theory can be exploited. However, this may bring in modeling uncertainty and performance degradation. Hence, we propose to exploit infinite-dimensional analysis for soft robotic systems. Our control design is based on the increasingly adopted Coss…
The compliance of soft robotic arms renders the development of accurate kinematic & dynamical models especially challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC fails to effectively handle external forces, or even the influence of gravity, since the robot does not deform with a constant curvature under these conditions. In this paper, we establish three-dimensional (3D) modeling of a multi-segment soft robotic arm under the less restrictive assumption that each segment of the arm is deformed on a plane without twisting.…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Beckman Institute for Advanced Science and Technology（贝克曼高等科学技术研究所）；University of Michigan（密歇根大学）
Many soft-body organisms found in nature flourish underwater. Similarly, soft robots are potentially well-suited for underwater environments partly because the problematic effects of gravity, friction, and harmonic oscillations are less severe underwater. However, it remains a challenge to design, fabricate, waterproof, model, and control underwater soft robotic systems. Furthermore, submersible robots usually do not have configurable components because of the need for sealed electronics and mechanical elements. This work presents the development of a modular and submersible soft robotic arm d…
The 'infinite' passive degrees of freedom of soft robotic arms render their control especially challenging. In this paper, we leverage a previously developed model, which drawing equivalence of the soft arm to a series of universal joints, to design two closed-loop controllers: a configuration space controller for trajectory tracking and a task space controller for position control of the end effector. Extensive experiments and simulations on a four-segment soft arm attest to substantial improvement in terms of: a) superior tracking accuracy of the configuration space controller and b) reduced…
This study examines a biology-inspired approach of using reconfigurable articulation to reduce the control requirement for soft robotic arms. We construct a robotic arm by assembling Kresling origami modules that exhibit predictable bistability. Via switching between their two stable states, these origami modules can behave either like a flexible joint with low bending stiffness or like a stiff link with high stiffness, without requiring any continuous power supply. In this way, the robotic arm can exhibit pseudo-linkage kinematics with lower control requirements and improved motion accuracy. …
The purpose of this research is to design, fabricate and test a tendon driven a continuum soft robot with three modular segments, each of which has a tunable stiffness enabled by layer jamming technology. Compared with previous studies, the robotic arm design of this project has a modular structure, which means the length of the robotic arm can be adjusted by addition of extra arm modules/segments to the existing robotic prototype. Furthermore, the new arm prototype supports motion within a 3-dimensional space. To achieve the goals, the design and fabrication for the variable stiffness robotic…
Controlling soft continuum robotic arms is challenging due to their hyper-redundancy and dexterity. In this paper we demonstrate, for the first time, closed-loop control of the configuration space variables of a soft robotic arm, composed of independently controllable segments, using a Cosserat rod model of the robot and the distributed sensing and actuation capabilities of the segments. Our controller solves the inverse dynamic problem by simulating the Cosserat rod model in MATLAB using a computationally efficient numerical solution scheme, and it applies the computed control output to the a…
A Novel Modular Cable-Driven Soft Robotic Arm with Multi-Segment Reconfigurability
Department of Agricultural and Biological Engineering, Mississippi State University（农业与生物工程系，密苏里州立大学）；Department of Electrical and Computer Engineering, Michigan State University（电气与计算机工程系，密苏里州立大学）；Texas A&M AgriLife Research and Extension Center in Dallas, Texas A&M University（德克萨斯A&M大学达拉斯分校农业与生命科学研究中心）
In recent times, developments in field of communication and robotics has progressed with leaps and bounds. In addition, the blend of both disciplines has contributed heavily in making human life easier and better. So in this work while making use of both the aforementioned technologies, a procedure for design and implementation of a mobile operated mechanical arm is proposed, that is, the proposed arm will be operated via a cellular device that connects with the receiver mounted on the robotic arm. Moreover, over the duration of a call, if any key is pressed from the cellular device than an in…
Fast Heuristic Scheduling and Trajectory Planning for Robotic Fruit Harvesters with Multiple Cartesian Arms
Department of Mechanical & Aerospace Engineering University of California, Davis（机械与航空航天工程系加州大学戴维斯分校）；Department of Biological and Agricultural Engineering University of California, Davis（生物与农业工程系加州大学戴维斯分校）
Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arm…
Shanghai Jiao Tong University（上海交通大学）；University of Oxford（牛津大学）；Tencent（腾讯）
Collision avoidance is a widely investigated topic in robotic applications. When applying collision avoidance techniques to a mobile robot, how to deal with the spatial structure of the robot still remains a challenge. In this paper, we design a configuration-aware safe control law by solving a Quadratic Programming (QP) with designed Control Barrier Functions (CBFs) constraints, which can safely navigate a mobile robotic arm to a desired region while avoiding collision with environmental obstacles. The advantage of our approach is that it correctly and in an elegant way incorporates the spati…
Automation applications are pushing the deployment of many high DoF manipulators in warehouse and manufacturing environments. This has motivated many efforts on optimizing manipulation tasks involving a single arm. Coordinating multiple arms for manipulation, however, introduces additional computational challenges arising from the increased DoFs, as well as the combinatorial increase in the available operations that many manipulators can perform, including handoffs between arms. The focus here is on the case of pick-and-place tasks, which require a sequence of handoffs to be executed, so as to…
Institute for Dynamic Systems and Control, ETH Zürich（苏黎世联邦理工学院动态系统与控制研究所）
This paper presents an offset-free model predictive controller for fast and accurate control of a spherical soft robotic arm. In this control scheme, a linear model is combined with an online disturbance estimation technique to systematically compensate model deviations. Dynamic effects such as material relaxation resulting from the use of soft materials can be addressed to achieve offset-free tracking. The tracking error can be reduced by 35% when compared to a standard model predictive controller without a disturbance compensation scheme. The improved tracking performance enables the realiza…
Transferring multiple objects between bins is a common task for many applications. In robotics, a standard approach is to pick up one object and transfer it at a time. However, grasping and picking up multiple objects and transferring them together at once is more efficient. This paper presents a set of novel strategies for efficiently grasping multiple objects in a bin to transfer them to another. The strategies enable a robotic hand to identify an optimal ready hand configuration (pre-grasp) and calculate a flexion synergy based on the desired quantity of objects to be grasped. This paper al…
Jan C. Willems Center for Systems and Control（扬·C·威廉姆斯系统与控制中心）；Engineering and Technology Institute Groningen (ENTEG)（格罗宁根工程技术研究所（ENTEG））；University of Groningen（格罗宁根大学）
In this work, we propose a passivity-based control approach that addresses the trajectory tracking problem for a class of mechanical systems that comprises a broad range of robotic arms. The resulting controllers can be naturally saturated and do not require velocity measurements. Moreover, the proposed methodology does not require the implementation of observers, and the structure of the closed-loop system permits the identification of a Lyapunov function, which eases the convergence analysis. To corroborate the effectiveness of the methodology, we perform experiments with the Philips Experim…
Central South University（中南大学）；Norwegian University of Science & Technology (NTNU)（挪威科技大学）
The use of robotics in food industry is becoming more popular in recent years. The trend seems to continue as long as the robotics technology meets diverse and challenging needs of the food producers. Rapid developments in digital computers and control systems technologies have significant impact in robotics like any other engineering fields. By utilizing new hardware and software tools, design of these complex systems that need strong integration of distinct disciplines is no longer difficult compared to the past. Therefore, the purpose of this paper is to design and implement a micro-control…
Digital Modeling of Massage Techniques and Reproduction by Robotic Arms
Shanghai Jiao Tong University（上海交通大学）；Meta Robotics Institute, Shanghai Jiao Tong University（上海交通大学元机器人研究所）；University of Shanghai for Science and Technology（上海理工大学）；Gongli Hospital Medical Technology（Gongli医院医疗技术学院）
We present joint learning of instance and semantic segmentation for visible and occluded region masks. Sharing the feature extractor with instance occlusion segmentation, we introduce semantic occlusion segmentation into the instance segmentation model. This joint learning fuses the instance- and image-level reasoning of the mask prediction on the different segmentation tasks, which was missing in the previous work of learning instance segmentation only (instance-only). In the experiments, we evaluated the proposed joint learning comparing the instance-only learning on the test dataset. We als…
Northeastern University（东北大学）；University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）
Assistive robot manipulators must be able to autonomously pick and place a wide range of novel objects to be truly useful. However, current assistive robots lack this capability. Additionally, assistive systems need to have an interface that is easy to learn, to use, and to understand. This paper takes a step forward in this direction. We present a robot system comprised of a robotic arm and a mobility scooter that provides both pick-and-drop and pick-and-place functionality for open world environments without modeling the objects or environment. The system uses a laser pointer to directly sel…
This document is an introduction to designing a multiple degree of freedom robotic manipulator. The goal of the robot is to sort an object based upon the objects color. The robot will be a synthesis of several linkages, servo motors, an Arduino system, and an end-effector. The design must be able to autonomously determine what object to move and where to place it. The robot will have a maximum reach of 24 horizontally. The operating conditions include being able to sort several different colored spheres within the area of a 180° up to 16 from the base of the arm. Sorting consists of picking up…
Accenture Labs（埃森哲实验室）；National Research Council Canada（加拿大国家研究委员会）
In this paper, we explore the ability of a robot arm to learn the underlying operation space defined by the positions (x, y, z) that the arm's end-effector can reach, including disturbances, by deploying and thoroughly evaluating a Spiking Neural Network SNN-based adaptive control algorithm. While traditional control algorithms for robotics have limitations in both adapting to new and dynamic environments, we show that the robot arm can learn the operational space and complete tasks faster over time. We also demonstrate that the adaptive robot control algorithm based on SNNs enables a fast res…
Rutgers, the State University of New Jersey（罗格斯新泽西州立大学）
Robotic pick-and-place (PnP) operations on moving conveyors find a wide range of industrial applications. In practice, simple greedy heuristics (e.g., prioritization based on the time to process a single object) are applied that achieve reasonable efficiency. We show analytically that, under a simplified telescoping robot model, these greedy approaches do not ensure time optimality of PnP operations. To address the shortcomings of classical solutions, we develop algorithms that compute optimal object picking sequences for a predetermined finite horizon. Employing dynamic programming techniques…
Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model a…
A Vision-Based Shared-Control Teleoperation Scheme for Controlling the Robotic Arm of a Four-Legged Robot
Department of Mechanical Engineering, University of São Paulo（机械工程系，圣保罗大学）；Instituto Israelita de Ensino e Pesquisa Albert Einstein, Hospital Israelita Albert Einstein（艾利哈德研究所，以色列医院）
Indian Institute of Technology Kanpur（印度理工学院坎普尔分校）；Tata Consultancy Services（塔塔咨询服务公司）
In this paper, we provide details of a robotic system that can automate the task of picking and stowing objects from and to a rack in an e-commerce fulfillment warehouse. The system primarily comprises of four main modules: (1) Perception module responsible for recognizing query objects and localizing them in the 3-dimensional robot workspace; (2) Planning module generates necessary paths that the robot end- effector has to take for reaching the objects in the rack or in the tote; (3) Calibration module that defines the physical workspace for the robot visible through the on-board vision syste…
A large number of robotic and human-assisted missions to the Moon and Mars are forecast. NASA's efforts to learn about the geology and makeup of these celestial bodies rely heavily on the use of robotic arms. The safety and redundancy aspects will be crucial when humans will be working alongside the robotic explorers. Additionally, robotic arms are crucial to satellite servicing and planned orbit debris mitigation missions. The goal of this work is to create a custom Computer Vision (CV) based Artificial Neural Network (ANN) that would be able to rapidly identify the posture of a 7 Degree of F…
Khoury College of Computer Sciences, Northeastern University（东北大学科里计算机科学学院）
We consider robotic pick-and-place of partially visible, novel objects, where goal placements are non-trivial, e.g., tightly packed into a bin. One approach is (a) use object instance segmentation and shape completion to model the objects and (b) use a regrasp planner to decide grasps and places displacing the models to their goals. However, it is critical for the planner to account for uncertainty in the perceived models, as object geometries in unobserved areas are just guesses. We account for perceptual uncertainty by incorporating it into the regrasp planner's cost function. We compare sev…
In the scope of the "Chess-Bot" project, this study's goal is to choose the right model for the robotic arm that the "the Chess-Bot" will use to move the pawn from a cell to another. In this paper, there is the definition and the structure of a robot arm. Also, the different engineering and kinematics fundamentals of the robot and its components will be detailed. Furthermore, the different structures of robotic arms will be presented and compared based on different criteria. Finally, a model for "the Chess-Bot" arm will be synthesized based on accurate algorithms and equations.
Inverse Kinematics (IK) solves the problem of mapping from the Cartesian space to the joint configuration space of a robotic arm. It has a wide range of applications in areas such as computer graphics, protein structure prediction, and robotics. With the vast advances of artificial neural networks (NNs), many researchers recently turned to data-driven approaches to solving the IK problem. Unfortunately, NNs become inadequate for robotic arms with redundant Degrees-of-Freedom (DoFs). This is because such arms may have multiple angle solutions to reach the same desired pose, while typical NNs on…
In this paper, we propose a novel edge and corner detection algorithm for an unorganized point cloud. Our edge detection method classifies a query point as an edge point by evaluating the distribution of local neighboring points around the query point. The proposed technique has been tested on generic items such as dragons, bunnies, and coffee cups from the Stanford 3D scanning repository. The proposed technique can be directly applied to real and unprocessed point cloud data of random clutter of objects. To demonstrate the proposed technique's efficacy, we compare it to the other solutions fo…
