Catch the Ball: Accurate High-Speed Motions for Mobile Manipulators via Inverse Dynamics Learning
University of Toronto Institute for Aerospace Studies (UTIAS)（多伦多大学航空航天研究所（UTIAS））；Vector Institute for artificial Intelligence（人工智能向量研究所）；University of Toronto（多伦多大学）
Ke Dong, Karime Pereida, Florian Shkurti, Angela P. Schoellig
Mobile manipulators consist of a mobile platform equipped with one or more robot arms and are of interest for a wide array of challenging tasks because of their extended workspace and dexterity. Typically, mobile manipulators are deployed in slow-motion collaborative robot scenarios. In this paper, we consider scenarios where accurate high-speed motions are required. We introduce a framework for this regime of tasks including two main components: (i) a bi-level motion optimization algorithm for real-time trajectory generation, which relies on Sequential Quadratic Programming (SQP) and Quadratic Programming (QP), respectively; and (ii) a learning-based controller optimized for precise tracking of high-speed motions via a learned inverse dynamics model. We evaluate our framework with a mobile manipulator platform through numerous high-speed ball catching experiments, where we show a success rate of 85.33%. To the best of our knowledge, this success rate exceeds the reported performance of existing related systems and sets a new state of the art.
Catch, Throw, Repeat: Planning for Human-Robot Partner Juggling
Technical University of Darmstadt（达姆施塔特工业大学）；Justus-Liebig University Gießen（吉森尤斯 - 李比希大学）；German Research Center for AI (DFKI)（德国人工智能研究中心（DFKI））；Robotics Institute Germany（德国机器人研究所）；Indian Institute of Technology Delhi - Abu Dhabi（印度理工学院德里分校 - 阿布扎比）
CushionCatch: A Compliant Catching Mechanism for Mobile Manipulators via Combined Optimization and Learning
Center for Artificial Intelligence and Robotics, Shenzhen International Graduate School, Tsinghua University（人工智能与机器人中心，深圳国际研究生院，清华大学）；School of Advanced Manufacturing, Sun Yat-Sen University（先进制造学院，中山大学）；School of Electrical and Automation Engineering, Hefei University of Technology（电气与自动化工程学院，合肥工业大学）；Navigation and Control Research Center, Department of Automation, Tsinghua University（导航与控制研究中心，自动化系，清华大学）
Playing the cup-and-ball game is an intriguing task for robotics research since it abstracts important problem characteristics including system nonlinearity, contact forces and precise positioning as terminal goal. In this paper, we present a learning model based control strategy for the cup-and-ball game, where a Universal Robots UR5e manipulator arm learns to catch a ball in one of the cups on a Kendama. Our control problem is divided into two sub-tasks, namely $(i)$ swinging the ball up in a constrained motion, and $(ii)$ catching the free-falling ball. The swing-up trajectory is computed o…
Impedance and Stability Targeted Adaptation for Aerial Manipulator with Unknown Coupling Dynamics
Robotics Research Center, International Institute of Information Technology Hyderabad (IIIT-H)（印度海得拉巴国际信息技术学院机器人研究中心）；The University of Manchester（曼彻斯特大学）；Southeast University（东南大学）
Catch It! Learning to Catch in Flight with Mobile Dexterous Hands
Shanghai Qi Zhi Institute（上海期智研究院）；Tsinghua University（清华大学）；Shanghai AI Lab（上海人工智能实验室）；Georgia Institute of Technology（佐治亚理工学院）；Stanford University（斯坦福大学）
Optimal Control Approach for Non-prehensile Ball Juggling Using a 7-DoF Manipulator
Technical University of Munich（慕尼黑技术大学）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；German Aerospace Center (DLR), Institute of Robotics and Mechatronics（德国航空航天中心（DLR）机器人与机电机构研究所）
TU Wien（维也纳工业大学）；University of Liverpool（利物浦大学）；Austrian Institute of Technology GmbH (AIT)（奥地利技术研究院有限公司）
Solving the analytical inverse kinematics (IK) of redundant manipulators in real time is a difficult problem in robotics since its solution for a given target pose is not unique. Moreover, choosing the optimal IK solution with respect to application-specific demands helps to improve the robustness and to increase the success rate when driving the manipulator from its current configuration towards a desired pose. This is necessary, especially in high-dynamic tasks like catching objects in mid-flights. To compute a suitable target configuration in the joint space for a given target pose in the t…
Autonomous mobile manipulation offers a dual advantage of mobility provided by a mobile platform and dexterity afforded by the manipulator. In this paper, we present a whole-body optimal control framework to jointly solve the problems of manipulation, balancing and interaction as one optimization problem for an inherently unstable robot. The optimization is performed using a Model Predictive Control (MPC) approach; the optimal control problem is transcribed at the end-effector space, treating the position and orientation tasks in the MPC planner, and skillfully planning for end-effector contac…
An internal model of the own body can be assumed a fundamental and evolutionary-early representation as it is present throughout the animal kingdom. Such functional models are, on the one hand, required in motor control, for example solving the inverse kinematic or dynamic task in goal-directed movements or a forward task in ballistic movements. On the other hand, such models are recruited in cognitive tasks as are planning ahead or observation of actions of a conspecific. Here, we present a functional internal body model that is based on the Mean of Multiple Computations principle. For the fi…
Keep it Upright: Model Predictive Control for Nonprehensile Object Transportation with Obstacle Avoidance on a Mobile Manipulator
Technical University of Munich（慕尼黑工业大学）；University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）；University of Toronto Robotics Institute（多伦多大学机器人研究所）；Munich Institute of Robotics and Machine Intelligence（慕尼黑机器人与机器智能研究所）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
To reliably model real robot characteristics, interval linear systems of equations allow to describe families of problems that consider sets of values. This allows to easily account for typical complexities such as sets of joint states and design parameter uncertainties. Inner approximations of the solutions to the interval linear systems can be used to describe the common capabilities of a robotic manipulator corresponding to the considered sets of values. In this work, several classes of problems are considered. For each class, reliable and efficient polytope, n-cube, and n-ball inner approx…
Athletics are a quintessential and universal expression of humanity. From French monks who in the 12th century invented jeu de paume, the precursor to modern lawn tennis, back to the K'iche' people who played the Maya Ballgame as a form of religious expression over three thousand years ago, humans have sought to train their minds and bodies to excel in sporting contests. Advances in robotics are opening up the possibility of robots in sports. Yet, key challenges remain, as most prior works in robotics for sports are limited to pristine sensing environments, do not require significant force gen…
Robots that can learn in the physical world will be important to en-able robots to escape their stiff and pre-programmed movements. For dynamic high-acceleration tasks, such as juggling, learning in the real-world is particularly challenging as one must push the limits of the robot and its actuation without harming the system, amplifying the necessity of sample efficiency and safety for robot learning algorithms. In contrast to prior work which mainly focuses on the learning algorithm, we propose a learning system, that directly incorporates these requirements in the design of the policy repre…
A limitation of model-based reinforcement learning (MBRL) is the exploitation of errors in the learned models. Black-box models can fit complex dynamics with high fidelity, but their behavior is undefined outside of the data distribution.Physics-based models are better at extrapolating, due to the general validity of their informed structure, but underfit in the real world due to the presence of unmodeled phenomena. In this work, we demonstrate experimentally that for the offline model-based reinforcement learning setting, physics-based models can be beneficial compared to high-capacity functi…
Arm-Constrained Curriculum Learning for Loco-Manipulation of the Wheel-Legged Robot
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Tsinghua University（清华大学）；Harbin Institute of Technology（哈尔滨工业大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）
Modern, torque-controlled service robots can regulate contact forces when interacting with their environment. Model Predictive Control (MPC) is a powerful method to solve the underlying control problem, allowing to plan for whole-body motions while including different constraints imposed by the robot dynamics or its environment. However, an accurate model of the robot-environment is needed to achieve a satisfying closed-loop performance. Currently, this necessity undermines the performance and generality of MPC in manipulation tasks. In this work, we combine an MPC-based whole-body controller …
Paul G. Allen School of Computer Science and Engineering（保罗·G·艾伦计算机科学与工程学院）
Although general purpose robotic manipulators are becoming more capable at manipulating various objects, their ability to manipulate millimeter-scale objects are usually very limited. On the other hand, ultrasonic levitation devices have been shown to levitate a large range of small objects, from polystyrene balls to living organisms. By controlling the acoustic force fields, ultrasonic levitation devices can compensate for robot manipulator positioning uncertainty and control the grasping force exerted on the target object. The material agnostic nature of acoustic levitation devices and their…
Mobile manipulation in robotics is challenging due to the need of solving many diverse tasks, such as opening a door or picking-and-placing an object. Typically, a basic first-principles system description of the robot is available, thus motivating the use of model-based controllers. However, the robot dynamics and its interaction with an object are affected by uncertainty, limiting the controller's performance. To tackle this problem, we propose a Bayesian multi-task learning model that uses trigonometric basis functions to identify the error in the dynamics. In this way, data from different …
