Detecting and localizing contacts is essential for robot manipulators to perform contact-rich tasks in unstructured environments. While robot skins can localize contacts on the surface of robot arms, these sensors are not yet robust or easily accessible. As such, prior works have explored using proprioceptive observations, such as joint velocities and torques, to perform contact localization. Many past approaches assume the robot is static during contact incident, a single contact is made at a time, or having access to accurate dynamics models and joint torque sensing. In this work, we relax these assumptions and propose using Domain Randomization to train a neural network to localize contacts of robot arms in motion without joint torque observations. Our method uses a novel cylindrical projection encoding of the robot arm surface, which allows the network to use convolution layers to process input features and transposed convolution layers to predict contacts. The trained network achieves a contact detection accuracy of 91.5% and a mean contact localization error of 3.0cm. We further demonstrate an application of the contact localization model in an obstacle mapping task, evaluated in both simulation and the real world.
Beihang University（北京航空航天大学）；Beijing Institute of Aeronautical Materials（北京航空材料研究院）
Object dropping may occur when the robotic arm grasps objects with uneven mass distribution due to additional moments generated by objects' gravity. To solve this problem, we present a novel work that does not require extra wrist and tactile sensors and large amounts of experiments for learning. First, we obtain the center-of-mass position of the rod object using the widely fixed joint torque sensors on the robot arm and RGBD camera. Further, we give the strategy of grasping to improve grasp stability. Simulation experiments are performed in "Mujoco". Results demonstrate that our work is effec…
Functional kinematic and kinetic requirements of the upper limb during activities of daily living: a recommendation on necessary joint capabilities for prosthetic arms
Chair of Robotics and Systems Intelligence, MIRMI - Munich Institute of Robotics and Machine Intelligence, Technical University of Munich (TUM)（慕尼黑工业大学机器人与系统智能主席席位、慕尼黑机器人与机器智能研究所（MIRMI））
Lifting objects, whose mass may produce high wrist torques that exceed the hardware strength limits, could lead to unstable grasps or serious robot damage. This work introduces a new Center-of-Mass (CoM)-based grasp pose adaptation method, for picking up objects using a combination of exteroceptive 3D perception and proprioceptive force/torque sensor feedback. The method works in two iterative stages to provide reliable and wrist torque efficient grasps. Initially, a geometric object CoM is estimated from the input range data. In the first stage, a set of hand-size handle grasps are localized …
LETS: Low-Overhead External-Torque Sensing for Force-Feedback Bilateral Teleoperation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Beijing Amigos Robot Technology Co., Ltd.（北京阿米格斯机器人科技有限公司）
University of Colorado, Boulder（科罗拉多大学博尔德分校）；Robotic Materials Inc.（机器人材料公司）
We present algorithms and results for a robotic manipulation system that was designed to be easily programmable and adaptable to various tasks common to industrial setting, which is inspired by the Industrial Assembly Challenge at the 2018 World Robotics Summit in Tokyo. This challenge included assembly of standard, commercially available industrial parts into 2D and 3D assemblies. We demonstrate three tasks that can be classified into "peg-in-hole" and "hole-on-peg" tasks and identify two canonical algorithms: spiral-based search and tilting insertion. Both algorithms use hand-coded threshold…
This paper proposes a task-space control protocol for the collaborative manipulation of a single object by N robotic agents. The proposed methodology is decentralized in the sense that each agent utilizes information associated with its own and the object's dynamic/kinematic parameters and no on-line communication takes place. Moreover, no feedback of the contact forces/torques is required, therefore employment of corresponding sensors is avoided. An adaptive version of the control scheme is also introduced, where the agents' and object's dynamic parameters are considered unknown. We also use …
The manipulation of large objects and safe operation in the vicinity of humans are key capabilities of a general purpose domestic robotic assistant. We present the design of a soft, tactile-sensing humanoid upper-body robot and demonstrate whole-body rich-contact manipulation strategies for handling large objects. We demonstrate our hardware design philosophy for outfitting off-the-shelf hard robot arms and other components with soft tactile-sensing modules, including: (i) low-cost, cut-resistant, contact pressure localizing coverings for the arms, (ii) paws based on TRI's Soft-bubble sensors …
Plant-inspired behavior-based controller to enable reaching in redundant continuum robot arms
The BioRobotics Institute, Sant’Anna School of Advanced Studies（圣安娜高等研究院生物机器人研究所）；Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）
This paper addresses the localization of contacts of an unknown grasped rigid object with its environment, i.e., extrinsic to the robot. We explore the key role that distributed tactile sensing plays in localizing contacts external to the robot, in contrast to the role that aggregated force/torque measurements play in localizing contacts on the robot. When in contact with the environment, an object will move in accordance with the kinematic and possibly frictional constraints imposed by that contact. Small motions of the object, which are observable with tactile sensors, indirectly encode thos…
Air-Chamber Based Soft Six-Axis Force/Torque Sensor for Human-Robot Interaction
Key Laboratory of the Ministry of Education for Image Processing and Intelligent Control（教育部图像处理与智能控制重点实验室）；School of Artificial Intelligence and Automation（人工智能与自动化学院）；Huazhong University of Science and Technology（华中科技大学）
For certain manipulation tasks, object pose estimation from head-mounted cameras may not be sufficiently accurate. This is at least in part due to our inability to perfectly calibrate the coordinate frames of today's high degree of freedom robot arms that link the head to the end-effectors. We present a novel framework combining pre-touch sensing and deep learning to more accurately estimate pose in an efficient manner. The use of pre-touch sensing allows our method to localize the object directly with respect to the robot's end effector, thereby avoiding error caused by miscalibration of the …
Department of Mechanical and Industrial Engineering, Northeastern University（机械与工业工程系，东北大学）；Department of Computer Science, Northeastern University（计算机科学系，东北大学）
Hong Kong University of Science and Technology（香港科技大学）
Retrieving rich contact information from robotic tactile sensing has been a challenging, yet significant task for the effective perception of object properties that the robot interacts with. This work is dedicated to developing an algorithm to estimate contact force and torque for vision-based tactile sensors. We first introduce the observation of the contact deformation patterns of hyperelastic materials under ideal single-axial loads in simulation. Then based on the observation, we propose a method of estimating surface forces and torque from the contact deformation vector field with the Hel…
Hanson Robotics Ltd（汉森机器人有限公司）；Indian Institute of Science（印度科学学院）；ARTPARK（ARTPARK（人工智能与机器人技术园区））
Open Arms is a novel open-source platform of realistic human-like robotic hands and arms hardware with 28 Degree-of-Freedom (DoF), designed to extend the capabilities and accessibility of humanoid robotic grasping and manipulation. The Open Arms framework includes an open SDK and development environment, simulation tools, and application development tools to build and operate Open Arms. This paper describes these hands controls, sensing, mechanisms, aesthetic design, and manufacturing and their real-world applications with a teleoperated nursing robot. From 2015 to 2022, the authors have desig…
Learning When to See and When to Feel: Adaptive Vision-Torque Fusion for Contact-Aware Manipulation
Edwardson School of Industrial Engineering, Purdue University（普渡大学爱德华森工业工程学院）；Zachry Department of Civil and Environmental Engineering, Texas A&M University（德州农工大学扎克里土木与环境工程系）
This paper presents an approach for learning online generation of collision-free and torque-limited robot trajectories. In order to generate future motions, a neural network is periodically invoked. Based on the current kinematic state of the robot and the network output, a trajectory for the current time interval can be calculated. The main idea of our paper is to execute the computed motion only if a collision-free and torque-limited way to continue the trajectory is known. In practice, the motion computed for the current time interval is extended by a braking trajectory and simulated using …
Fine Robotic Manipulation without Force/Torque Sensor
School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与宇航工程学院）；Eureka Robotics（尤里卡机器人公司）；Singapore Centre for 3D Printing (SC3DP)（新加坡3D打印中心）
PRISM: Precision and contact-rich Real-world Industrial Skill dataset with Multimodal sensing
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院通用人工智能国家重点实验室）；Delta Intelligence（三角洲智能公司）；PKU-Wuhan Institute for Artificial Intelligence（北京大学武汉人工智能研究院）；Hubei Humanoid Robot Innovation Center Co., Ltd.（湖北人形机器人创新中心有限公司）；China Academy of Information and Communications Technology（中国信息通信研究院）
Dept. of Mechanical Engineering（机械工程系）；Dept. of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）
Teleoperation platforms often require the user to be situated at a fixed location to both visualize and control the movement of the robot and thus do not provide the operator with much mobility. One example is in existing robotic surgery solutions that require the surgeons to be away from the patient, attached to consoles where their heads must be fixed and their arms can only move in a limited space. This creates a barrier between physicians and patients that does not exist in normal surgery. To address this issue, we propose a mobile telesurgery solution where the surgeons are no longer mech…
Force Myography based Torque Estimation in Human Knee and Ankle Joints
High Performance Humanoid Technologies Lab, Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT), Germany（高性能人形机器人实验室，人机学与机器人研究所，卡尔斯鲁厄理工学院（KIT））；BioMotion Center, Institute of Sports and Sports Sciences, Karlsruhe Institute of Technology (KIT), Germany（生物运动中心，体育与体育科学研究所，卡尔斯鲁厄理工学院（KIT））
With worldwide implementation, millions of surgeries are assisted by surgical robots. The cable-drive mechanism on many surgical robots allows flexible, light, and compact arms and tools. However, the slack and stretch of the cables and the backlash of the gears introduce inevitable errors from motor poses to joint poses, and thus forwarded to the pose and orientation of the end-effector. In this paper, a learning-based calibration using a deep neural network is proposed, which reduces the unloaded pose RMSE of joints 1, 2, 3 to 0.3003 deg, 0.2888 deg, 0.1565 mm, and loaded pose RMSE of joints…
