Having a precise and robust transformation between the robot coordinate system and the AR-device coordinate system is paramount during human-robot interaction (HRI) based on augmented reality using Head mounted displays (HMD), both for intuitive information display and for the tracking of human motions. Most current solutions in this area rely either on the tracking of visual markers, e.g. QR codes, or on manual referencing, both of which provide unsatisfying results. Meanwhile a plethora of object detection and referencing methods exist in the wider robotic and machine vision communities. The precision of the referencing is likewise almost never measured. Here we would like to address this issue by firstly presenting an overview of currently used referencing methods between robots and HMDs. This is followed by a brief overview of object detection and referencing methods used in the field of robotics. Based on these methods we suggest three classes of referencing algorithms we intend to pursue - semi-automatic, on-shot; automatic, one-shot; and automatic continuous. We describe the general workflows of these three classes as well as describing our proposed algorithms in each of these classes. Finally we present the first experimental results of a semi-automatic referencing algorithm, tested on an industrial KUKA KR-5 manipulator.
Kinetostatics and Particle-Swarm Optimization of Vehicle-Mounted Underactuated Metamorphic Loading Manipulators
Institute for Robotics Research（机器人研究所）；Centre for Robotics Research（机器人研究中心）；Singapore Institute of Manufacturing Technology, Agency for Science, Technology and Research (A*STAR)（新加坡制造技术研究所，科技研究局（A*STAR））
A Task-Driven, Planner-in-the-Loop Computational Design Framework for Modular Manipulators
Humanoids and Human Centered Mechatronics Research Line, Italian Institute of Technology (IIT) Genoa, Italy（意大利理工学院（IIT）热那亚人类与以人为中心的机电研究组）；Department of Industrial Engineering, University of Trento, Trento, Italy（特伦托大学工业工程系）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用技术大学）
The field of Augmented Reality (AR) based Human Robot Interaction (HRI) has progressed significantly since its inception more than two decades ago. With more advanced devices, particularly head-mounted displays (HMD), freely available programming environments and better connectivity, the possible application space expanded significantly. Here we present concepts and systems currently being developed at our lab to enable a truly end-to-end application of AR in HRI, from setting up the working environment of the robot, through programming and finally interaction with the programmed robot. Releva…
Robotic systems performing end-user oriented autonomous exploration can be deployed in different scenarios which not only require mapping but also simultaneous inspection of regions of interest for the end-user. In this work, we propose a novel Next-Best-View (NBV) planner which can perform full exploration and user-oriented exploration with inspection of the regions of interest using a mobile manipulator robot. We address the exploration-inspection problem as an instance of Multi-Objective Optimization (MOO) and propose a weighted-sum-based information gain function for computing NBVs for the…
NVIDIA（英伟达）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera (without depth) that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF. We show results on a real dataset collect…
In this paper, the problem of making a safe compliant contact between a human and an assistive robot is considered. Users with disabilities have a need to utilize their assistive robots for physical human-robot interaction (PHRI) during certain activities of daily living (ADLs). Specifically, we propose a hybrid force/velocity/attitude control for a PHRI system based on measurements from a 6-axis force/torque sensor mounted on the robot wrist. While automatically aligning the end-effector surface with the unknown environmental (human) surface, a desired commanded force is applied in the normal…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）
Hand guidance emerged from the safety requirements for collaborative robots, namely possessing joint-torque sensors. Since then it has proven to be a powerful tool for easy trajectory programming, allowing lay-users to reprogram robots intuitively. Going beyond, a robot can learn tasks by user demonstrations through kinesthetic teaching, enabling robots to generalise tasks and further reducing the need for reprogramming. However, hand guidance is still mostly relegated to collaborative robots. Here we propose a method that doesn't require any sensors on the robot or in the robot cell, by using…
In this paper, a self-calibration approach for eye-in-hand robots using SLAM is considered. The goal is to calibrate the positioning of a robotic arm, with a camera mounted on the end-effector automatically using a SLAM-based method like Extended Kalman Filter (EKF). Given the camera intrinsic parameters and a set of feature markers in a work-space, the camera extrinsic parameters are approximated. An EKF based measurement model is deployed to effectively localize the camera and compute the camera to end-effector transformation. The proposed approach is tested on a UR5 manipulator with a depth…
Motion control of mobile manipulators (a robotic arm mounted on a mobile base) can be challenging for complex tasks such as material and package handling. In this paper, a task-space stabilization controller based on Nonlinear Model Predictive Control (NMPC) is designed and implemented to a 10 Degrees of Freedom (DOF) mobile manipulator which consists of a 7-DOF robotic arm and a 3-DOF mobile base. The system model is based on kinematic models where the end-effector orientation is parameterized directly by a rotation matrix. The state and control constraints as well as singularity constraints …
Quadruped robots are usually equipped with additional arms for manipulation, negatively impacting price and weight. On the other hand, the requirements of legged locomotion mean that the legs of such robots often possess the needed torque and precision to perform manipulation. In this paper, we present a novel design for a small-scale quadruped robot equipped with two leg-mounted manipulators inspired by crustacean chelipeds and knuckle-walker forelimbs. By making use of the actuators already present in the legs, we can achieve manipulation using only 3 additional motors per limb. The design e…
Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院控制论系）；Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague（布拉格捷克理工大学捷克信息学、机器人学与控制论研究所）
We present a robot kinematic calibration method that combines complementary calibration approaches: self-contact, planar constraints, and self-observation. We analyze the estimation of the end effector parameters, joint offsets of the manipulators, and calibration of the complete kinematic chain (DH parameters). The results are compared with ground truth measurements provided by a laser tracker. Our main findings are: (1) When applying the complementary calibration approaches in isolation, the self-contact approach yields the best and most stable results. (2) All combinations of more than one …
We present and analyze methods for the kinematic and kinetostatic calibration of, typically, wrist mounted force/torque sensors in robotics. The algorithms are based on matrix factorization and require no special equipment. The only requirement is the ability to reorient the sensor and to measure its orientation in a fixed coordinate system, such as through the forward kinematics of a robot manipulator, or using an external tracking system. We present methods to find the rotation matrix between the coordinate system of the sensor and that of the tool flange, the mass held by the force sensor a…
AnyUser: Translating Sketched User Intent into Domestic Robots
College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University（湖南大学国家机器人视觉感知与控制技术工程研究中心）
We present a cost-efficient and versatile method to map an unknown 3D freeform surface using only sparse measurements while the end-effector of a robotic manipulator moves along the surface. The geometry is locally approximated by a plane, which is defined by measured points on the surface. The method relies on linear Kalman filters, estimating the height of each point on a 2D grid. Therefore, the approximation covariance for each grid point is determined using a radial basis function to consider the measured point positions. We propose different update strategies for the grid points exploitin…
With the potential applications of capsule robots in medical endoscopy, accurate dynamic control of the capsule robot is becoming more and more important. In the scale of a capsule robot, the friction between capsule and the environment plays an essential role in the dynamic model, which is usually difficult to model beforehand. In the paper, a tethered capsule robot system driven by a robot manipulator is built, where a strong magnetic Halbach array is mounted on the robot's end-effector to adjust the state of the capsule. To increase the control accuracy, the friction between capsule and the…
We are motivated by the problem of performing failure prediction for safety-critical robotic systems with high-dimensional sensor observations (e.g., vision). Given access to a black-box control policy (e.g., in the form of a neural network) and a dataset of training environments, we present an approach for synthesizing a failure predictor with guaranteed bounds on false-positive and false-negative errors. In order to achieve this, we utilize techniques from Probably Approximately Correct (PAC)-Bayes generalization theory. In addition, we present novel class-conditional bounds that allow us to…
A novel autonomous microplastics surveying robot for beach environments
Oden Institute for Computational Engineering & Sciences（奥登计算工程与科学研究所）；Department of Civil, Architecture and Civil Engineering（土木工程与建筑系）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Department of Civil, Architecture and Environmental Engineering（土木工程与环境工程系）；Department of Chemistry（化学系）
Tactile-Based Human Intent Recognition for Robot Assistive Navigation
Electrical & Computer Engineering, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校电气与计算机工程系）；Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院）
Johns Hopkins University（约翰斯·霍普金斯大学）；Department of Computer Science, Johns Hopkins University（约翰斯·霍普金斯大学计算机科学系）；Laboratory for Computer Aided Medical Procedures, Johns Hopkins University（约翰斯·霍普金斯大学计算机辅助医疗程序实验室）
Robot-assisted minimally invasive surgery has shown to improve patient outcomes, as well as reduce complications and recovery time for several clinical applications. While increasingly configurable robotic arms can maximize reach and avoid collisions in cluttered environments, positioning them appropriately during surgery is complicated because safety regulations prevent automatic driving. We propose a head-mounted display (HMD) based augmented reality (AR) system designed to guide optimal surgical arm set up. The staff equipped with HMD aligns the robot with its planned virtual counterpart. I…
Mitsubishi Electric Research Laboratories (MERL)（三菱电机研究实验室）
We present the design of a learning-based compliance controller for assembly operations for industrial robots. We propose a solution within the general setting of learning from demonstration (LfD), where a nominal trajectory is provided through demonstration by an expert teacher. This can be used to learn a suitable representation of the skill that can be generalized to novel positions of one of the parts involved in the assembly, for example the hole in a peg-in-hole (PiH) insertion task. Under the expectation that this novel position might not be entirely accurately estimated by a vision or …
