Robotic manipulators are widely used in various industries for complex and repetitive tasks. However, they remain vulnerable to unexpected hardware failures. In this study, we address the challenge of enabling a robotic manipulator to complete tasks despite joint malfunctions. Specifically, we develop a reinforcement learning (RL) framework to adaptively compensate for a non-functional joint during task execution. Our experimental platform is the Franka robot with 7 degrees of freedom (DOFs). We formulate the problem as a partially observable Markov decision process (POMDP), where the robot is trained under various joint failure conditions and tested in both seen and unseen scenarios. We consider scenarios where a joint is permanently broken and where it functions intermittently. Additionally, we demonstrate the effectiveness of our approach by comparing it with traditional inverse kinematics-based control methods. The results show that the RL algorithm enables the robot to successfully complete tasks even with joint failures, achieving a high success rate with an average rate of 93.6%. This showcases its robustness and adaptability. Our findings highlight the potential of RL to enhance the resilience and reliability of robotic systems, making them better suited for unpredictable environments. All related codes and models are published online.
Harbin Institute of Technology（哈尔滨工业大学）；Beihang University（北京航空航天大学）；The Hong Kong Polytechnic University（香港理工大学）
The robotic manipulation of compliant objects is currently one of the most active problems in robotics due to its potential to automate many important applications. Despite the progress achieved by the robotics community in recent years, the 3D shaping of these types of materials remains an open research problem. In this paper, we propose a new vision-based controller to automatically regulate the shape of compliant objects with robotic arms. Our method uses an efficient online surface/curve fitting algorithm that quantifies the object's geometry with a compact vector of features; This feedbac…
Ecole des Mines de Nantes（南特高等矿业学院）；Institut de Recherches en Communications et Cybernétique de Nantes（南特通信与控制研究所）；Université de Nantes（南特大学）
The paper deals with the problem of compliance errors compensation in robotic-based milling. Contrary to previous works that assume that the forces/torques generated by the manufacturing process are constant, the interaction between the milling tool and the workpiece is modeled in details. It takes into account the tool geometry, the number of teeth, the feed rate, the spindle rotation speed and the properties of the material to be processed. Due to high level of the disturbing forces/torques, the developed compensation technique is based on the non-linear stiffness model that allows us to mod…
Massachusetts Institute of Technology（麻省理工学院）；Woods Hole Oceanographic Institution（伍兹霍尔海洋研究所）
We present PLUMES, a planner to localizing and collecting samples at the global maximum of an a priori unknown and partially observable continuous environment. The "maximum-seek-and-sample" (MSS) problem is pervasive in the environmental and earth sciences. Experts want to collect scientifically valuable samples at an environmental maximum (e.g., an oil-spill source), but do not have prior knowledge about the phenomenon's distribution. We formulate the MSS problem as a partially-observable Markov decision process (POMDP) with continuous state and observation spaces, and a sparse reward signal.…
Institute for Dynamic Systems and Control, ETH Zürich（苏黎世联邦理工学院动态系统与控制研究所）
This paper presents an offset-free model predictive controller for fast and accurate control of a spherical soft robotic arm. In this control scheme, a linear model is combined with an online disturbance estimation technique to systematically compensate model deviations. Dynamic effects such as material relaxation resulting from the use of soft materials can be addressed to achieve offset-free tracking. The tracking error can be reduced by 35% when compared to a standard model predictive controller without a disturbance compensation scheme. The improved tracking performance enables the realiza…
Universität Hamburg（汉堡大学）；National University of Singapore（新加坡国立大学）；Aalto University（阿尔托大学）；Technische Universität Darmstadt（达姆施塔特工业大学）
Noisy sensing, imperfect control, and environment changes are defining characteristics of many real-world robot tasks. The partially observable Markov decision process (POMDP) provides a principled mathematical framework for modeling and solving robot decision and control tasks under uncertainty. Over the last decade, it has seen many successful applications, spanning localization and navigation, search and tracking, autonomous driving, multi-robot systems, manipulation, and human-robot interaction. This survey aims to bridge the gap between the development of POMDP models and algorithms at on…
Korea Advanced Institute of Science and Technology（韩国科学技术院）；Siemens Healthineers（西门子医疗）
Tendon-sheath-driven manipulators (TSM) are widely used in minimally invasive surgical systems due to their long, thin shape, flexibility, and compliance making them easily steerable in narrow or tortuous environments. Many commercial TSM-based medical devices have non-linear phenomena resulting from their composition such as backlash hysteresis and dead zone, which lead to a considerable challenge for achieving precise control of the end effector pose. However, many recent works in the literature do not consider the combined effects and compensation of these phenomena, and less focus on pract…
Enhancing Robotic Precision in Construction: A Modular Factor Graph-Based Framework to Deflection and Backlash Compensation Using High-Accuracy Accelerometers
In this paper, we present a novel flow model and compensation strategy for high-viscosity fluid deposition that yields high quality parts in the face of large transient delays and nonlinearity. Robotic high-viscosity fluid deposition is an essential process for a broad range of manufacturing applications including additive manufacturing, adhesive and sealant dispensing, and soft robotics. However, high-viscosity fluid deposition without compensation can lead to poor part quality and defects due to large transient delays and complex fluid dynamics. Our computationally efficient model is well-su…
Chinese University of Hong Kong（香港中文大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
External disturbance forces caused by nonlinear springy electrical cables in the Master Tool Manipulator (MTM) of the da Vinci Research Kit (dVRK) limits the usage of the existing gravity compensation methods. Significant motion drifts at the MTM tip are often observed when the MTM is located far from its identification trajectory, preventing the usage of these methods for the entire workspace reliably. In this paper, we propose a general and systematic framework to address the problems of the gravity compensation for the MTM of the dVRK. Particularly, high order polynomial models were used to…
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Hamburg（汉堡大学）
Drilling is one of the hardest parts of pedicle screw fixation, and it is one of the most dangerous operations because inaccurate screw placement would injury vital tissues, particularly when the vertebra is not stationary. Here we demonstrate the drilling state recognition method for moving tissue by compensating the displacement based on a simplified motion predication model of a vertebra with respect to the tidal volume. To adapt it to different patients, the prediction model was built based on the physiological data recorded from subjects themselves. In addition, the spindle speed of the d…
Towards Motion Compensation in Autonomous Robotic Subretinal Injections
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Hospital（约翰斯·霍普金斯医院）；Klinikum rechts der Isar der Technische Universität München（慕尼黑工业大学右岸医院）
In this work, we propose a trajectory generation method for robotic systems with contact force constraint based on optimal control and reachability analysis. Normally, the dynamics and constraints of the contact-constrained robot are nonlinear and coupled to each other. Instead of linearizing the model and constraints, we directly solve the optimal control problem to obtain the feasible state trajectory and the control input of the system. A tractable optimal control problem is formulated which is addressed by dual approaches, which are sampling-based dynamic programming and rigorous reachabil…
This study proposes an integrated task and motion planning method for dynamic locomotion in partially observable environments with multi-level safety guarantees. This layered planning framework is composed of a high-level symbolic task planner and a low-level phase-space motion planner. A belief abstraction at the task planning level enables belief estimation of dynamic obstacle locations and guarantees navigation safety with collision avoidance. The high-level task planner, i.e., a two-level navigation planner, employs linear temporal logic for a reactive game synthesis between the robot and …
Reinforcement learning with timed constraints for robotics motion planning
Department of Mechanical Engineering, Iowa Technology Institute, University of Iowa（机械工程系，Iowa技术研究所，爱荷华大学）；Talus Renewables, Inc.（Talus Renewables公司）；Department of Industrial, Electronic, and Mechanical Engineering, Roma Tre University（工业、电子和机械工程系，罗马三大学）
University of Technology Sydney（悉尼科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Beijing Institute of Technology（北京理工大学）
Underwater robots in shallow waters usually suffer from strong wave forces, which may frequently exceed robot's control constraints. Learning-based controllers are suitable for disturbance rejection control, but the excessive disturbances heavily affect the state transition in Markov Decision Process (MDP) or Partially Observable Markov Decision Process (POMDP). Also, pure learning procedures on targeted system may encounter damaging exploratory actions or unpredictable system variations, and training exclusively on a prior model usually cannot address model mismatch from the targeted system. …
IST Austria (Institute of Science and Technology Austria)（奥地利科学技术研究所）；Indian Institute of Technology, Bombay（印度理工学院孟买分校）
We consider partially observable Markov decision processes (POMDPs), that are a standard framework for robotics applications to model uncertainties present in the real world, with temporal logic specifications. All temporal logic specifications in linear-time temporal logic (LTL) can be expressed as parity objectives. We study the qualitative analysis problem for POMDPs with parity objectives that asks whether there is a controller (policy) to ensure that the objective holds with probability 1 (almost-surely). While the qualitative analysis of POMDPs with parity objectives is undecidable, rece…
A Human Motion Compensation Framework for a Supernumerary Robotic Arm
Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences（中国科学院机器人与智能制造创新研究院）；Istituto Italiano di Tecnologia（意大利理工学院）
University of Tennessee（田纳西大学）；Miami University（迈阿密大学）；University of Tennessee Health Science Center（田纳西大学健康科学中心）；University of Iowa（爱荷华大学）
People with Alzheimer's disease and related dementias (ADRD) often show the problem of repetitive questioning, which brings a great burden on persons with ADRD (PwDs) and their caregivers. Conversational robots hold promise of coping with this problem and hence alleviating the burdens on caregivers. In this paper, we proposed a partially observable markov decision process (POMDP) model for the PwD-robot interaction in the context of repetitive questioning, and used Q-learning to learn an adaptive conversation strategy (i.e., rate of follow-up question and difficulty of follow-up question) towa…
University of Southern California（南加利福尼亚大学）；HomeAway, Inc.（HomeAway公司）；Indiana University - Bloomington（印第安纳大学伯明顿分校）
We investigate the scenario that a robot needs to reach a designated goal after taking a sequence of appropriate actions in a non-static environment that is partially structured. One application example is to control a marine vehicle to move in the ocean. The ocean environment is dynamic and oftentimes the ocean waves result in strong disturbances that can disturb the vehicle's motion. Modeling such dynamic environment is non-trivial, and integrating such model in the robotic motion control is particularly difficult. Fortunately, the ocean currents usually form some local patterns (e.g. vortex…
Autonomous agents are limited in their ability to observe the world state. Partially observable Markov decision processes (POMDPs) formally model the problem of planning under world state uncertainty, but POMDPs with continuous actions and nonlinear dynamics suitable for robotics applications are challenging to solve. In this paper, we present an efficient differential dynamic programming (DDP) algorithm for belief space planning in POMDPs with uncertainty over a discrete latent state, and continuous states, actions, observations, and nonlinear dynamics. This representation allows planning of …
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）
Mutual localization provides a consensus of reference frame as an essential basis for cooperation in multirobot systems. Previous works have developed certifiable and robust solvers for relative transformation estimation between each pair of robots. However, recovering relative poses for robotic swarm with partially mutual observations is still an unexploited problem. In this paper, we present a complete algorithm for it with optimality, scalability and robustness. Firstly, we fuse all odometry and bearing measurements in a unified minimization problem among the Stiefel manifold. Furthermore, …
We propose a new algorithm to simplify the controller development for distributed robotic systems subject to external observations, disturbances, and communication delays. Unlike prior approaches that propose specialized solutions to handling communication latency for specific robotic applications, our algorithm uses an arbitrary centralized controller as the specification and automatically generates distributed controllers with communication management and delay compensation. We formulate our goal as nonlinear optimal control -- using a regret minimizing objective that measures how much the d…
A neural network based flexible object manipulation system for a humanoid robot on FPGA is proposed. Although the manipulations of flexible objects using robots attract ever increasing attention since these tasks are the basic and essential activities in our daily life, it has been put into practice only recently with the help of deep neural networks. However such systems have relied on GPU accelerators, which cannot be implemented into the space limited robotic body. Although field programmable gate arrays (FPGAs) are known to be energy efficient and suitable for embedded systems, the model s…
University of Baghdad（巴格达大学）；University of Kirkuk（基尔库克大学）；Al-kitab University（阿尔基塔布大学）；University of Swat（斯瓦特大学）；Manipal University Jaipur（马尼帕尔大学斋浦尔分校）；University of Engineering & Technology Peshawar（白沙瓦工程技术大学）
Robotic manipulators are widely used in applications that require fast and precise motion. Such devices, however, are prompt to nonlinear control issues due to the flexibility in joints and the friction in the motors within the dynamics of their rigid part. To address these issues, the Linear Matrix Inequalities (LMIs) and Parallel Distributed Compensation (PDC) approaches are implemented in the Takagy-Sugeno Fuzzy Model (T-SFM). We propose the following methodology; initially, the state space equations of the nonlinear manipulator model are derived. Next, a Takagy-Sugeno Fuzzy Model (T-SFM) t…
SPI-BoTER: Error Compensation for Industrial Robots via Sparse Attention Masking and Hybrid Loss with Spatial-Physical Information
School of Software（软件学院）；Northwestern Polytechnical University（西北工业大学）；School of Computer Science and Technology（计算机科学与技术学院）；University of Science and Technology of China（中国科学技术大学）；Suzhou Key Lab of Aerospace Industry Software and Data Science（苏州航空航天软件与数据科学重点实验室）；Yangtze River Delta Research Institute of NPU（南京航空航天大学长江三角洲研究院）
Anytime a robot manipulator is controlled via visual feedback, the transformation between the robot and camera frame must be known. However, in the case where cameras can only capture a portion of the robot manipulator in order to better perceive the environment being interacted with, there is greater sensitivity to errors in calibration of the base-to-camera transform. A secondary source of uncertainty during robotic control are inaccuracies in joint angle measurements which can be caused by biases in positioning and complex transmission effects such as backlash and cable stretch. In this wor…
Technical University of Munich（慕尼黑工业大学）；Zhejiang University（浙江大学）；German Heart Center Munich（慕尼黑德国心脏中心）
Robotic ultrasound (US) imaging has been seen as a promising solution to overcome the limitations of free-hand US examinations, i.e., inter-operator variability. However, the fact that robotic US systems cannot react to subject movements during scans limits their clinical acceptance. Regarding human sonographers, they often react to patient movements by repositioning the probe or even restarting the acquisition, in particular for the scans of anatomies with long structures like limb arteries. To realize this characteristic, we proposed a vision-based system to monitor the subject's movement an…
Partially Observable Adversarial Patch Attacks on Vision-Language-Action Models in Robotics
Department of Automation, University of Science and Technology of China（自动化系，中国科学技术大学）；SmartMore Corporation（SmartMore公司）；Cyberspace Institute of Advanced Technology, Guangzhou University（广西亚技术空间研究所，广州大学）；th Medical Center of Chinese PLA General Hospital（中国人民解放军总医院第八医学中心）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Benchmarking and Enhancing Skill-Level Memory for Partially Observable Robotic Manipulation
University of Science and Technology of China（中国科学技术大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Zhejiang University（浙江大学）；Shanghai Jiaotong University（上海交通大学）；Nanjing University（南京大学）
Johns Hopkins University（约翰斯·霍普金斯大学）；Technische Universität München（慕尼黑工业大学）
Retinal microsurgery is a high-precision surgery performed on an exceedingly delicate tissue. It now requires extensively trained and highly skilled surgeons. Given the restricted range of instrument motion in the confined intraocular space, and also potentially restricting instrument contact with the sclera, snake-like robots may prove to be a promising technology to provide surgeons with greater flexibility, dexterity, space access, and positioning accuracy during retinal procedures requiring high precision and advantageous tooltip approach angles, such as retinal vein cannulation and epiret…
Massachusetts Institute of Technology（麻省理工学院）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）
Representing and reasoning about uncertainty is crucial for autonomous agents acting in partially observable environments with noisy sensors. Partially observable Markov decision processes (POMDPs) serve as a general framework for representing problems in which uncertainty is an important factor. Online sample-based POMDP methods have emerged as efficient approaches to solving large POMDPs and have been shown to extend to continuous domains. However, these solutions struggle to find long-horizon plans in problems with significant uncertainty. Exploration heuristics can help guide planning, but…
