Safe Uncertainty-Aware Learning of Robotic Suturing
SDU Robotics, The Maersk Mc-Kinney Moller Institute, University of Southern Denmark(SDU机器人研究所、马士基麦金尼莫勒研究所、南部丹麦大学);Department of Mechanical Engineering, Myongji University(机械工程系、明州大学)
Wilbert Peter Empleo, Yitaek Kim, Hansoul Kim, Thiusius Rajeeth Savarimuthu, Iñigo Iturrate
Robot-Assisted Minimally Invasive Surgery is currently fully manually controlled by a trained surgeon. Automating this has great potential for alleviating issues, e.g., physical strain, highly repetitive tasks, and shortages of trained surgeons. For these reasons, recent works have utilized Artificial Intelligence methods, which show promising adaptability. Despite these advances, there is skepticism of these methods because they lack explainability and robust safety guarantees. This paper presents a framework for a safe, uncertainty-aware learning method. We train an Ensemble Model of Diffusion Policies using expert demonstrations of needle insertion. Using an Ensemble model, we can quantify the policy's epistemic uncertainty, which is used to determine Out-Of-Distribution scenarios. This allows the system to release control back to the surgeon in the event of an unsafe scenario. Additionally, we implement a model-free Control Barrier Function to place formal safety guarantees on the predicted action. We experimentally evaluate our proposed framework using a state-of-the-art robotic suturing simulator. We evaluate multiple scenarios, such as dropping the needle, moving the camera, and moving the phantom. The learned policy is robust to these perturbations, showing corrective behaviors and generalization, and it is possible to detect Out-Of-Distribution scenarios. We further demonstrate that the Control Barrier Function successfully limits the action to remain within our specified safety set in the case of unsafe predictions.
Safety Aware Task Planning via Large Language Models in Robotics
University of Minnesota – Twin Cities(明尼苏达大学双城分校);Institute for Robotics and Intelligent Machines (IRIM), Georgia Institute of Technology(佐治亚理工学院机器人与智能机器研究所);Pontificia Universidad Católica de Chile(智利天主教大学)
Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic Manipulation
UNIST InnoCORE AI-Space Solar Initiative(UNIST创新核心人工智能空间太阳能计划);Ulsan National Institute of Science and Technology (UNIST)(乌山国立科学技术研究院);Automation and Systems Research Institute(自动化与系统研究所);Department of Electrical and Computer Engineering(电气与计算机工程系);Interdisciplinary Program in Artificial Intelligence(人工智能跨学科项目);LG Electronics(LG电子)
Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery
The Chinese University of Hong Kong(香港中文大学);Johns Hopkins University(约翰斯·霍普金斯大学)
自动手术手势识别对于机器人手术中的智能认知辅助至关重要。随着近年来机器人辅助微创手术的发展,可以记录包括手术视频和机器人运动学在内的丰富信息,这些信息为理解手术手势提供了补充知识。然而,现有的方法要么单独采用单模态数据或直接连接多模态表示,这不能充分利用视觉和运动学数据中固有的信息相关性来提高手势识别的准确性。在这方面,我们提出了一种新的多模态关系图网络的在线方法(即,MRG-Net)通过在潜在特征空间中的交互式消息传播来动态地整合视觉和运动学信息。具体来说,我们首先使用时间卷积网络和LSTM单元从视频和运动序列中提取嵌入。接下来,我们识别这些多模态嵌入中的多关系,并通过层次关系图学习模块来利用它们。我们的方法的有效性证明了国家的最先进的结果在公共JIGSAWS数据集,优于目前的单模态和多模态的方法在两个节点和结类型的任务。此外,我们在两个中心的da Vinci Research Kit(dVRK)平台收集的内部视觉运动学数据集上验证了我们的方法,并实现了一致的有希望的性能。
Confidence-based Intent Prediction for Teleoperation in Bimanual Robotic Suturing
Department of Mechanical Engineering, Imperial College London(机械工程系,帝国理工学院伦敦分校);Department of Surgery & Cancer, Imperial College London(外科与癌症系,帝国理工学院伦敦分校);Department of Engineering and Design, University of Sussex(工程与设计系,Sussex大学);Department of Bioengineering, Imperial College London(生物工程系,帝国理工学院伦敦分校)
CT-SAFR: Safe and Interpretable Chain-of-Thought Reasoning for Autonomous Robots: A Multi-Layered Verification Framework for Trustworthy AI-Driven Robotic Decision Making
PinPoint: Monocular Needle Pose Estimation for Robotic Suturing via Stein Variational Newton and Geometric Residuals
Department of Mechanical Engineering, Vanderbilt University(范德比尔特大学机械工程系);Department of Electrical and Computer Engineering and Department of Computer Science, Vanderbilt University(范德比尔特大学电气与计算机工程系和计算机科学系);The Robotics Center and the Kahlert School of Computing, University of Utah(犹他大学机器人中心和Kahlert计算学院)
Robust Adaptive Safe Robotic Grasping with Tactile Sensing
Maersk Mc-Kinney Moller Institute, University of Southern Denmark(马士基麦金尼默勒研究所,丹麦南方大学);Departments of Mechanical and Civil Engineering and Computing and Mathematical Sciences , California Institute of Technology(机械与土木工程系和计算与数学科学系,加州理工学院)
Predictive Relative-Velocity Steering for Safe Robotic Manipulator Teleoperation in Dynamic Environments
Zhejiang University(浙江大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Tsinghua University(清华大学);Institute for Embodied Intelligence and Robotics, Tsinghua University(清华大学具身智能与机器人研究所);TetraBOT
Model selection and real-time skill assessment for suturing in robotic surgery
Department of Mechanical Engineering, Imperial College London(帝国理工学院机械工程系);Department of Computing, Imperial College London(帝国理工学院计算机系);Department of Mechanical Engineering, Stanford University(斯坦福大学机械工程系);Intuitive Surgical, Inc.(Intuitive Surgical公司);Department of Bioengineering, Imperial College London(帝国理工学院生物工程系)
Robotic Control Optimization Through Kernel Selection in Safe Bayesian Optimization
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Singapore Institute of Manufacturing Technology, A*STAR(新加坡制造技术研究院(新加坡科研局));Department of Electrical and Computer Engineering, National University of Singapore(新加坡国立大学电气与计算机工程系);Robotics and Autonomous Systems Thrust, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)机器人与自主系统学部)
本文提出一种基于学习的控制优化方法,通过核选择增强加性高斯过程的安全贝叶斯优化算法,高效解决高维复杂系统的多控制器优化问题,并在Safe Control Gym基准的无人机PID控制中验证了其优于现有技术的显著改进。
safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(向量人工智能研究所)
近年来,强化学习和基于学习的控制-以及对它们的安全性的研究,这对于在现实世界的机器人中部署至关重要-都获得了巨大的吸引力。然而,为了充分衡量新结果的进展和适用性,我们需要工具来公平地比较控制和强化学习社区提出的方法。在这里,我们提出了一个新的开源基准套件,称为安全控制健身房,支持基于模型和基于数据的控制技术。我们提供了三个动态系统的实现-车杆,1D和2D四旋翼-和两个控制任务-稳定和轨迹跟踪。我们建议扩展OpenAI的Gym API -强化学习研究中的事实标准-(i)指定(和查询)符号动力学的能力和(ii)约束,以及(iii)(可重复)在控制输入,状态测量和惯性属性中注入模拟干扰。为了证明我们的建议,并试图使研究社区更紧密地联系在一起,我们展示了如何使用安全控制健身房定量比较控制性能,数据效率和安全性的多种方法从传统的控制,基于学习的控制,和强化学习的领域。
Safety-Ensured Robotic Control Framework for Cutting Task Automation in Endoscopic Submucosal Dissection
Division of Mechanical System Engineering, Myongji University, Yongin-si, Gyeonggi-do, Republic of Korea(机械系统工程系,明州大学,韩国京畿道 Yongin市);SDU Robotics, the Maersk Mc-Kinney Moller Institute, University of Southern Denmark, Odense, Denmark(SDU机器人,马士克麦金尼莫勒研究所,丹麦南部大学,Odense);Department of Mechanical Engineering, Myongji University, Yongin-si, Gyeonggi-do, Republic of Korea(机械工程系,明州大学,韩国京畿道 Yongin市)
Dynamic Log-Gaussian Process Control Barrier Function for Safe Robotic Navigation in Dynamic Environments
School of Intelligence Science and Engineering, Harbin Institute of Technology(智能科学与工程学院,哈尔滨工业大学);Department of Control Science and Engineering, Harbin Institute of Technology(控制科学与工程学院,哈尔滨工业大学)