基于屏蔽多智能体强化学习的连续机械手安全控制
Towards Safe Control of Continuum Manipulator Using Shielded Multiagent Reinforcement Learning
摘要：
Continuum机器人操作器越来越多地用于微创手术。然而，它们的非线性行为是具有挑战性的准确建模，特别是当受到外部相互作用，可能导致控制性能差。在这封信中，我们调查的可行性，采用无模型的多智能体强化学习（RL），即多智能体深度Q网络（MADQN），控制2自由度（DoF）电缆驱动的连续手术机械手。在MADQN框架中，机器人的控制被公式化为一个自由度，一个代理问题，以提高学习效率。结合屏蔽方案，使动态变化的动作集边界，MADQN导致有效的和重要的是更安全的控制机器人。屏蔽MADQN使机器人能够在外部负载，软障碍物和刚性碰撞下执行具有亚毫米均方根误差的点和轨迹跟踪，这些都是手术机械手遇到的常见交互场景。该控制器在具有高结构非线性的小型连续体机器人中得到了有效的应用，在外部载荷作用下实现了亚毫米级精度的轨迹跟踪。
英文摘要：
Continuum robotic manipulators are increasingly adopted in minimal invasive surgery. However, their nonlinear behavior is challenging to model accurately, especially when subject to external interaction, potentially leading to poor control performance. In this letter, we investigate the feasibility of adopting a model-free multiagent reinforcement learning (RL), namely multiagent deep Q network (MADQN), to control a 2-degree of freedom (DoF) cable-driven continuum surgical manipulator. The control of the robot is formulated as a one-DoF, one agent problem in the MADQN framework to improve the learning efficiency. Combined with a shielding scheme that enables dynamic variation of the action set boundary, MADQN leads to efficient and importantly safer control of the robot. Shielded MADQN enabled the robot to perform point and trajectory tracking with submillimeter root mean square errors under external loads, soft obstacles, and rigid collision, which are common interaction scenarios encountered by surgical manipulators. The controller was further proven to be effective in a miniature continuum robot with high structural nonlinearitiy, achieving trajectory tracking with submillimeter accuracy under external payload.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
