用于安全结肠镜导航的约束强化学习与形式化验证
Constrained Reinforcement Learning and Formal Verification for Safe Colonoscopy Navigation
AI总结：
针对机器人结肠镜自主导航在遇阻和急弯时的安全问题，提出无需管腔追踪的DRL策略，并用CRL约束及FV模型选择，在虚拟环境中从300个策略筛出3个完全安全策略。
摘要：
机器人柔性内窥镜（Flexible Endoscopes, FEs）领域已取得显著进展，为减轻患者不适提供了一种有前景的解决方案。然而，大多数机器人柔性内窥镜的自主能力有限，导致操作缺乏直观性且难度较高，从而限制了其在临床环境中的应用。尽管以往研究采用管腔追踪实现自主导航，但当内窥镜面对结肠壁时，这些方法无法适应障碍物和急弯的存在。在本研究中，我们提出一种基于深度强化学习（Deep Reinforcement Learning, DRL）的导航策略，无需进行管腔追踪。然而，使用DRL方法存在安全风险，因为这些方法没有考虑所采取动作可能带来的潜在危险。为确保安全，我们采用约束强化学习（Constrained Reinforcement Learning, CRL）方法，将策略限制在预定义的安全范围内。此外，我们提出一种模型选择策略，利用形式化验证（Formal Verification, FV）在部署前选择完全安全的策略。我们在虚拟结肠镜检查环境中验证了该方法，并报告在300个经过训练的策略中，能够识别出3个完全安全的策略。我们的工作表明，CRL与通过FV进行的模型选择相结合，可以提高机器人行为在外科应用中的鲁棒性和安全性。
英文摘要：
The field of robotic Flexible Endoscopes (FEs) has progressed significantly, offering a promising solution to reduce patient discomfort. However, the limited autonomy of most robotic FEs results in non-intuitive and challenging manoeuvres, constraining their application in clinical settings. While previous studies have employed lumen tracking for autonomous navigation, they fail to adapt to the presence of obstructions and sharp turns when the endoscope faces the colon wall. In this work, we propose a Deep Reinforcement Learning (DRL)-based navigation strategy that eliminates the need for lumen tracking. However, the use of DRL methods poses safety risks as they do not account for potential hazards associated with the actions taken. To ensure safety, we exploit a Constrained Reinforcement Learning (CRL) method to restrict the policy in a predefined safety regime. Moreover, we present a model selection strategy that utilises Formal Verification (FV) to choose a policy that is entirely safe before deployment. We validate our approach in a virtual colonoscopy environment and report that out of the 300 trained policies, we could identify three policies that are entirely safe. Our work demonstrates that CRL, combined with model selection through FV, can improve the robustness and safety of robotic behaviour in surgical applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
