面向机械循环辅助装置智能撤机的守护正则化安全离线强化学习
Guardian-regularized Safe Offline Reinforcement Learning for Smart Weaning of Mechanical Circulatory Devices
AI总结：
本文提出CORMPO密度正则化离线强化学习算法和基于Transformer的概率数字孪生，用于机械循环支持装置的安全自动撤机决策，在真实和合成数据上奖励提升28%、临床指标提升82.6%。
摘要：
我们研究了心源性休克患者机械循环支持（MCS）装置自动撤机的序贯决策问题。MCS装置是经皮微轴流泵，可提供左心室卸载和前向血流，但目前的撤机策略在不同医护团队之间差异很大，且缺乏数据驱动的方法。离线强化学习（RL）已在序贯决策任务中取得成功，但我们的场景给训练和评估传统离线RL方法带来了挑战：禁止在线患者交互、由于并发治疗导致循环动力学高度不确定，以及数据可用性有限。我们开发了一个端到端机器学习框架，包含两项关键贡献：（1）临床感知的分布外正则化基于模型的策略优化（CORMPO），这是一种用于分布外抑制的密度正则化离线RL算法，同时融合了临床信息引导的奖励塑形；（2）一个基于Transformer的概率数字孪生，用于对MCS循环动力学建模，以丰富的生理和临床指标进行策略评估。我们证明了CORMPO在温和假设下可获得理论性能保证。在真实和合成数据集上，CORMPO获得的奖励比离线RL基线高28%，临床指标得分高82.6%。我们的方法为高风险医疗应用中的安全离线策略学习提供了一个有原则的框架，在这些应用中领域专业知识和安全约束至关重要。
英文摘要：
We study the sequential decision-making problem for automated weaning of mechanical circulatory support (MCS) devices in cardiogenic shock patients. MCS devices are percutaneous micro-axial flow pumps that provide left ventricular unloading and forward blood flow, but current weaning strategies vary significantly across care teams and lack data-driven approaches. Offline reinforcement learning (RL) has proven to be successful in sequential decision-making tasks, but our setting presents challenges for training and evaluating traditional offline RL methods: prohibition of online patient interaction, highly uncertain circulatory dynamics due to concurrent treatments, and limited data availability. We developed an end-to-end machine learning framework with two key contributions (1) Clinically-aware OOD-regularized Model-based Policy Optimization (CORMPO), a density-regularized offline RL algorithm for out-of-distribution suppression that also incorporates clinically-informed reward shaping and (2) a Transformer-based probabilistic digital twin that models MCS circulatory dynamics for policy evaluation with rich physiological and clinical metrics. We prove that \textsf{CORMPO} achieves theoretical performance guarantees under mild assumptions. CORMPO attains a higher reward than the offline RL baselines by 28% and higher scores in clinical metrics by 82.6% on real and synthetic datasets. Our approach offers a principled framework for safe offline policy learning in high-stakes medical applications where domain expertise and safety constraints are essential.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
