在重症护理中评估离线多目标强化学习基准
Benchmarking Offline Multi-Objective Reinforcement Learning in Critical Care
AI总结：
本文评估了三种离线多目标强化学习算法在重症护理中的性能，发现PEDA DT在灵活性和多目标决策方面表现优异，为个性化医疗决策提供了新思路。
摘要：
在重症监护室等关键护理环境中，临床医生面临平衡相互冲突目标的复杂挑战，主要是在最大化患者存活率的同时最小化资源利用（例如住院时间）。单目标强化学习方法通常通过优化固定的标量化奖励函数来解决这一问题，导致刚性策略，无法适应变化的临床优先事项。多目标强化学习（MORL）通过学习Pareto前沿上的最佳策略集，允许在测试时动态选择偏好。然而，在医疗领域应用MORL需要严格从历史数据中进行离线学习。在本文中，我们对三种离线MORL算法，即条件保守Pareto Q学习（CPQL）、自适应CPQL以及改进的Pareto高效决策代理（PEDA）决策变换器（PEDA DT），与三种标量单目标基线（BC、CQL和DDQN）在MIMIC-IV数据集上进行基准测试。利用离策略评估（OPE）指标，我们证明PEDA DT算法相比静态标量基线具有更优越的灵活性。值得注意的是，我们的结果扩展了之前在医疗领域单目标决策变换器的研究发现，证实了当扩展到多目标条件生成时，序列建模架构仍保持稳健和有效。这些发现表明，离线MORL是一种有前景的框架，可以在不重新训练的情况下，为重症护理提供个性化、可调节的决策能力。
英文摘要：
In critical care settings such as the Intensive Care Unit, clinicians face the complex challenge of balancing conflicting objectives, primarily maximizing patient survival while minimizing resource utilization (e.g., length of stay). Single-objective Reinforcement Learning approaches typically address this by optimizing a fixed scalarized reward function, resulting in rigid policies that fail to adapt to varying clinical priorities. Multi-objective Reinforcement Learning (MORL) offers a solution by learning a set of optimal policies along the Pareto Frontier, allowing for dynamic preference selection at test time. However, applying MORL in healthcare necessitates strict offline learning from historical data. In this paper, we benchmark three offline MORL algorithms, Conditioned Conservative Pareto Q-Learning (CPQL), Adaptive CPQL, and a modified Pareto Efficient Decision Agent (PEDA) Decision Transformer (PEDA DT), against three scalarized single-objective baselines (BC, CQL, and DDQN) on the MIMIC-IV dataset. Using Off-Policy Evaluation (OPE) metrics, we demonstrate that PEDA DT algorithm offers superior flexibility compared to static scalarized baselines. Notably, our results extend previous findings on single-objective Decision Transformers in healthcare, confirming that sequence modeling architectures remain robust and effective when scaled to multi-objective conditioned generation. These findings suggest that offline MORL is a promising framework for enabling personalized, adjustable decision-making in critical care without the need for retraining.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
