Clinician-in-the-Loop Decision Making: Reinforcement Learning with Near-Optimal Set-Valued Policies
University of Michigan（密歇根大学）；Michigan Medicine, University of Michigan（密歇根大学密歇根医学院）；Institute for Healthcare Policy & Innovation, University of Michigan（密歇根大学医疗政策与创新研究所）
Shengpu Tang, Aditya Modi, Michael W. Sjoding, Jenna Wiens
Standard reinforcement learning (RL) aims to find an optimal policy that identifies the best action for each state. However, in healthcare settings, many actions may be near-equivalent with respect to the reward (e.g., survival). We consider an alternative objective -- learning set-valued policies to capture near-equivalent actions that lead to similar cumulative rewards. We propose a model-free algorithm based on temporal difference learning and a near-greedy heuristic for action selection. We analyze the theoretical properties of the proposed algorithm, providing optimality guarantees and demonstrate our approach on simulated environments and a real clinical task. Empirically, the proposed algorithm exhibits good convergence properties and discovers meaningful near-equivalent actions. Our work provides theoretical, as well as practical, foundations for clinician/human-in-the-loop decision making, in which humans (e.g., clinicians, patients) can incorporate additional knowledge (e.g., side effects, patient preference) when selecting among near-equivalent actions.
Synthesising Reinforcement Learning Policies through Set-Valued Inductive Rule Learning
Vrije Universiteit Brussel（布鲁塞尔自由大学（荷兰语））；Université Libre de Bruxelles（布鲁塞尔自由大学（法语））；Delft University of Technology（代尔夫特理工大学）；Leiden Institute of Advanced Computer Science（莱顿大学高级计算机科学研究所）
Representation and Reinforcement Learning for Personalized Glycemic Control in Septic Patients
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Philips Connected Sensing Venture（飞利浦互联感知风险投资公司）；Philips Research North America（飞利浦北美研究院）；Masschusetts General Hospital（麻省总医院）
Vector-Valued Distributional Reinforcement Learning Policy Evaluation: A Hilbert Space Embedding Approach
Department of Statistics, University of Illinois Urbana-Champaign（统计系，伊利诺伊大学厄巴纳-香槟分校）；Department of Bioinformatics and Biostatistics, University of Louisville（生物信息学与生物统计学系，路易斯维尔大学）
Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies
College of Computer Science, Zhejiang University（浙江大学计算机科学与技术学院）；School of Medicine, Zhejiang University（浙江大学医学院）；Department of Biomedical Informatics, Harvard University（哈佛大学生物医学信息学系）
A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations
Universidade Federal de Pernambuco（伯南布哥联邦大学）；Centro de Informática（信息中心）；Western Sydney University（西悉尼大学）；MARCS Institute（MARCS研究所）；International Centre for Neuromorphic Engineering（国际神经形态工程中心）
The Chinese University of Hong Kong（香港中文大学）；University of Michigan（密歇根大学）；University of Oxford（牛津大学）；Tencent（腾讯）；Nanyang Technological University（南洋理工大学）
Quantum Policy Gradient in Reproducing Kernel Hilbert Space
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（高性能计算研究所（IHPC），科技研究局（A*STAR））；Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR)（前沿人工智能研究中心（CFAR），科技研究局（A*STAR））；Centre for Quantum Engineering, Research and Education, TCG CREST（量子工程、研究与教育中心（TCG CREST））
Learning Preference-Based Objectives from Clinical Narratives for Dynamic Sepsis Treatment
Institute of Data Science, National University of Singapore, Singapore（新加坡国立大学数据科学研究所）；National University Hospital, Singapore（新加坡国立大学医院）；Saw Swee Hock School of Public Health, National University of Singapore, Singapore（新加坡国立大学 Saw Swee Hock 公共卫生学院）
Treatment Policy Learning in Multiobjective Settings with Fully Observed Outcomes
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；IMES（医学工程与科学研究所）；Carnegie Mellon University（卡内基梅隆大学）；Harvard Medical School（哈佛医学院）；Harvard Pilgrim Healthcare Institute（哈佛朝圣者医疗保健研究所）
Patient-Centered Treatment Planning for Chronic Multimorbidity: A Hierarchical Reinforcement Learning Framework for Preference Modeling
University of Tennessee, Knoxville（田纳西大学诺克斯维尔分校）；University of Tennessee Health Science Center College of Medicine（田纳西大学健康科学中心医学院）；University of Tennessee Medical Center（田纳西大学医学中心）；University of Arkansas（阿肯色大学）
Symbolic Regression Methods for Reinforcement Learning
Czech Institute of Informatics, Robotics, and Cybernetics, Czech Technical University in Prague（布拉格捷克理工大学信息学、机器人与控制论研究所）；Faculty of Electrical Engineering, Czech Technical University in Prague（布拉格捷克理工大学电气工程学院）；Cognitive Robotics, Delft University of Technology（代尔夫特理工大学认知机器人系）
Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions
Department of Computer Science, University of Southern California（计算机科学系，南加州大学）；Department of Data Sciences and Operations and Computer Science（数据科学与运营与计算机科学系，南加州大学）
Tsinghua University（清华大学）；University of Science and Technology Beijing（北京科技大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
