Our aim is to establish a framework where reinforcement learning (RL) of optimizing interventions retrospectively allows us a regulatory compliant pathway to prospective clinical testing of the learned policies in a clinical deployment. We focus on infections in intensive care units which are one of the major causes of death and difficult to treat because of the complex and opaque patient dynamics, and the clinically debated, highly-divergent set of intervention policies required by each individual patient, yet intensive care units are naturally data rich. In our work, we build on RL approaches in healthcare ("AI Clinicians"), and learn off-policy continuous dosing policy of pharmaceuticals for sepsis treatment using historical intensive care data under partially observable MDPs (POMDPs). POMPDs capture uncertainty in patient state better by taking in all historical information, yielding an efficient representation, which we investigate through ablations. We compensate for the lack of exploration in our retrospective data by evaluating each encountered state with a best-first tree search. We mitigate state distributional shift by optimizing our policy in the vicinity of the clinicians' compound policy. Crucially, we evaluate our model recommendations using not only conventional policy evaluations but a novel framework that incorporates human experts: a model-agnostic pre-clinical evaluation method to estimate the accuracy and uncertainty of clinician's decisions versus our system recommendations when confronted with the same individual patient history ("shadow mode").
Cambridge University（剑桥大学）；Imperial College London（伦敦帝国学院）；MIT（麻省理工学院）
Sepsis is a leading cause of mortality in intensive care units and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. In this work, we propose an approach to deduce treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Our model learns clinically interpretable treatment policies, similar in important aspects to the treatment policies of physicians. The learned policie…
University of New South Wales（新南威尔士大学）；Australian National University（澳大利亚国立大学）；The George Institute for Global Health（乔治全球健康研究院）；Imperial College London（伦敦帝国理工学院）
These two synthetic datasets comprise vital signs, laboratory test results, administered fluid boluses and vasopressors for 3,910 patients with acute hypotension and for 2,164 patients with sepsis in the Intensive Care Unit (ICU). The patient cohorts were built using previously published inclusion and exclusion criteria and the data were created using Generative Adversarial Networks (GANs) and the MIMIC-III Clinical Database. The risk of identity disclosure associated with the release of these data was estimated to be very low (0.045%). The datasets were generated and published as part of the …
Beijing Institute of Technology（北京理工大学）；Tsinghua University（清华大学）
Sepsis is a leading cause of death in the ICU. It is a disease requiring complex interventions in a short period of time, but its optimal treatment strategy remains uncertain. Evidence suggests that the practices of currently used treatment strategies are problematic and may cause harm to patients. To address this decision problem, we propose a new medical decision model based on historical data to help clinicians recommend the best reference option for real-time treatment. Our model combines offline reinforcement learning and deep reinforcement learning to solve the problem of traditional rei…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of Chicago（芝加哥大学）；LLNL（劳伦斯利弗莫尔国家实验室）
Sepsis is a life-threatening condition affecting one million people per year in the US in which dysregulation of the body's own immune system causes damage to its tissues, resulting in a 28 - 50% mortality rate. Clinical trials for sepsis treatment over the last 20 years have failed to produce a single currently FDA approved drug treatment. In this study, we attempt to discover an effective cytokine mediation treatment strategy for sepsis using a previously developed agent-based model that simulates the innate immune response to infection: the Innate Immune Response agent-based model (IIRABM).…
Sepsis is a leading cause of mortality and its treatment is very expensive. Sepsis treatment is also very challenging because there is no consensus on what interventions work best and different patients respond very differently to the same treatment. Deep Reinforcement Learning methods can be used to come up with optimal policies for treatment strategies mirroring physician actions. In the healthcare scenario, the available data is mostly collected offline with no interaction with the environment, which necessitates the use of offline RL techniques. The Offline RL paradigm suffers from action …
UC, Davis Health（加州大学戴维斯健康中心）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Sepsis is a leading cause of death in the Intensive Care Units (ICU). Early detection of sepsis is critical for patient survival. In this paper, we propose a multimodal Transformer model for early sepsis prediction, using the physiological time series data and clinical notes for each patient within $36$ hours of ICU admission. Specifically, we aim to predict sepsis using only the first 12, 18, 24, 30 and 36 hours of laboratory measurements, vital signs, patient demographics, and clinical notes. We evaluate our model on two large critical care datasets: MIMIC-III and eICU-CRD. The proposed meth…
ETH Zurich（苏黎世联邦理工学院）；SIB Swiss Institute of Bioinformatics（SIB瑞士生物信息学研究所）
Despite decades of clinical research, sepsis remains a global public health crisis with high mortality, and morbidity. Currently, when sepsis is detected and the underlying pathogen is identified, organ damage may have already progressed to irreversible stages. Effective sepsis management is therefore highly time-sensitive. By systematically analysing trends in the plethora of clinical data available in the intensive care unit (ICU), an early prediction of sepsis could lead to earlier pathogen identification, resistance testing, and effective antibiotic and supportive treatment, and thereby be…
Sepsis is an important cause of mortality, especially in intensive care unit (ICU) patients. Developing novel methods to identify early mortality is critical for improving survival outcomes in sepsis patients. Using the MIMIC-III database, we integrated demographic data, physiological measurements and clinical notes. We built and applied several machine learning models to predict the risk of hospital mortality and 30-day mortality in sepsis patients. From the clinical notes, we generated clinically meaningful word representations and embeddings. Supervised learning classifiers and a deep learn…
We study multiple rule-based and machine learning (ML) models for sepsis detection. We report the first neural network detection and prediction results on three categories of sepsis. We have used the retrospective Medical Information Mart for Intensive Care (MIMIC)-III dataset, restricted to intensive care unit (ICU) patients. Features for prediction were created from only common vital sign measurements. We show significant improvement of AUC score using neural network based ensemble model compared to single ML and rule-based models. For the detection of sepsis, severe sepsis, and septic shock…
Ben-Gurion University of the Negev（内盖夫本-古里安大学）；Soroka University Medical Center（索罗卡大学医学中心）
Sepsis is a condition caused by the body's overwhelming and life-threatening response to infection, which can lead to tissue damage, organ failure, and finally death. Common signs and symptoms include fever, increased heart rate, increased breathing rate, and confusion. Sepsis is difficult to predict, diagnose, and treat. Patients who develop sepsis have an increased risk of complications and death and face higher health care costs and longer hospitalization. Today, sepsis is one of the leading causes of mortality among populations in intensive care units (ICUs). In this paper, we look at the …
Computer Science and Artificial Intelligence Lab, MIT（麻省理工学院计算机科学与人工智能实验室）
Sepsis is a leading cause of mortality in intensive care units (ICUs) and costs hospitals billions annually. Treating a septic patient is highly challenging, because individual patients respond very differently to medical interventions and there is no universally agreed-upon treatment for sepsis. Understanding more about a patient's physiological state at a given time could hold the key to effective treatment policies. In this work, we propose a new approach to deduce optimal treatment policies for septic patients by using continuous state-space models and deep reinforcement learning. Learning…
University of New South Wales（新南威尔士大学）；Australian National University（澳大利亚国立大学）；The George Institute for Global Health（乔治全球健康研究院）；Imperial College London（帝国理工学院）；Hospital Universitario San Cecilio（圣塞西里奥大学医院）；Karolinska Institutet（卡罗林斯卡学院）；Università degli Studi di Siena（锡耶纳大学）；Universitätsklinikum Köln（科隆大学医院）
In recent years, the machine learning research community has benefited tremendously from the availability of openly accessible benchmark datasets. Clinical data are usually not openly available due to their highly confidential nature. This has hampered the development of reproducible and generalisable machine learning applications in health care. Here we introduce the Health Gym - a growing collection of highly realistic synthetic medical datasets that can be freely accessed to prototype, evaluate, and compare machine learning algorithms, with a specific focus on reinforcement learning. The th…
A federated learning framework with knowledge graph and temporal transformer for early sepsis prediction in multi-center ICUs
Chengdu Medical College（成都医学院）；Kunming Medical University（昆明医科大学）；National Yang Ming Chiao Tung University（国立阳明交通大学）；University of Southern California（南加州大学）；Yanshan University（燕山大学）
Background Sepsis is one of the most life-threatening circumstances for critically ill patients in the US, while a standardized criteria for sepsis identification is still under development. Disparities in social determinants of sepsis patients can interfere with the risk prediction performances using machine learning. Methods Disparities in social determinants, including race, gender, marital status, insurance types and languages, among patients identified by six available sepsis criteria were revealed by forest plots. Sixteen machine learning classifiers were trained to predict in-hospital m…
Enversion A/S（恩维申公司）；Aarhus University（奥胡斯大学）；Aarhus University Hospital（奥胡斯大学医院）；Horsens Regional Hospital（霍森斯地区医院）；Aarhus University School of Engineering（奥胡斯大学工程学院）
The timeliness of detection of a sepsis event in progress is a crucial factor in the outcome for the patient. Machine learning models built from data in electronic health records can be used as an effective tool for improving this timeliness, but so far the potential for clinical implementations has been largely limited to studies in intensive care units. This study will employ a richer data set that will expand the applicability of these models beyond intensive care units. Furthermore, we will circumvent several important limitations that have been found in the literature: 1) Models are evalu…
AI-powered skin spectral imaging enables instant sepsis diagnosis and outcome prediction in critically ill patients
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Helmholtz Information and Data Science School for Health（亥姆霍兹健康信息与数据科学学院）；Heidelberg University（海德堡大学）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；Heidelberg University Hospital（海德堡大学医院）；University Medical Center Mannheim（曼海姆大学医学中心）；DKFZ Hector Cancer Institute（DKFZ赫克托癌症研究所）
Heidelberg University（海德堡大学）；University of Manitoba（曼尼托巴大学）
Sepsis is the leading cause of death in non-coronary intensive care units. Moreover, a delay of antibiotic treatment of patients with severe sepsis by only few hours is associated with increased mortality. This insight makes accurate models for early prediction of sepsis a key task in machine learning for healthcare. Previous approaches have achieved high AUROC by learning from electronic health records where sepsis labels were defined automatically following established clinical criteria. We argue that the practice of incorporating the clinical criteria that are used to automatically define g…
Rare Event Early Detection: A Dataset of Sepsis Onset for Critically Ill Trauma Patients
University of Washington Tacoma WA USA（华盛顿大学塔科马分校）；Department of Statistics University of Washington Seattle WA USA（华盛顿大学统计学系）；Department of Surgery University of Washington Seattle WA USA（华盛顿大学外科系）
Georgia Institute of Technology（佐治亚理工学院）；Emory University School of Medicine（埃默里大学医学院）；University of California San Diego Health（加州大学圣地亚哥分校健康中心）
Sepsis, a dysregulated immune system response to infection, is among the leading causes of morbidity, mortality, and cost overruns in the Intensive Care Unit (ICU). Early prediction of sepsis can improve situational awareness amongst clinicians and facilitate timely, protective interventions. While the application of predictive analytics in ICU patients has shown early promising results, much of the work has been encumbered by high false-alarm rates. Efforts to improve specificity have been limited by several factors, most notably the difficulty of labeling sepsis onset time and the low preval…
IMEDACS；Vivace Health Solutions Inc（Vivace健康解决方案公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；Scripps Memorial Hospital - Encinitas（斯克里普斯恩西尼塔斯纪念医院）
Although timely sepsis diagnosis and prompt interventions in Intensive Care Unit (ICU) patients are associated with reduced mortality, early clinical recognition is frequently impeded by non-specific signs of infection and failure to detect signs of sepsis-induced organ dysfunction in a constellation of dynamically changing physiological data. The goal of this work is to identify patient at risk of life-threatening sepsis utilizing a data-centered and machine learning-driven approach. We derive a mortality risk predictive dynamic Bayesian network (DBN) guided by a customized sepsis knowledgeba…
Machine Learning-Based Prediction of ICU Mortality in Sepsis-Associated Acute Kidney Injury Patients Using MIMIC-IV Database with Validation from eICU Database
University of Southern California（南加州大学）；Iran University of Medical Sciences（伊朗医学科学大学）；California State University（加利福尼亚州立大学）；California State Polytechnic University（加利福尼亚州_polytechnic大学）
Heidelberg University Hospital（海德堡大学医院）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；HIDSS4Health - Helmholtz Information and Data Science School for Health（亥姆霍兹健康信息与数据科学学校）；Heidelberg University（海德堡大学）；University Hospital Essen（埃森大学医院）；University Duisburg-Essen（杜伊斯堡-埃森大学）
Sepsis is a leading cause of mortality and critical illness worldwide. While robust biomarkers for early diagnosis are still missing, recent work indicates that hyperspectral imaging (HSI) has the potential to overcome this bottleneck by monitoring microcirculatory alterations. Automated machine learning-based diagnosis of sepsis based on HSI data, however, has not been explored to date. Given this gap in the literature, we leveraged an existing data set to (1) investigate whether HSI-based automated diagnosis of sepsis is possible and (2) put forth a list of possible confounders relevant for …
Data-Driven Machine Learning Approaches for Predicting In-Hospital Sepsis Mortality
University of Southern California（南加州大学）；California State Polytechnic University, Pomona（加州州立理工大学波莫纳分校）；California State University, Long Beach（加州州立大学长滩分校）
A machine learning framework for interpretable predictions in patient pathways: The case of predicting ICU admission for patients with symptoms of sepsis
Technische Hochschule Nürnberg Georg Simon Ohm（纽伦堡乔治·西蒙·欧姆应用技术大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；University of Regensburg（雷根斯堡大学）；Leipzig University（莱比锡大学）
Clinically Interpretable Sepsis Early Warning via LLM-Guided Simulation of Temporal Physiological Dynamics
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Department of Information Engineering and Computer Science, University of Trento（特伦托大学信息工程与计算机科学系）；Department of Orthopedics, Affiliated Kunshan Hospital of Jiangsu University（江苏大学附属昆山医院骨科部）；Tianjin University Chest Hospital（天津大学胸科医院）；Haihe Hospital, Tianjin University（天津大学海河医院）；Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University（广西师范大学教育区块链与智能技术重点实验室）
Computer Technology Associates（计算机技术协会）；Imedacs（伊梅达克斯公司）；Children’s National Health System（儿童国家医疗系统）；NYU Langone Health（纽约大学朗格尼医学中心）；MedStar Health System（梅德斯塔医疗系统）
Background: While machine learning (ML) models are rapidly emerging as promising screening tools in critical care medicine, the identification of homogeneous subphenotypes within populations with heterogeneous conditions such as pediatric sepsis may facilitate attainment of high-predictive performance of these prognostic algorithms. This study is aimed to identify subphenotypes of pediatric sepsis and demonstrate the potential value of partitioned data/subtyping-based training. Methods: This was a retrospective study of clinical data extracted from medical records of 6,446 pediatric patients t…
Sepsis Prediction Using Graph Convolutional Networks over Patient-Feature-Value Triplets
College of Computer and Information Science（计算机与信息科学学院）；State Key Laboratory of Public Big Data（公共大数据国家重点实验室）；Department of Anesthesiology（麻醉科）；Key Laboratory of Perioperative Multi-organ Protection（围术期多器官保护关键实验室）
Contextual Phenotyping of Pediatric Sepsis Cohort Using Large Language Models
Department of Surgery, Duke university school of medicine（杜克大学医学学院外科部）；Department of Anesthesiology, Duke university school of medicine（杜克大学医学学院麻醉科）；Indian Institute of Technology, Goa（印度去邦理工学院）；Institute for Global Health, BC Children’s Hospital and BC Women’s Hospital + Health Centre（全球健康研究所，BC儿童医院和BC妇女医院及健康中心）；Department of Anesthesia, Pharmacology & Therapeutics, University of British Columbia（不列颠哥伦比亚大学麻醉学、药理学与治疗学系）；BC Children’s Hospital Research Institute, BC Children’s Hospital（BC儿童医院研究学院，BC儿童医院）；Department of Pediatrics, University of British Columbia（不列颠哥伦比亚大学儿科系）；Walimu, Kampala, Uganda（乌干达坎帕拉沃尔imu）；Department of Community Health, Mbarara University of Science and Technology（马巴拉拉大学科学与技术学院社区健康系）
Advancing clinical trial outcomes using deep learning and predictive modelling: bridging precision medicine and patient-centered care
Luddy School of Informatics, Computing and Engineering, IU Indianapolis（印第安纳大学印第安纳波利斯分校拉迪信息学、计算与工程学院）；New York Medical College（纽约医学院）；University of North Carolina Greensboro（北卡罗来纳大学格林斯伯勒分校）
Data reuse enables cost-efficient randomized trials of medical AI models
UC Berkeley（加州大学伯克利分校）；UCSF（加州大学旧金山分校）；UC Berkeley Biostatistics（加州大学伯克利分校生物统计学）；UC Berkeley College of Computing, Data Science, and Society（加州大学伯克利分校计算、数据科学与社会学院）；UCSF Department of Radiology and Biomedical Imaging（加州大学旧金山分校放射学与生物医学影像学系）
Algorithms and technologies are essential tools that pervade all aspects of our daily lives. In the last decades, health care research benefited from new computer-based recruiting methods, the use of federated architectures for data storage, the introduction of innovative analyses of datasets, and so on. Nevertheless, health care datasets can still be affected by data bias. Due to data bias, they provide a distorted view of reality, leading to wrong analysis results and, consequently, decisions. For example, in a clinical trial that studied the risk of cardiovascular diseases, predictions were…
Georgia Institute of Technology（佐治亚理工学院）；IQVIA（艾昆纬）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Counterfactual prediction is about predicting outcome of the unobserved situation from the data. For example, given patient is on drug A, what would be the outcome if she switch to drug B. Most of existing works focus on modeling counterfactual outcome based on static data. However, many applications have time-varying confounding effects such as multiple treatments over time. How to model such time-varying effects from longitudinal observational data? How to model complex high-dimensional dependency in the data? To address these challenges, we propose Deep Recurrent Inverse TreatmEnt weighting…
Organ-Agents: Virtual Human Physiology Simulator via LLMs
Tianjin University（天津大学）；University of Trento（特伦托大学）；Tianjin Medical University General Hospital（天津医科大学总医院）；Chest Hospital, Tianjin University（天津大学胸科医院）；The Affiliated Suzhou Hospital of Nanjing Medical University（南京医科大学附属苏州医院）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
We introduce a novel apprenticeship learning algorithm to learn an expert's underlying reward structure in off-policy model-free \emph{batch} settings. Unlike existing methods that require a dynamics model or additional data acquisition for on-policy evaluation, our algorithm requires only the batch data of observed expert behavior. Such settings are common in real-world tasks---health care, finance or industrial processes ---where accurate simulators do not exist or data acquisition is costly. To address challenges in batch settings, we introduce Deep Successor Feature Networks(DSFN) that est…
Massachusetts Institute of Technology（麻省理工学院）；Harvard University（哈佛大学）；Universitat de Barcelona（巴塞罗那大学）；Instituto de Salud Carlos III（卡洛斯三世卫生研究所）；Hospital Universitari Germans Trias i Pujol（Germans Trias i Pujol大学医院）；Universidad de Guadalajara（瓜达拉哈拉大学）；Centro Universitario de Ciencias de la Salud（健康科学中心大学学院）；OPD Hospital Civil de Guadalajara（瓜达拉哈拉市立医院）
Just like your phone can detect what song is playing in crowded spaces, we show that Artificial Intelligence transfer learning algorithms trained on cough phone recordings results in diagnostic tests for COVID-19. To gain adoption by the health care community, we plan to validate our results in a clinical trial and three other venues in Mexico, Spain and the USA . However, if we had data from other on-going clinical trials and volunteers, we may do much more. For example, for confirmed stay-at-home COVID-19 patients, a longitudinal audio test could be developed to determine contact-with-hospit…
