One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models
Department of Computer Science, Columbia University（哥伦比亚大学计算机科学系）；Department of Biomedical Informatics, Columbia University（哥伦比亚大学生物医学信息学系）；Department of Computational Medicine, UCLA（洛杉矶大学计算医学系）；Formation Bio
Zilin Jing, Vincent Jeanselme, Yuta Kobayashi, Simon A. Lee, Chao Pang, Aparajita Kashyap, Yanwei Li, Xinzhuo Jiang, Shalmali Joshi
Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages. The sequential nature of EHR, analogous to natural language, has motivated the use of next-token prediction to train prior EHR Foundation Models (FMs) over events. However, this training fails to capture the full structure of EHR. When a given event occurs must be captured, but the event value (abnormal lab) also modulates the likelihood of other clinical events. Most existing EHR FMs do not jointly model this likelihood and are unable to capture the full observation process, impacting downstream capabilities. We propose ORA, a marked time-to-event pretraining objective that jointly models event timing and associated measurements. Across multiple datasets, downstream tasks, and model backbones, this objective consistently yields more generalizable representations than next-token prediction and pretraining losses that ignore continuous measurements. Importantly, the proposed objective yields improvements beyond traditional classification evaluation, including better regression and time-to-event prediction. Beyond introducing a new family of FMs, our ablations suggest a broader takeaway: pretraining objectives that account for EHR structure are critical for expanding downstream capabilities and generalizability.
An Investigation of Memorization Risk in Healthcare Foundation Models
MIT Broad Institute of MIT and Harvard（MIT Broad研究所（MIT和哈佛））；Vector Institute（向量研究所）；MIT；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；University of Toronto（多伦多大学）；University Health Network (UHN)（大学健康网络）；Worcester Polytechnic Institute（沃思维尔理工学院）；Computer Science Department（计算机科学系）
Coefficient of Variation Masking: A Volatility-Aware Strategy for EHR Foundation Models
Massachusetts Institute of Technology (MIT)（麻省理工学院）；Technical University of Munich (TUM)（慕尼黑技术大学）；MICS CentraleSupélec – Université Paris-Saclay（巴黎萨克雷大学CentraleSupélec研究所）；Harvard Medical School（哈佛医学院）；Beth Israel Deaconess Medical Center（贝斯以色列医疗中心）；Institute of Pathology Technical University of Munich（慕尼黑技术大学病理研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Tokenization Tradeoffs in Structured EHR Foundation Models
Child Health Evaluative Sciences（儿童健康评估科学）；The Hospital for Sick Children（圣迈克尔医院）；University Health Network（大学健康网络）；Department of Medicine（医学部）；Division of Computational Medicine, Department of Medicine（计算医学科，医学部）；Stanford University（斯坦福大学）
ICU Bloodstream Infection Prediction: A Transformer-Based Approach for EHR Analysis
Taub Faculty of Computer Science, Technion-Israel Institute of Technology（以色列理工学院陶布计算机科学学院）；Faculty of Biology, Technion-Israel Institute of Technology（以色列理工学院生物学院）
Scaling Recurrence-aware Foundation Models for Clinical Records via Next-Visit Prediction
Center for Data Science（数据科学中心）；New York University（纽约大学）；Department of Radiology（放射科）；NYU Grossman School of Medicine（NYU Grossman医学院）；Center for Biomedical Imaging（生物医学成像中心）；Department of Population Health（人群健康部）
Time-Aware Attention for Enhanced Electronic Health Records Modeling
Department of Statistics and Data Science, National University of Singapore（统计与数据科学系，新加坡国立大学）；School of Computing, National University of Singapore（计算学院，新加坡国立大学）；Department of Biostatistics, Harvard T.H. Chan School of Public Health, USA（生物统计学系，哈佛T.H. Chan公共卫生学院，美国）
A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records
Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, 1 Canada（儿童健康评估科学计划，多伦多医院儿童医学中心，多伦多，加拿大）；Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA（斯坦福大学生物医学信息学研究中心，斯坦福大学，帕洛阿尔托，美国）；Universidad del Norte, Barranquilla, Columbia（北方大学，巴兰基利亚，哥伦比亚）；Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, ON, Canada（血液肿瘤科，多伦多医院儿童医学中心，多伦多，加拿大）
Large Language Models are Powerful Electronic Health Record Encoders
Berlin Institute of Health at Charité – Universitätsmedizin Berlin（柏林夏里特医学院健康研究所）；BIH Biomedical Innovation Academy, BIH Charité Digital Clinician Scientist Program（BIH生物医学创新学院，BIH夏里特数字临床科学家计划）；Deutsches Herzzentrum der Charité – Medical Heart Center of Charité and German Heart Institute Berlin（夏里特德国心脏中心 – 夏里特医学院和德国心脏研究所柏林）；Layer Health, Inc.（Layer Health公司）
CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRT
University of Copenhagen（哥本哈根大学）；Copenhagen University Hospital - Bispebjerg and Frederiksberg（哥本哈根大学医院 - 比斯珀比约和腓特烈斯贝）；Copenhagen University Hospital - Rigshospitalet（哥本哈根大学医院 - 里格斯医院）
本文提出 CORE-BEHRT，通过增量优化研究基于 BERT 的 EHR 建模，分离数据表示、技术组件和训练过程的影响，在 25 项临床预测任务中显著提升了性能和泛化能力。
DT-Transformer: A Foundation Model for Disease Trajectory Prediction on a Real-world Health System
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School（药理流行病学与药效学部，医学部，布里奇沃特医院，哈佛医学院）；Harvard T.H. Chan School of Public Health, Harvard University（哈佛大学T.H. 陈公共卫生学院）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University（自然与人工智能研究学院，哈佛大学）；Broad Institute of MIT and Harvard, Cambridge, MA, USA（MIT与哈佛大学Broad研究所，剑桥，马萨诸塞州，美国）；Harvard Data Science Initiative, Harvard University（哈佛大学数据科学计划）
Stanford University（斯坦福大学）；Stanford Medical Center（斯坦福医学中心）；Stanford Health Care（斯坦福医疗保健系统）；Stanford Center for Artificial Intelligence in Medicine and Imaging（斯坦福医学与影像人工智能中心）；Prealize Health；Stanford School of Medicine（斯坦福医学院）；Human-Centered Artificial Intelligence Institute, Stanford University（斯坦福大学以人为本人工智能研究所）；Clinical Excellence Research Center, Stanford School of Medicine（斯坦福医学院临床卓越研究中心）
Generalist Foundation Models Are Not Clinical Enough for Hospital Operations
Courant Institute School of Mathematics, Computing, and Data Science（Courant学院数学、计算与数据科学学院）；New York University（纽约大学）；Department of Neurosurgery（神经外科部）；Global AI Frontier Lab（全球人工智能前沿实验室）；Electrical and Computer Engineering（电气与计算机工程）；Tandon School of Engineering（Tandon工程学院）；Grossman School of Medicine（Grossman医学院）；Department of Medicine（医学部）；Department of Computer Science（计算机科学部）；Department of Surgery（外科部）；Division of Applied AI Technologies（应用人工智能技术部）；Department of Population Health（人口健康部）；ETH Zurich（苏黎世联邦理工学院）
Zero-shot Medical Event Prediction Using a Generative Pre-trained Transformer on Electronic Health Records
Biomedical AI Research Lab, University of California, Los Angeles, Los Angeles, USA（生物医学人工智能研究实验室，加州大学洛杉矶分校）；Department of Radiology, University of California, Los Angeles, Los Angeles, USA（放射科，加州大学洛杉矶分校）；Department of Bioengineering, University of California, Los Angeles, Los Angeles, USA（生物工程系，加州大学洛杉矶分校）；Bioinformatics, University of California, Los Angeles, Los Angeles, USA（生物信息学，加州大学洛杉矶分校）；Department of Pediatrics, Division of Immunology, Allergy and Rheumatology, University of California, Los Angeles, Los Angeles, USA（儿科，免疫学、过敏与风湿病学部，加州大学洛杉矶分校）；Health Department of Information Services and Solutions, University of California, Los Angeles, University of California, Los Angeles, Los Angeles, USA（信息服务与解决方案健康部门，加州大学洛杉矶分校）；Department of Computational Medicine, University of California Los Angeles, Los Angeles, CA, USA（计算医学系，加州大学洛杉矶分校）
Foresight-England: Development of a National-Scale Generative AI Model of Electronic Health Records for Medical Event Prediction across the COVID-19 Pandemic
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）；University College London Hospitals National Institute for Health Research Biomedical Research Centre（伦敦大学学院医院国家卫生研究院生物医学研究中心）；Interdisciplinary Transformation University（跨学科转型大学）；British Heart Foundation Data Science Centre（英国心脏基金会数据科学中心）；Health Data Research UK（英国健康数据研究中心）；The University of Edinburgh（爱丁堡大学）；University of Cambridge（剑桥大学）；Victor Phillip Dahdaleh Heart and Lung Research Institute, University of Cambridge（剑桥大学维克多·菲利普·达德赫心肺研究所）；British Heart Foundation Centre of Research Excellence, University of Cambridge（剑桥大学英国心脏基金会卓越研究中心）
Representation Learning of Structured Data for Medical Foundation Models
ASUS Intelligent Cloud Services (AICS)（华硕智能云服务（AICS））；Imperial College London（伦敦帝国学院）；University of Manchester（曼彻斯特大学）；Cheng Hsin General Hospital（程新总医院）；Crayon Software（克雷扬软件公司）
