Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.
CarDS-Plus ECG Platform: Development and Feasibility Evaluation of a Multiplatform Artificial Intelligence Toolkit for Portable and Wearable Device Electrocardiograms
Yale School of Medicine(耶鲁医学院);Yale New Haven Hospital(耶鲁-纽黑文医院);Yale School of Public Health(耶鲁公共卫生学院)
MPIB: A Benchmark for Medical Prompt Injection Attacks and Clinical Safety in LLMs
Seoul National University College of Medicine(首尔国立大学医学院);Seoul National University(首尔国立大学);Seoul National University College of Medicine, Seoul National University Hospital(首尔国立大学医学院、首尔国立大学医院)
Enhancing LLMs' Clinical Reasoning with Real-World Data from a Nationwide Sepsis Registry
Korea Advanced Institute of Science and Technology(韩国科学技术院);Microsoft(微软);Asan Medical Center, University of Ulsan College of Medicine(釜山大学医学院阿桑医疗中心);Samsung Medical Center, Sungkyunkwan University School of Medicine(成均馆大学医学院三星医疗中心);Seoul National University Bundang Hospital, Seoul National University College of Medicine(首尔国立大学医学院首尔国立大学医院)
National University of Singapore(新加坡国立大学);SAP Asia Pte Ltd(SAP亚洲私人有限公司);Nanyang Technological University(南洋理工大学);Singapore Institute of Technology(新加坡理工大学)
AutoMedic: An Automated Evaluation Framework for Clinical Conversational Agents with Medical Dataset Grounding
Yonsei University College of Medicine(延世大学医学院);Yonsei Institute for Digital Health(延世大学数字健康研究所);Yonsei University(延世大学);Institute of Behavioral Sciences in Medicine(医学行为科学研究所)
Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology
Microsoft Research(微软研究院);Providence Health & Services(普罗维登斯医疗服务机构);Earle A. Chiles Research Institute(厄尔·A·奇尔斯研究所);Providence Cancer Institute(普罗维登斯癌症研究所)
Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning
Beijing University of Posts and Telecommunications(北京邮电大学);Boston University(波士顿大学);University of Southern California(南加州大学);GDIIST;Renyixun Health Technology Co., Ltd.(仁心迅健康科技有限公司)
PREBA: Surgical Duration Prediction via PCA-Weighted Retrieval-Augmented LLMs and Bayesian Averaging Aggregation
School of Information Science and Engineering, Yunnan University(云南大学信息科学与工程学院);School of Computer Science, Wuhan University(武汉大学计算机学院);Lee Kong Chian School of Medicine, Nanyang Technological University(南洋理工大学李科钦医学院);First People’s Hospital of Yunnan Province(云南省第一人民医院)
DALL-M: Context-Aware Clinical Data Augmentation with LLMs
School of Information Systems, Queensland University of Technology(昆士兰科技大学信息系统学院);School of Computer Science, Queensland University of Technology(昆士兰科技大学计算机科学学院);Data Science Institute, University of Technology Sydney(悉尼科技大学数据科学研究所);Imagiology Department, Grupo Lusiadas(卢西塔达斯集团影像学系)
Reconstructing Sepsis Trajectories from Clinical Case Reports using LLMs: the Textual Time Series Corpus for Sepsis
Machine Learning Department and Heinz College Carnegie Mellon University(卡内基梅隆大学机器学习系和海恩兹学院);National Library of Medicine National Institutes of Health(美国国家医学图书馆)
ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks
School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院);Centre for Medical Informatics, The University of Edinburgh(爱丁堡大学医学信息学中心);Health Data Research UK(英国健康数据研究所);School of Computer Science, Peking University(北京大学计算机学院);National Engineering Research Center for Software Engineering, Peking University(北京大学软件工程国家工程研究中心);ETH Zurich(苏黎世联邦理工学院);School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院)