LLM4EHR：通过大语言模型将临床时间序列与医疗事件序列对齐

中文摘要 AI 辅助

临床机器学习中关于重症监护室结果预测的研究已从定制监督模型转向基础模型。现有方法利用电子健康记录（EHR）训练临床基础模型，但未充分探索临床事件与EHR中记录的时间序列（TS）观察之间的共享时间结构。为此提出LLM4EHR，结合领域适应大语言模型与变压器TS编码器，通过时间对齐EHR事件和TS进行预训练，提出正则化对比目标学习鲁棒表示。消融研究支持下，其在下游临床任务中表现良好且能学习可转移临床TS嵌入。这些发现朝着构建更通用和高性能临床基础模型迈进。

英文摘要

Recent research in clinical machine learning, focusing on outcome predictions in intensive care unit (ICU), has shifted from bespoke supervised models to foundation models, utilising modern representation learning methods. Here, foundation models are pre-trained on mixtures of complex clinical data modalities, useful for various downstream tasks. Existing works often utilise Electronic Health Records (EHR) to provide rich and diverse patient observations to train clinical foundation models. However, existing methods do not sufficiently explore the shared temporal structures between clinical events and time series (TS) observations recorded in EHRs. This limitation potentially leads to less robust and adaptive clinical foundation models, resulting in reduced performance on downstream tasks. To fully exploit this temporal structure, we propose LLM4EHR, a new clinical foundation model trained on ICU EHR data. Combining domain adapted large language models with a transformer TS encoder, we pre-trained LLM4EHR by temporally aligning the EHR events and TS. For this, we propose a regularised contrastive objective to learn robust EHR TS representations conditioned on EHR event embeddings produced by the domain adapted LLM. Supported by an ablation study, we find that learnt EHR TS embeddings from LLM4EHR improve performance on various downstream clinical tasks with competitive performance. Further, we empirically demonstrate that LLM4EHR learns transferable clinical TS embeddings that can be deployed to new cohorts via k-shot adaptation. These findings provide a step towards building more generalisable and performant clinical foundation models.

发表机构

• UK Dementia Research Institute（英国痴呆症研究所）

机构由 AI 辅助整理，请以论文原文为准。