发表机构
Lawrence Livermore National Laboratory; Kaiser Permanente（劳伦斯利弗莫尔国家实验室; 凯撒医疗集团）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
EHRAdapt提出一种适配器，将电子健康记录元组直接映射到冻结语言模型嵌入空间，通过语义先验与证据残差之和表示事件向量，在罕见临床事件预测和下游分类任务上优于基线。
AI 中文摘要
电子健康记录（EHR）将临床历史编码为（时间、模态、代码）元组，而预训练语言模型期望的是文本标记。将它们序列化为文本会膨胀序列长度并冗余编码结构。我们提出EHRAdapt，一个将元组直接映射到冻结语言模型嵌入空间的适配器。模态获得一个学习到的嵌入，时间间隔通过学习到的注意力偏置进入，事件代码获得专用向量。学习事件向量是核心挑战：临床词汇呈长尾分布，导致罕见事件观测太少而无法可靠估计。因此，EHRAdapt将每个事件向量表示为语义先验与证据残差之和。先验是来自在临床本体上训练的生物医学语言模型的事件临床描述的冻结嵌入，通过共享的学习投影映射到模型的输入空间，从而在观测稀缺时也能提供临床意义。残差是学习到的低秩事件特定修正，随着证据积累而细化它。我们在约400万患者的记录上使用三个冻结的LLM骨干（OLMo2 1B、Llama3.2 1B和OLMo2 7B）进行持续预训练，仅训练适配器（占全部参数的0.1%–0.6%）。在留出的下一事件预测中，完整适配器在每个骨干上均优于所有消融。移除语义通路对罕见事件的伤害是最频繁事件的十倍以上，而移除残差损害整体预测但改善最罕见事件的预测。在可报告的传染病和症候群下游分类任务上，EHRAdapt优于基于文本的LLM和基于计数的基线，且两条通路均改善罕见疾病判别。因此，两条通路发挥互补作用，这仅在按事件频率而非平均值分解结果时才可见。
英文摘要
Electronic health records (EHRs) encode clinical histories as (time, modality, code) tuples, whereas pretrained language models expect text tokens. Serializing them as text inflates sequence length and redundantly encodes structure. We introduce EHRAdapt, an adapter that maps tuples directly into a frozen language model's embedding space. Modality receives a learned embedding, time gaps enter through learned attention biases, and event codes receive dedicated vectors. Learning event vectors is the central challenge: clinical vocabularies are long-tailed, leaving rare events too few observations for reliable estimates. EHRAdapt therefore represents each event vector as the sum of a semantic prior and an evidence residual. The prior is a frozen embedding of the event's clinical description from a biomedical language model trained on clinical ontologies, mapped into the model's input space by a shared learned projection, so it supplies clinical meaning even when observations are scarce. The residual, a learned low-rank event-specific correction, refines it as evidence accumulates. We run continued pretraining on about 4 million patients' records with three frozen LLM backbones (OLMo2 1B, Llama3.2 1B, and OLMo2 7B), training only the adapter (0.1--0.6% of all parameters). The full adapter outperforms all ablations in held-out next-event prediction on every backbone. Removing the semantic pathway hurts rare events over ten times more than the most frequent ones, whereas removing the residual hurts overall prediction but improves it for the rarest events. On reportable infectious-disease and syndromic downstream classification tasks, EHRAdapt outperforms text-based LLM and count-based baselines, and both pathways improve rare-disease discrimination. The two pathways therefore play complementary roles, visible only when results are broken down by event frequency rather than averaged.
