发表机构
University of Oxford（牛津大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究揭示多模态临床预测中视觉-语言模型对心电图的利用不足（ECG Mirage），并提出视觉提示调优方法，有效提升ICU入住与恶化预测性能。
AI 中文摘要
急诊科（ED）的决策依赖于异构的临床信息，包括患者病史、生命体征、实验室结果和心电图（ECGs）。视觉-语言模型（VLMs）能够联合处理这些模态，但强大的预测性能并不一定意味着对正确患者心电图的实质性利用。我们将这种失败模式称为“ECG Mirage”：表面上具备多模态能力，但实际上并未有效依赖患者特定的心电图信息。我们区分了两种形式：心电图忽视（ECG neglect），即心电图提供的预测收益甚微；以及心电图混淆（ECG confusion），即匹配的心电图优于无图像输入，但不优于不匹配的心电图。为评估这些行为，我们在保持临床文本和预测目标固定的情况下，比较了使用匹配心电图、结果不一致的不匹配心电图以及无图像输入所获得的预测结果。在MDS-ED数据集上的四种视觉-语言模型中，匹配的心电图在ICU入住预测和临床恶化预测方面均未提供一致的优势。随后，我们使用监督学习结合条件直接偏好优化训练了四个受限视觉提示，同时冻结VLM骨干网络。所得模型在ICU入住预测上达到70.6%的平衡准确率，在恶化预测上达到67.5%，并将匹配与不匹配之间的性能差距分别提升至约16.5和5.5个百分点。总体而言，我们的研究识别了多模态临床预测中的ECG Mirage，并引入视觉提示调优作为一种有效的缓解策略。
英文摘要
Emergency department (ED) decision-making relies on heterogeneous clinical information, including patient history, vital signs, laboratory results, and electrocardiograms (ECGs). Vision--language models (VLMs) can jointly process these modalities, but strong predictive performance does not necessarily imply meaningful use of the correct patient's ECG. We term this failure mode ECG Mirage: apparent multimodal capability without useful dependence on patient-specific ECG information. We distinguish two forms: ECG neglect, where ECGs provide little predictive benefit, and ECG confusion, where matched ECGs outperform no-image inputs but not mismatched ECGs. To evaluate these behaviours, we compare predictions obtained with matched ECGs, outcome-discordant mismatched ECGs, and no-image inputs while holding the clinical text and prediction targets fixed. Across four VLMs on MDS-ED, matched ECGs provide no consistent advantage for either ICU admission or clinical deterioration prediction. We then train four restricted visual prompts using supervised learning followed by conditional direct preference optimisation, while keeping the VLM backbone frozen. The resulting models achieve balanced accuracies of 70.6% for ICU admission and 67.5% for deterioration and increase the matched-versus-mismatched performance gap to approximately 16.5 and 5.5 percentage points, respectively. Overall, our study identifies ECG Mirage in multimodal clinical prediction and introduces visual prompt tuning as an efficient mitigation strategy.
