发表机构
Houston Methodist; University of Wisconsin–Madison; Walmart（休斯顿卫理公会医院; 威斯康星大学麦迪逊分校; 沃尔玛）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对临床多模态数据缺失问题，提出GLR-MM框架，利用图注意力与局部跨模态重建缺失嵌入，在严重缺失（50%）时优于现有方法，提升ICU死亡率预测性能。
AI 中文摘要
临床多模态模型通常必须在所有胸部X光（CXR）和电子健康记录（EHR）输入可用之前进行预测。现有方法对齐观测到的表示、对缺失性建模或跨模态重建，但未联合利用患者内部和临床相似的患者间证据。我们提出GLR-MM，一种基于图的全局-局部重建框架，用于早期ICU死亡率预测。它将五种CXR-EHR模态映射到共享空间，通过互补的局部跨模态和全局图注意力分支重建缺失嵌入，自适应融合其估计，并优化类别平衡的预测、重建和对比目标。在9,620个MIMIC衍生的ICU住院记录上，我们使用共享确定性掩码评估了10%、30%和50%的随机模态缺失。MUSE在轻度和中度缺失下表现更好，而GLR-MM在50%缺失下分别实现了更高的AUROC和AUPRC，分别提高了0.0088和0.0249。这些结果表明，当输入严重不完整时，图引导的重建最为有用。
英文摘要
Clinical multimodal models must often predict before all chest X-ray (CXR) and electronic health record (EHR) inputs are available. Existing approaches align observed representations, model missingness, or reconstruct across modalities, but do not jointly exploit within-patient and clinically similar inter-patient evidence. We propose GLR-MM, a Graph-Based Global-Local Reconstruction framework for early ICU mortality prediction. It maps five CXR-EHR modalities to a shared space, reconstructs missing embeddings through complementary local cross-modal and global graph-attention branches, adaptively fuses their estimates, and optimizes class-balanced prediction, reconstruction, and contrastive objectives. On 9,620 MIMIC-derived ICU stays, we evaluate 10%, 30%, and 50% random modality missingness with shared deterministic masks. MUSE performs better under mild and moderate missingness, whereas GLR-MM achieves higher AUROC and AUPRC at 50% by 0.0088 and 0.0249, respectively. These results indicate that graph-guided reconstruction is most useful when inputs are severely incomplete.
CommentsAccepted in MICCAI
