arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.01899cs.LGcs.AIcs.HC
收藏

多模态基础模型用于早期疾病检测

Multimodal Foundation Models for Early Disease Detection

The University of Tulsa(塔尔萨大学)

Md Talha Mohsin, Ismail Abdulrashid

AI总结:

本文提出一种多模态基础模型,通过整合电子病历、医学影像、基因组学和可穿戴传感器数据,提升早期疾病检测的准确性和鲁棒性。

摘要:

医疗数据现在涵盖了电子病历、医学影像、基因组学和可穿戴传感器,但大多数诊断模型仍单独处理这些模态。这限制了它们捕捉早期跨模态疾病特征的能力。本文介绍了一种基于变压器架构的多模态基础模型,通过模态特定编码器和跨模态注意力机制整合异构临床数据。每个模态被映射到共享的潜在空间,并通过多头注意力与残差归一化进行融合。我们使用一个模拟早期疾病模式的多模态数据集实现该框架,包括电子病历序列、影像片段、基因组资料和可穿戴信号,以及缺失模态场景和标签噪声。模型通过监督分类以及自监督重建和对比对齐进行训练,以提高鲁棒性。实验评估显示在早期检测设置中表现强劲,具有稳定的分类指标、可靠的不确定性估计和可解释的注意力模式。该方法朝着灵活的、预训练和微调的基础模型方向发展,支持精确诊断,处理不完整输入,并在肿瘤学、心脏病学和神经学应用中提高早期疾病检测。

英文摘要:

Healthcare data now span EHRs, medical imaging, genomics, and wearable sensors, but most diagnostic models still process these modalities in isolation. This limits their ability to capture early, cross-modal disease signatures. This paper introduces a multimodal foundation model built on a transformer architecture that integrates heterogeneous clinical data through modality-specific encoders and cross-modal attention. Each modality is mapped into a shared latent space and fused using multi-head attention with residual normalization. We implement the framework using a multimodal dataset that simulates early-stage disease patterns across EHR sequences, imaging patches, genomic profiles, and wearable signals, including missing-modality scenarios and label noise. The model is trained using supervised classification together with self-supervised reconstruction and contrastive alignment to improve robustness. Experimental evaluation demonstrates strong performance in early-detection settings, with stable classification metrics, reliable uncertainty estimates, and interpretable attention patterns. The approach moves toward a flexible, pretrain-and-fine-tune foundation model that supports precision diagnostics, handles incomplete inputs, and improves early disease detection across oncology, cardiology, and neurology applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑