MMAP：用于纵向阿尔茨海默病预测的多模态缺失感知预训练
MMAP: Multimodal Missing-Aware Pretraining for Longitudinal Alzheimer's Prediction
浏览论文内容
中文总结 AI 辅助
针对医学数据中模态缺失和不完整表格数据问题，提出MMAP多模态缺失感知预训练方法，通过sigmoid对比学习与生成式重建及缺失令牌生成器学习图像-表格表征，在阿尔茨海默病纵向预测任务上优于强基线。
中文摘要 AI 辅助
临床决策在很大程度上依赖于通过理解患者健康状况来预测疾病进展轨迹，而患者的健康状况由多模态医学数据表征。人工智能在从多模态医学数据中学习有用表征以预测疾病进展并辅助临床决策方面具有巨大潜力。然而，预测性人工智能模型的发展受到医学数据集中频繁出现的模态缺失和不完整表格数据的制约。此外，仅靠疾病标签可能只能为从高维多模态数据中学习表征提供有限的监督信号。在此，我们提出MMAP，一种新颖的多模态缺失感知对齐预训练方法，用于从不完整数据中学习图像-表格表征。图像编码器通过高效的sigmoid对比学习结合生成式重建进行预训练。表格编码器基于表格基础模型构建。缺失令牌生成器使两个编码器能够将不完整数据作为输入，从而使模型对模态缺失具有鲁棒性，无论是图像缺失还是表格数据缺失。我们在阿尔茨海默病的两个具有挑战性的纵向临床任务上评估了所学多模态表征的临床实用性：预测疾病阶段转换和预测淀粉样蛋白状态。所提出的方法优于强大的多模态和单模态基线。
英文摘要
Clinical decision making heavily relies on predicting the disease progression trajectory by seeking to understand patient's health status which is characterised by multimodal medical data. AI holds great potential for learning useful representations from multimodal medical data to predict disease progression and aid clinical decision making. However, development of predictive AI models is constrained by missing modalities and incomplete tabular data frequently occurring in medical datasets. In addition, disease labels alone may only provide limited supervisory signals for learning representations from high-dimensional multimodal data. Here, we present MMAP, a novel Multimodal Missing-aware Alignment Pretraining method for learning image-tabular representations from incomplete data. An image encoder is pretrained with efficient sigmoid contrastive learning combined with generative reconstruction. A tabular encoder is built upon a tabular foundation model. A missing token generator enables the two encoders to take incomplete data as input, enabling the model to be robust against missing modalities, either with missing images or missing tabular data. We evaluate the clinical usefulness of the learnt multimodal representations on two challenging longitudinal clinical tasks for Alzheimer's disease: predicting disease stage conversion and predicting amyloid status. The proposed method outperforms strong multimodal and unimodal baselines.
发表机构
- Imperial College London（伦敦帝国理工学院）
- FAU Erlangen-Nürnberg（埃尔朗根-纽伦堡大学）
- Rosalind Franklin Institute（罗莎琳德·富兰克林研究所）
机构由 AI 辅助整理，请以论文原文为准。
