面向不完整数据阿尔茨海默病分类的级联多模态混合Transformer
Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data
AI总结：
针对阿尔茨海默病分类中多模态数据缺失导致数据利用率低、模型性能受限的问题，提出带交叉注意力与模态丢弃机制的3MT模型，可处理任意数量模态与缺失数据，在ADNI、AIBL数据集上取得优异表现。
摘要：
精准的医学分类需要大量多模态数据，且在很多场景下需要不同类型的特征。已有研究表明，使用多模态数据在阿尔茨海默病（Alzheimer's Disease, AD）等疾病分类任务中效果优于单模态模型，取得了可观进展。但这类模型通常灵活性不足，无法处理模态缺失的情况。目前最常见的应对方案是丢弃存在模态缺失的样本，这会导致数据利用率大幅下降。再加上标注医学图像本就稀缺的现状，深度学习这类数据驱动方法的性能会受到严重制约。因此，亟需一种能够适配各类临床场景下缺失数据的多模态方法。\n本文提出了多模态混合Transformer（Multi-Modal Mixing Transformer, 3MT），这是一种疾病分类Transformer模型，既能够利用多模态数据，也可以应对数据缺失场景。本研究使用临床数据与神经影像数据，测试3MT在AD与认知正常（Cognitively normal, CN）分类、轻度认知障碍（mild cognitive impairment, MCI）向进展性MCI（progressive MCI, pMCI）或稳定型MCI（stable MCI, sMCI）的转化预测任务上的表现。该模型采用新颖的级联模态Transformer架构，通过交叉注意力融合多模态信息，以实现更可靠的预测。我们提出了一种全新的模态丢弃机制，以实现前所未有的模态独立性与鲁棒性，从而适配数据缺失场景。最终得到的通用网络能够融合任意数量、不同特征类型的模态，同时在数据缺失场景下保证数据的充分利用。该模型在ADNI数据集上进行训练与评估，性能达到SOTA，还在存在数据缺失的AIBL数据集上开展了进一步验证。
英文摘要：
Accurate medical classification requires a large number of multi-modal data, and in many cases, different feature types. Previous studies have shown promising results when using multi-modal data, outperforming single-modality models when classifying diseases such as Alzheimer's Disease (AD). However, those models are usually not flexible enough to handle missing modalities. Currently, the most common workaround is discarding samples with missing modalities which leads to considerable data under-utilization. Adding to the fact that labeled medical images are already scarce, the performance of data-driven methods like deep learning can be severely hampered. Therefore, a multi-modal method that can handle missing data in various clinical settings is highly desirable. In this paper, we present Multi-Modal Mixing Transformer (3MAT), a disease classification transformer that not only leverages multi-modal data but also handles missing data scenarios. In this work, we test 3MT for AD and Cognitively normal (CN) classification and mild cognitive impairment (MCI) conversion prediction to progressive MCI (pMCI) or stable MCI (sMCI) using clinical and neuroimaging data. The model uses a novel Cascaded Modality Transformer architecture with cross-attention to incorporate multi-modal information for more informed predictions. We propose a novel modality dropout mechanism to ensure an unprecedented level of modality independence and robustness to handle missing data scenarios. The result is a versatile network that enables the mixing of arbitrary numbers of modalities with different feature types and also ensures full data utilization missing data scenarios. The model is trained and evaluated on the ADNI dataset with the SOTRA performance and further evaluated with the AIBL dataset with missing data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
