arXiv 2403.165202024-03-25cs.CV

CMViM：用于AD分类的3D多模态表示学习的对比掩码Vim自编码器

CMViM: Contrastive Masked Vim Autoencoder for 3D Multi-modal Representation Learning for AD classification

The Hong Kong Polytechnic University（香港理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Shandong University（山东大学）；Research Institute for Smart Ageing, The Hong Kong Polytechnic University（香港理工大学智能老龄化研究院）

Guangqian Yang, Kangrui Du, Zhihan Yang, Ye Du, Yongping Zheng, Shujun Wang

备注：: 11 pages, 1 figure

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出CMViM框架，结合掩码Vim自编码器与模态内和模态间对比学习，实现3D多模态医学图像的高效表示学习，在AD分类任务上AUC提升2.7%。

摘要：

阿尔茨海默病（AD）是一种无法治愈的神经退行性疾病，会导致认知和功能衰退。由于缺乏治愈方法，及时且准确的AD诊断至关重要，这是一个依赖于多种因素和多模态数据的复杂过程。尽管在将多模态表示学习整合到医学数据集方面已取得成功，但3D医学图像仍鲜受关注。在本文中，我们提出了对比掩码Vim自编码器（CMViM），这是首个为3D多模态数据量身定制的高效表示学习方法。我们提出的框架建立在掩码Vim自编码器之上，以学习3D医学图像中包含的统一多模态表示和长依赖关系。我们还引入了模态内对比学习模块，以增强多模态Vim编码器对同一模态内判别性特征进行建模的能力，并引入了模态间对比学习模块，以缓解模态间表示的未对齐问题。我们的框架包含两个主要步骤：1）将Vision Mamba（Vim）整合到掩码自编码器中，以高效地重建3D掩码多模态数据。2）通过模态内和模态间两方面的对比学习机制来对齐多模态表示。我们的框架在ADNI2数据集上进行了预训练和验证，并在AD分类的下游任务上进行了验证。与其他最先进的方法相比，所提出的CMViM实现了2.7%的AUC性能提升。

英文摘要：

Alzheimer's disease (AD) is an incurable neurodegenerative condition leading to cognitive and functional deterioration. Given the lack of a cure, prompt and precise AD diagnosis is vital, a complex process dependent on multiple factors and multi-modal data. While successful efforts have been made to integrate multi-modal representation learning into medical datasets, scant attention has been given to 3D medical images. In this paper, we propose Contrastive Masked Vim Autoencoder (CMViM), the first efficient representation learning method tailored for 3D multi-modal data. Our proposed framework is built on a masked Vim autoencoder to learn a unified multi-modal representation and long-dependencies contained in 3D medical images. We also introduce an intra-modal contrastive learning module to enhance the capability of the multi-modal Vim encoder for modeling the discriminative features in the same modality, and an inter-modal contrastive learning module to alleviate misaligned representation among modalities. Our framework consists of two main steps: 1) incorporate the Vision Mamba (Vim) into the mask autoencoder to reconstruct 3D masked multi-modal data efficiently. 2) align the multi-modal representations with contrastive learning mechanisms from both intra-modal and inter-modal aspects. Our framework is pre-trained and validated ADNI2 dataset and validated on the downstream task for AD classification. The proposed CMViM yields 2.7\% AUC performance improvement compared with other state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。