arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2311.14902cs.CV
收藏

使用对比图跨视图学习结合SPECT图像与临床特征多模态融合的帕金森病分类

Parkinson's Disease Classification Using Contrastive Graph Cross-View Learning with Multimodal Fusion of SPECT Images and Clinical Features

School of Systems and Enterprises, Stevens Institute of Technology(史蒂文斯理工大学系统与企业学院);Kaohsiung Chang Gung Memorial Hospital(高雄长庚纪念医院)

Jun-En Ding, Chien-Chin Hsu, Feng Liu

AI总结:

提出结合SPECT图像与临床特征的多模态对比跨视图图融合方法,通过共注意力模块与对比损失提升特征提取与融合,在帕金森病分类中实现0.91准确率和0.93 AUC,优于传统机器学习方法。

摘要:

帕金森病(PD)影响全球数百万人,损害运动能力。先前研究利用深度学习进行PD预测,主要关注医学图像,忽略了数据潜在的流形结构。本研究提出一种多模态方法,包含图像和非图像特征,利用对比跨视图图融合进行PD分类。我们引入了一个新颖的多模态共注意力模块,整合来自图像和临床特征低维表示的独立图视图的嵌入。这实现了更鲁棒和结构化的特征提取,以改进多视图数据分析。此外,设计了一种简化的基于对比损失的融合方法,以增强跨视图融合学习。我们的图视图多模态方法在五折交叉验证中实现了0.91的准确率和0.93的受试者工作特征曲线下面积(AUC)。与仅基于机器学习的方法相比,它在非图像数据上也展现出更优的预测能力。

英文摘要:

Parkinson's Disease (PD) affects millions globally, impacting movement. Prior research utilized deep learning for PD prediction, primarily focusing on medical images, neglecting the data's underlying manifold structure. This work proposes a multimodal approach encompassing both image and non-image features, leveraging contrastive cross-view graph fusion for PD classification. We introduce a novel multimodal co-attention module, integrating embeddings from separate graph views derived from low-dimensional representations of images and clinical features. This enables more robust and structured feature extraction for improved multi-view data analysis. Additionally, a simplified contrastive loss-based fusion method is devised to enhance cross-view fusion learning. Our graph-view multimodal approach achieves an accuracy of 0.91 and an area under the receiver operating characteristic curve (AUC) of 0.93 in five-fold cross-validation. It also demonstrates superior predictive capabilities on non-image data compared to solely machine learning-based methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑