发表机构
Texas State University; University of Technology and Applied Sciences（德克萨斯州立大学; 科技大学与应用科学大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对rs-fMRI多模态异质性和文本缺失问题，提出文本锚定多模态对比学习框架fMRI-TAMCL，整合图像、稀疏FC和生成文本，在五个数据集上以78.6%-86.4%准确率优于29个基线。
AI 中文摘要
静息态功能磁共振成像（rs-fMRI）在脑疾病分类中具有重要意义，但其高度多模态且表现出强烈的多中心异质性。现有方法融合图像、基于BOLD的功能连接和表型数据模态。与其他医学影像数据集不同，rs-fMRI数据集很少包含文本模态，因此文本通常由表型数据或BOLD激活生成。这些文本生成方法依赖于对受试者、中心、设备和协议的固定假设，导致跨数据集泛化能力差。我们提出fMRI-TAMCL，一种文本锚定的多模态对比学习框架，整合fMRI图像、稀疏功能连接（FC）和生成的受试者特定文本。其受试者自适应阈值推导模块生成BOLD激活文本，而特征值序列化模块生成表型文本。所有三种模态被编码为聚类图，投影到共享的单位超球面空间，通过成对、文本锚定的监督对比学习进行对齐，并使用注意力机制进行融合。fMRI-TAMCL在五个数据集上证明了其泛化能力，在下游分类中以78.6%-86.4%的准确率优于29个基线方法。
英文摘要
Resting-state fMRI is important in the classification of brain disorders, but highly multimodal and exhibits strong multisite heterogeneity. Existing methods fuse images, BOLD-based functional connectivity, and phenotypic data modalities. Unlike other medical imaging datasets, rs-fMRI datasets rarely include a text modality, so they are generated from phenotypic data or BOLD activations. These text generation methods rely on fixed assumptions for subjects, sites, devices, and protocols, leading to poor generalization across datasets. We propose fMRI-TAMCL, a text-anchored multimodal contrastive learning framework that integrates fMRI images, sparse FC, and generated subject-specific text. Its Subject-Adaptive Threshold Derivation module generates BOLD activation text, while Feature-Value Serialization module generates phenotypic text. All three modalities are encoded as clustered graphs, projected onto a shared unit hypersphere space, aligned using pairwise, text-anchored supervised contrastive learning, and fused with attention. fMRI-TAMCL proves its generalization capability across five datasets outperforming 29 baselines with 78.6%-86.4% accuracy in downstream classification.
Comments10 pages, 6 figures
