FedMM：计算病理学中具有模态异质性的联邦多模态学习
FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology
AI总结：
本文提出FedMM框架，通过联邦训练多个单模态特征提取器解决计算病理学中多模态数据异构与隐私问题，使资源有限的医院也能完成下游分类任务，并在两个公开数据集上显著优于基线方法。
摘要：
互补多模态信息的融合在计算病理学中对于准确诊断至关重要。然而，现有的多模态学习方法需要访问用户的原始数据，带来了重大的隐私风险。虽然联邦学习（FL）作为一种保护隐私的替代方案，但它在应对不同医院之间异构（但可能重叠）模态数据所带来的挑战方面仍显不足。为了弥补这一差距，我们提出了一种联邦多模态（FedMM）学习框架，该框架以联邦方式训练多个单模态特征提取器，以提升后续分类性能，而不是像现有联邦学习那样旨在训练一个统一的多模态融合模型。任何参与的医院，即使只有小规模数据集或有限设备，都可以利用这些联邦训练好的提取器来执行本地下游任务（例如分类），同时确保数据隐私。通过对两个公开可用数据集的全面评估，我们证明FedMM在准确率和AUC指标上显著优于两个基线方法。
英文摘要：
The fusion of complementary multimodal information is crucial in computational pathology for accurate diagnostics. However, existing multimodal learning approaches necessitate access to users' raw data, posing substantial privacy risks. While Federated Learning (FL) serves as a privacy-preserving alternative, it falls short in addressing the challenges posed by heterogeneous (yet possibly overlapped) modalities data across various hospitals. To bridge this gap, we propose a Federated Multi-Modal (FedMM) learning framework that federatedly trains multiple single-modal feature extractors to enhance subsequent classification performance instead of existing FL that aims to train a unified multimodal fusion model. Any participating hospital, even with small-scale datasets or limited devices, can leverage these federated trained extractors to perform local downstream tasks (e.g., classification) while ensuring data privacy. Through comprehensive evaluations of two publicly available datasets, we demonstrate that FedMM notably outperforms two baselines in accuracy and AUC metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
