FedEPA：增强多模态联邦学习中的个性化与模态对齐
FedEPA: Enhancing Personalization and Modality Alignment in Multimodal Federated Learning
AI总结：
针对联邦学习中客户端多模态数据及标签不足的问题，提出FedEPA框架，通过个性化本地模型聚合和无监督模态对齐策略，在有限标签下提升多模态分类性能。
摘要：
联邦学习（FL）支持多方跨设备的去中心化模型训练，同时保护隐私。然而，大多数FL系统假设客户端仅持有单模态数据，这限制了其实际应用场景，因为机构通常拥有多模态数据。此外，缺乏标签数据进一步制约了大多数FL方法的性能。在这项工作中，我们提出FedEPA，一种面向多模态学习的新型FL框架。FedEPA采用个性化本地模型聚合策略，利用客户端的标签数据学习个性化聚合权重，从而缓解数据异质性的影响。我们还提出一种无监督模态对齐策略，在有限标签数据下有效工作。具体而言，我们将多模态特征分解为对齐特征和上下文特征，然后采用对比学习来对齐跨模态的对齐特征，确保每个模态内对齐特征与上下文特征的独立性，并促进上下文特征的多样性。我们引入了一种多模态特征融合策略以获取联合嵌入。实验结果表明，在有限标签数据条件下，FedEPA在多模态分类任务中显著优于现有FL方法。
英文摘要：
Federated Learning (FL) enables decentralized model training across multiple parties while preserving privacy. However, most FL systems assume clients hold only unimodal data, limiting their real-world applicability, as institutions often possess multimodal data. Moreover, the lack of labeled data further constrains the performance of most FL methods. In this work, we propose FedEPA, a novel FL framework for multimodal learning. FedEPA employs a personalized local model aggregation strategy that leverages labeled data on clients to learn personalized aggregation weights, thereby alleviating the impact of data heterogeneity. We also propose an unsupervised modality alignment strategy that works effectively with limited labeled data. Specifically, we decompose multimodal features into aligned features and context features. We then employ contrastive learning to align the aligned features across modalities, ensure the independence between aligned features and context features within each modality, and promote the diversity of context features. A multimodal feature fusion strategy is introduced to obtain a joint embedding. The experimental results show that FedEPA significantly outperforms existing FL methods in multimodal classification tasks under limited labeled data conditions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
