FedMFS：具有选择性模态通信的联邦多模态融合学习
FedMFS: Federated Multimodal Fusion Learning with Selective Modality Communication
AI总结：
针对多模态联邦学习中设备模态多样性与通信限制的挑战，提出FedMFS方法，通过基于Shapley值和模型大小的模态选择标准，在保持准确性的同时大幅降低通信开销。
摘要：
多模态联邦学习（FL）旨在丰富联邦学习设置中的模型训练，其中设备正在跨多种模态（例如，测量压力、运动和其他类型数据的传感器）收集测量数据。然而，多模态联邦学习的关键挑战仍未得到解决，特别是在异构网络设置中：（i）每个设备收集的模态集合将是多样的，并且（ii）通信限制阻止设备将其所有本地训练的模态模型上传到服务器。在本文中，我们提出了具有选择性模态通信的联邦多模态融合学习（FedMFS），一种新的多模态融合联邦学习方法论，可以解决上述挑战。关键思想是为每个设备引入模态选择标准，该标准权衡（i）模态的影响，通过Shapley值分析衡量，与（ii）模态模型大小作为通信开销的衡量标准。这使得FedMFS能够根据资源约束和应用需求灵活地平衡性能与通信成本。在真实世界的ActionSense数据集上的实验表明，FedMFS能够实现与几种基线相当的准确性，同时将通信开销减少超过4倍。
英文摘要：
Multimodal federated learning (FL) aims to enrich model training in FL settings where devices are collecting measurements across multiple modalities (e.g., sensors measuring pressure, motion, and other types of data). However, key challenges to multimodal FL remain unaddressed, particularly in heterogeneous network settings: (i) the set of modalities collected by each device will be diverse, and (ii) communication limitations prevent devices from uploading all their locally trained modality models to the server. In this paper, we propose Federated Multimodal Fusion learning with Selective modality communication (FedMFS), a new multimodal fusion FL methodology that can tackle the above mentioned challenges. The key idea is the introduction of a modality selection criterion for each device, which weighs (i) the impact of the modality, gauged by Shapley value analysis, against (ii) the modality model size as a gauge for communication overhead. This enables FedMFS to flexibly balance performance against communication costs, depending on resource constraints and application requirements. Experiments on the real-world ActionSense dataset demonstrate the ability of FedMFS to achieve comparable accuracy to several baselines while reducing the communication overhead by over 4x.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
