arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.11635cs.LG
收藏

联邦领域泛化的特征分布匹配

Feature Distribution Matching for Federated Domain Generalization

University of Tokyo(东京大学);RIKEN AIP(理化学研究所先进智能项目中心);United Nations University(联合国大学)

Yuwei Sun, Ng Chong, Hideya Ochiai

摘要:

多源领域自适应已被广泛研究。特定领域固有的特征的分布偏移导致了负转移问题,降低了模型对看不见的任务的通用性。在联邦学习(FL)中,共享学习的模型参数以训练全局模型,该全局模型利用在单独的数据域上训练的客户端模型之间的基础知识。然而,FL的数据机密性阻碍了传统领域自适应方法的有效性,这些方法需要不同领域数据的先验知识。提出了一种新的联邦领域泛化方法-联邦知识对齐(Federated Knowledge Alignment,FedKA)。FedKA利用全局工作空间中的特征分布匹配,使得全局模型可以在未知客户端数据的约束下学习域不变的客户端特征。FedKA采用联邦投票机制,根据客户端的共识生成目标域伪标签,以促进全局模型的微调。我们进行了广泛的实验,包括消融研究,以评估所提出的方法在使用不同的模型架构的图像和文本分类任务的有效性。实证结果表明,FedKA在Digit-Five和Digi-Caltech 10中分别实现了8.8%和3.5%的性能提升,在训练数据极其有限的情况下,在Amazon Review中实现了0.7%的性能提升。此外,我们研究了FedKA在减轻负迁移的外语的基础上的一个新的标准称为群体效应的有效性。结果表明,FedKA可以减少负迁移,通过模型聚合将性能增益提高4倍。

英文摘要:

Multi-source domain adaptation has been intensively studied. The distribution shift in features inherent to specific domains causes the negative transfer problem, degrading a model's generality to unseen tasks. In Federated Learning (FL), learned model parameters are shared to train a global model that leverages the underlying knowledge across client models trained on separate data domains. Nonetheless, the data confidentiality of FL hinders the effectiveness of traditional domain adaptation methods that require prior knowledge of different domain data. We propose a new federated domain generalization method called Federated Knowledge Alignment (FedKA). FedKA leverages feature distribution matching in a global workspace such that the global model can learn domain-invariant client features under the constraint of unknown client data. FedKA employs a federated voting mechanism that generates target domain pseudo-labels based on the consensus from clients to facilitate global model fine-tuning. We performed extensive experiments, including an ablation study, to evaluate the effectiveness of the proposed method in both image and text classification tasks using different model architectures. The empirical results show that FedKA achieves performance gains of 8.8% and 3.5% in Digit-Five and Office-Caltech10, respectively, and a gain of 0.7% in Amazon Review with extremely limited training data. Moreover, we studied the effectiveness of FedKA in alleviating the negative transfer of FL based on a new criterion called Group Effect. The results show that FedKA can reduce negative transfer, improving the performance gain via model aggregation by 4 times.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑