arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.14653cs.LGcs.AIcs.DCcs.MA
收藏

联邦域泛化中的数据无关服务器端匹配梯度方法

Federated Domain Generalization with Data-free On-server Matching Gradient

Pusan National University(釜山国立大学);Trinity College Dublin(都柏林三一学院)

Trong-Binh Nguyen, Minh-Duong Nguyen, Jinsun Park, Quoc-Viet Pham, Won Joo Hwang

AI总结:

提出FedOMG方法,通过服务器端梯度内积最大化利用分布式域信息,实现联邦域泛化,无需额外通信成本,在多个基准数据集上超越现有SOTA。

摘要:

域泛化(DG)旨在从多个已知源域中学习一个能够良好泛化到未知目标域的模型。DG中的关键方法之一是训练一个生成域不变表示的编码器。然而,这种方法不适用于联邦域泛化(FDG),因为在FDG中,来自不同域的数据分布在不同的客户端上。在本文中,我们提出了一种新颖的方法,称为基于服务器端匹配梯度的联邦学习(FedOMG),它能够高效地利用来自分布式域的域信息。具体来说,我们利用局部梯度作为分布式模型的信息,通过梯度内积最大化来找到跨所有域的梯度不变方向。其优点有两个方面:1)FedOMG可以在集中式服务器上聚合分布式模型的特性,而无需产生任何额外的通信成本;2)FedOMG与许多现有的联邦学习(FL)/联邦域泛化(FDG)方法正交,可以通过与它们无缝集成来进一步提升性能。我们在各种设置下进行了广泛的实验评估,以证明FedOMG相对于其他FL/FDG基线方法的稳健性。我们的方法在四个FL基准数据集(MNIST、EMNIST、CIFAR-10和CIFAR-100)以及三个FDG基准数据集(PACS、VLCS和OfficeHome)上均优于最新的SOTA基线方法。

英文摘要:

Domain Generalization (DG) aims to learn from multiple known source domains a model that can generalize well to unknown target domains. One of the key approaches in DG is training an encoder which generates domain-invariant representations. However, this approach is not applicable in Federated Domain Generalization (FDG), where data from various domains are distributed across different clients. In this paper, we introduce a novel approach, dubbed Federated Learning via On-server Matching Gradient (FedOMG), which can \emph{efficiently leverage domain information from distributed domains}. Specifically, we utilize the local gradients as information about the distributed models to find an invariant gradient direction across all domains through gradient inner product maximization. The advantages are two-fold: 1) FedOMG can aggregate the characteristics of distributed models on the centralized server without incurring any additional communication cost, and 2) FedOMG is orthogonal to many existing FL/FDG methods, allowing for additional performance improvements by being seamlessly integrated with them. Extensive experimental evaluations on various settings to demonstrate the robustness of FedOMG compared to other FL/FDG baselines. Our method outperforms recent SOTA baselines on four FL benchmark datasets (MNIST, EMNIST, CIFAR-10, and CIFAR-100), and three FDG benchmark datasets (PACS, VLCS, and OfficeHome).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑