FedDiv:针对带噪声标签联邦学习的协作式噪声过滤
FedDiv: Collaborative Noise Filtering for Federated Learning with Noisy Labels
AI总结:
针对带噪声标签联邦学习(F-LNL)中噪声过滤性能次优且训练不稳定的问题,提出 FedDiv 框架,通过建模全局噪声分布构建全局噪声过滤器,并引入基于预测一致性的采样器选择可信数据,有效防止噪声记忆并提升模型性能与训练稳定性。
摘要:
带噪声标签的联邦学习(F-LNL)旨在通过聚合多个使用本地噪声或干净样本训练的客户端模型,借助协作式分布式学习来寻求最优的服务器模型。在联邦学习框架的基础上,近来的进展主要采用标签噪声过滤技术,在每个客户端上将干净样本与噪声样本分离,从而减轻标签噪声的负面影响。然而,这些先前的方法并未通过利用所有客户端的知识来学习噪声过滤器,导致噪声过滤性能次优且较差,进而损害了训练稳定性。在本文中,我们提出了 FedDiv 来应对 F-LNL 的挑战。具体而言,我们提出了一种名为 Federated Noise Filter 的全局噪声过滤器,用于在每个客户端上有效识别带有噪声标签的样本,从而提升本地训练期间的稳定性。在不牺牲数据隐私的前提下,这是通过建模所有客户端间标签噪声的全局分布来实现的。随后,为了使全局模型获得更高性能,我们引入了一种基于预测一致性的采样器,以识别更可信的本地数据用于本地模型训练,从而防止噪声记忆并进一步提升训练稳定性。在 CIFAR-10、CIFAR-100 和 Clothing1M 上的大量实验表明,对于 IID 和 non-IID 数据划分,FedDiv 在不同的标签噪声设置下均取得了优于最先进 F-LNL 方法的性能。源代码已公开于 https://github.com/lijichang/FLNL-FedDiv。
英文摘要:
Federated learning with noisy labels (F-LNL) aims at seeking an optimal server model via collaborative distributed learning by aggregating multiple client models trained with local noisy or clean samples. On the basis of a federated learning framework, recent advances primarily adopt label noise filtering to separate clean samples from noisy ones on each client, thereby mitigating the negative impact of label noise. However, these prior methods do not learn noise filters by exploiting knowledge across all clients, leading to sub-optimal and inferior noise filtering performance and thus damaging training stability. In this paper, we present FedDiv to tackle the challenges of F-LNL. Specifically, we propose a global noise filter called Federated Noise Filter for effectively identifying samples with noisy labels on every client, thereby raising stability during local training sessions. Without sacrificing data privacy, this is achieved by modeling the global distribution of label noise across all clients. Then, in an effort to make the global model achieve higher performance, we introduce a Predictive Consistency based Sampler to identify more credible local data for local model training, thus preventing noise memorization and further boosting the training stability. Extensive experiments on CIFAR-10, CIFAR-100, and Clothing1M demonstrate that \texttt{FedDiv} achieves superior performance over state-of-the-art F-LNL methods under different label noise settings for both IID and non-IID data partitions. Source code is publicly available at https://github.com/lijichang/FLNL-FedDiv.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。