FedCC：面向解决基于蒸馏的联邦学习中的标签分布偏差问题
FedCC: Towards Addressing Label Distribution Skews in Distillation-Based Federated Learning
浏览论文内容
中文总结 AI 辅助
针对基于蒸馏的联邦学习中标签分布偏差导致的性能下降问题，提出FedCC算法，允许客户端将模糊样本标记为“未知”并结合校准伪标签，在极端标签偏差场景下准确率大幅优于基线方法。
中文摘要 AI 辅助
联邦学习（FL）使分布式客户端能够在不共享原始数据的情况下协同训练模型，有望利用通信网络中的海量设备。在基于蒸馏的联邦学习中，每个客户端在未标记的公共数据集上应用其本地模型，仅与服务器共享预测结果。异构本地数据会引入标签分布偏差，使客户端模型偏向多数类，可能导致预测不准确。公共数据集缺乏真实标签，阻碍了服务器校准预测的能力，最终降低整体性能。为解决此问题，我们提出FedCC，一种缓解客户端误分类的简单有效算法。客户端无需被迫分类并面临误差传播风险，而是可将模糊样本标记为“未知”。这个额外类别，结合公共数据上校准的伪标签，平衡了多数类的置信度与欠代表性类别的不确定性。大量实验表明，FedCC显著优于现有方法，尤其是在严重标签偏差情况下；在每个客户端仅持有10个类别中某一类样本的极端场景中，FedCC达到67.3%的准确率，而基线方法则崩溃至接近随机的结果。
英文摘要
Federated Learning (FL) enables distributed clients to collaboratively train models without sharing raw data, making it promising for leveraging massive devices in communication networks. In distillation-based FL, each client applies its local model on an unlabeled public dataset, and shares only prediction results with the server. While heterogeneous local data introduces label distribution skew, thus biasing client models toward majority classes and leading to potentially inaccurate predictions. The lack of ground-truth labels in the public dataset hampers the server's ability to calibrate predictions, which ultimately degrades overall performance. To address this, we propose FedCC, a simple and effective algorithm for mitigating client misclassification. Instead of being forced to classify and risking error propagation, clients are allowed to tag ambiguous samples as 'unknown'. This additional class, together with calibrated pseudo-labels on the public data, balances confidence in majority classes against uncertainty in under-represented ones. Extensive experiments demonstrate that FedCC significantly outperforms existing methods, especially under severe label skew. In the extreme scenario where each client holds samples from only one of ten classes, FedCC achieves 67.3% accuracy, while baselines collapse to near-random results.
发表机构
- Huawei Heisenberg Research Center, Huawei Technologies Duesseldorf GmbH（华为海森堡研究中心（华为技术有限公司杜塞尔多夫分部））
- TUM School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）
机构由 AI 辅助整理，请以论文原文为准。
