CALM：面向去中心化联邦学习的类别一致性与标签门控分歧调制

中文摘要 AI 辅助

传统联邦学习依赖参数平均，这迫使客户端具有双重同质性：所有客户端必须运行相同的架构，且当本地数据非独立同分布（non-IID）时精度会下降。去中心化联邦蒸馏规避了这两个问题：每个客户端将其对等方的模型快照作为教师，在自身本地数据上进行教学，并从其软预测中蒸馏知识，无需服务器、公共数据或共享架构。然而，在严重的非IID偏差下，聚合教师目标的可信度是一个程度问题，而现有流程做出的是硬性的、全有或全无的决策：离群教师通过阈值被丢弃，而任何幸存的目标都被完全信任。我们提出CALM，它在三个层面用平滑的信任门替代每个硬决策：在类别层面，教师根据与对等共识的一致性进行加权；在样本层面，蒸馏根据教师与该目标的发散程度进行缩放；标签门根据目标对样本真实标签的支持强度进行缩放。这些操作均不增加通信或辅助数据。在CIFAR-10、SVHN、OrganAMNIST和Google Speech Commands上，采用异构客户端架构并施加Dirichlet标签偏差的情况下，CALM持续优于均匀蒸馏和硬过滤蒸馏，并达到或超过竞争性的异构联邦学习方法。

英文摘要

Conventional federated learning relies on parameter averaging, which forces clients to be doubly homogeneous: all must run an identical architecture, and accuracy degrades when local data are non-IID. Decentralized federated distillation sidesteps both: each client runs its peers' model snapshots as teachers on its own local data and distills from their soft predictions, with no server, no public data, and no shared architecture. Under severe non-IID skew, however, the trustworthiness of the aggregated teacher target is a matter of degree, yet existing pipelines make hard, all-or-nothing decisions: outlier teachers are discarded by threshold, and whatever target survives is trusted in full. We propose CALM, which replaces every hard decision with a smooth trust gate at three levels: per class, teachers are weighted by agreement with the peer consensus; per sample, distillation is scaled by the teachers' divergence from that target; and a label gate scales it by how strongly the target supports the sample's true label. None of this adds communication or auxiliary data. On CIFAR-10, SVHN, OrganAMNIST, and Google Speech Commands with heterogeneous client architectures under Dirichlet label skew, CALM consistently outperforms uniform and hard-filtered distillation and matches or exceeds competing heterogeneous-FL methods.

发表机构

• University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）

机构由 AI 辅助整理，请以论文原文为准。