Fed-GraB：基于自调整梯度平衡器的联邦长尾学习
Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancer
AI总结：
本文研究联邦长尾学习任务，提出Fed-GraB方法，通过DPA模块评估全局分布并利用SGB模块闭环重加权客户端梯度，有效缓解数据异质性导致的分布漂移，在保持多数类性能的同时提升少数类性能，在多个数据集上达到SOTA。
摘要：
数据隐私和长尾分布在许多现实任务中是常态而非例外。本文研究联邦长尾学习（Fed-LT）任务，其中每个客户端持有一个本地异构数据集；如果将数据集全局聚合，它们共同呈现长尾分布。在这种设置下，现有的联邦优化和/或集中式长尾学习方法难以适用，原因在于存在以下挑战：(a) 在隐私约束下刻画全局长尾分布，以及 (b) 调整本地学习策略以应对头尾不平衡。作为回应，我们提出了一种称为 $\ exttt{Fed-GraB}$ 的方法，包含一个自调整梯度平衡器（SGB）模块，该模块基于 Direct Prior Analyzer（DPA）模块评估的全局长尾分布反馈，以闭环方式重新加权客户端的梯度。使用 $\ exttt{Fed-GraB}$，客户端能够有效缓解模型训练过程中由数据异质性引起的分布漂移，并获得一个在少数类上具有更好性能的全局模型，同时保持多数类的性能。大量实验表明，$\ exttt{Fed-GraB}$ 在 CIFAR-10-LT、CIFAR-100-LT、ImageNet-LT 和 iNaturalist 等代表性数据集上取得了最先进的性能。
英文摘要：
Data privacy and long-tailed distribution are the norms rather than the exception in many real-world tasks. This paper investigates a federated long-tailed learning (Fed-LT) task in which each client holds a locally heterogeneous dataset; if the datasets can be globally aggregated, they jointly exhibit a long-tailed distribution. Under such a setting, existing federated optimization and/or centralized long-tailed learning methods hardly apply due to challenges in (a) characterizing the global long-tailed distribution under privacy constraints and (b) adjusting the local learning strategy to cope with the head-tail imbalance. In response, we propose a method termed $\texttt{Fed-GraB}$, comprised of a Self-adjusting Gradient Balancer (SGB) module that re-weights clients' gradients in a closed-loop manner, based on the feedback of global long-tailed distribution evaluated by a Direct Prior Analyzer (DPA) module. Using $\texttt{Fed-GraB}$, clients can effectively alleviate the distribution drift caused by data heterogeneity during the model training process and obtain a global model with better performance on the minority classes while maintaining the performance of the majority classes. Extensive experiments demonstrate that $\texttt{Fed-GraB}$ achieves state-of-the-art performance on representative datasets such as CIFAR-10-LT, CIFAR-100-LT, ImageNet-LT, and iNaturalist.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
