超越非独立同分布：联邦学习中的学习者-客户端分布不匹配
Beyond Non-IID: Learner--Client Distribution Mismatch in Federated Learning
- Northeastern University（东北大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文针对联邦学习中学习者与客户端的分布不匹配问题，提出动态感知影响力的客户端选择框架，在CIFAR-10数据集上验证其性能优于基线，可加快收敛并提升准确率。
AI中文摘要：
联邦学习系统正越来越多地被部署，以促进在异构客户端群体中开展协作式模型训练。现有实践大多隐含地假设聚合后的客户端数据分布能代表学习者的目标分布，或者从所有可用客户端学习对学习者分布均有同等益处，但这种假设在现实中往往不成立。联邦学习文献中的传统客户端选择策略在很大程度上忽略了这种错位，而多数多源迁移学习的现有工作要么要求直接访问本地数据，要么采用一次性模型/特征聚合。在本文中，我们主动研究并缓解这种学习者-客户端群体错位的影响，具体考虑学习者持有小型代理数据集的实际场景。我们观察到，客户端贡献在各训练轮次中差异显著，而传统技术无法在多源迁移多样性下识别有益的来源。随后，我们提出一种动态的、感知影响力的客户端选择框架，该框架利用学习者特定代理集上的代理影响力信号，估算每个客户端对学习者优化目标的潜在效用。通过使用留一法评估，我们优先选择最具信息性的知识来源，同时控制统计噪声和数据异构性的负面影响。在异构数据划分下的CIFAR-10数据集上进行的实验表明，我们的方法始终优于静态和动态基线，实现了更快的收敛速度和更高的准确率。
英文摘要：
Federated learning systems are increasingly deployed to facilitate collaborative model training across a heterogeneous client population. Existing practice mostly implicitly assumes that the aggregated client data distribution is representative of the learner's target distribution or that learning from all available clients is uniformly beneficial for the learner distribution. However, such an assumption often does not hold in reality. Traditional client selection strategies in FL literature largely overlook such misalignment, while most existing work on multi-source transfer learning either requires direct access to local data or uses one-shot model/feature aggregation. In this paper, we take the initiative to understand and mitigate the impacts of such learner-client population misalignment. In particular, we consider the practical setting where the learner keeps a small proxy dataset. We observe that client contributions vary significantly across training rounds, and traditional technology is insufficient to identify beneficial sources under multi-source transfer diversity. Then, we propose a dynamic, influence-aware client selection framework that estimates each client's potential utility to the learner's optimization objective using proxy influence signals on a learner-specific proxy set. Via using leave-one-out evaluations, we prioritize the most informative sources of knowledge while controlling the negative impacts of statistical noise and data heterogeneity. Experiments on CIFAR-10 under heterogeneous data partitions demonstrate that our approach consistently outperforms static and dynamic baselines, achieving faster convergence and higher accuracy.
