超越参数空间：面向鲁棒联邦学习的NTK引导个性化聚合

中文摘要 AI 辅助

联邦学习（FL）支持分布式客户端在保留数据本地化的前提下开展协同模型训练，核心挑战在于确定哪些客户端更新对聚合各客户端目标域有益。现有方法通常在参数空间中通过比较模型参数或梯度解决该问题，但参数空间相似性可能无法很好地代表预测行为，尤其在异构、非独立同分布（non-IID）数据场景下。因此，与客户端目标域不一致的更新（包括由异构数据或故障客户端导致的更新）可能会降低本地模型性能。我们提出用于异构训练环境的基于局部推理引导聚合、通过一致性与正则化实现增强的联邦学习框架LIGHTYEAR，该框架在函数空间中执行更新选择。LIGHTYEAR使用基于神经正切核（NTK）的一致性分数来表征预测行为，并为每个客户端确定个性化聚合集。通过将模型参数与本地预测响应关联，神经正切核（NTK）相较于单纯的参数空间相似性，为更新选择提供了更具表达力的准则。由于传统集中式联邦学习在聚合前无法获取函数空间信息，LIGHTYEAR采用对等（P2P）拓扑，客户端直接交换更新并在私有验证数据上评估传入模型。每个客户端仅选择对自身目标域有益的更新，并使用正则化规则聚合这些更新，以提升异构场景下的稳定性。在五个数据集和九种基准方法上，LIGHTYEAR始终优于集中式联邦学习基准及现有P2P方法。

英文摘要

Federated learning (FL) enables collaborative model training across distributed clients while keeping data local. A central challenge is determining which client updates are beneficial for aggregation with respect to each client's target domain. Existing methods typically address this problem in parameter space by comparing model parameters or gradients. However, parameter-space similarity can be a poor proxy for predictive behavior, especially under heterogeneous, non-IID data. Consequently, updates that are misaligned with a client's target domain, including those caused by heterogeneous data or malfunctioning clients, may degrade local model performance. We propose Local Inference Guided Aggregation for Heterogeneous Training Environments to Yield Enhancement Through Agreement and Regularization (LIGHTYEAR), a federated learning framework that performs update selection in function space. LIGHTYEAR uses an NTK-based agreement score to characterize predictive behavior and determine a personalized aggregation set for each client. By relating model parameters to local predictive responses, the Neural Tangent Kernel (NTK) provides a more expressive criterion for update selection than parameter-space similarity alone. Because function-space information is not available before aggregation in conventional centralized FL, LIGHTYEAR uses a peer-to-peer (P2P) topology in which clients exchange updates directly and evaluate incoming models on private validation data. Each client selects only updates that are beneficial for its own target domain and aggregates them using a regularized rule that improves stability under heterogeneity. Across five datasets and nine baseline methods, LIGHTYEAR consistently outperforms centralized FL baselines and existing P2P approaches.

发表机构

• Zuse Institute Berlin (ZIB)（柏林祖斯研究所）
• Technical University of Darmstadt（达姆施塔特工业大学）

机构由 AI 辅助整理，请以论文原文为准。