发表机构
The Ohio State University（俄亥俄州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
该研究提出VANEB框架，通过结合局部M估计器的参数相关渐近方差与广义Tweedie公式，实现异构客户端的个性化联邦学习，在MNIST、CIFAR-10数据集上的DNN场景中表现优异。
AI 中文摘要
我们提出了一种利用非参数经验贝叶斯（NPEB）处理异构客户端间个性化联邦学习的新方法。该方法利用通过经验风险最小化或M估计得到的局部参数估计的渐近正态性，将这些估计值作为含噪观测值，通过非参数极大似然估计未知的共享先验。在该场景下应用NPEB的一个关键挑战是，现有方法假设方差已知且固定，而这在实际中并不成立。为解决该问题，我们引入了方差感知非参数经验贝叶斯（VANEB）框架，该框架利用局部M估计器的参数相关渐近方差。一项关键技术贡献是针对这种异方差场景的广义Tweedie公式。我们随后在平均平方Hellinger距离下建立了密度估计的非渐近误差率，并推导了为估计量提供误差界的神谕去噪不等式。虽然我们的理论保证植根于M估计器的渐近 regime，但我们实证探索了VANEB到涉及深度神经网络（DNN）的现代联邦学习场景的启发式扩展。对于DNN，我们提出了VANEB-head和VANEB-FT，它们通过使用近似对角方差估计器的NPEB步骤来个性化最后一个全连接层。我们表明，该方法在流行的视觉数据集MNIST和CIFAR-10上使用卷积神经网络架构具有强大的性能。
英文摘要
We develop a new approach to Personalized Federated Learning across heterogeneous clients using Nonparametric Empirical Bayes (NPEB). Leveraging the asymptotic normality of local parameter estimates obtained from Empirical Risk Minimization or M-estimation, our method formulates these estimates as noisy observations to estimate an unknown shared prior via Nonparametric Maximum Likelihood. A key challenge in applying NPEB in this setting is that existing approaches assume known fixed variances, which is not true in practice. To address this, we introduce a Variance-Aware Nonparametric Empirical Bayes (VANEB) framework that leverages the parameter-dependent asymptotic variance of local M-estimators. A key technical contribution is a generalized Tweedie's formula for this heteroskedastic setting. We then establish non-asymptotic error rates for density estimation in the average squared Hellinger distance and derive an oracle denoising inequality that provides error bounds for our estimator. While our theoretical guarantees are rooted in the asymptotic regime of M-estimators, we empirically explore heuristic extensions of VANEB to modern federated learning settings involving Deep Neural Networks (DNNs). For DNNs, we propose VANEB-head and VANEB-FT, which personalize the last fully connected layer via an NPEB step using an approximate diagonal variance estimator. We show that our method has strong performance on popular vision datasets MNIST and CIFAR-10, using a convolutional neural network architecture.
