DP-FedSOFIM: 使用正则化Fisher信息矩阵的差分隐私联邦随机优化
DP-FedSOFIM: Differentially Private Federated Stochastic Optimization using Regularized Fisher Information Matrix
AI总结：
针对差分隐私联邦学习在严格隐私预算下收敛慢的问题，提出DP-FedSOFIM方法，利用正则化Fisher信息矩阵的代理进行二阶优化，无需完整Hessian计算，在CIFAR-10和PathMNIST上实现更快收敛和更高精度。
摘要：
差分隐私联邦学习（DP-FL）在严格的隐私预算下常常收敛缓慢，因为隐私保护所需的噪声会降低梯度质量。尽管二阶优化可以加速训练，但现有的DP-FL方法面临显著的可扩展性限制：牛顿型方法需要客户端计算Hessian矩阵，而特征协方差方法随模型维度扩展性差。我们提出DP-FedSOFIM，一种简单且可扩展的基于Hessian近似的二阶优化方法用于DP-FL。该方法仅使用私有化的聚合梯度在服务器端构建Fisher信息矩阵的正则化代理，捕获有用的曲率信息，无需完整的Hessian计算或特征协方差估计。基于Sherman-Morrison公式的高效秩一更新使得通信成本与模型大小成比例，并且仅需要O(d)的客户端内存。由于所有曲率和预处理操作都在服务器端对已经私有化的梯度进行，DP-FedSOFIM除了底层的私有化梯度发布机制外，不引入额外的隐私成本。在CIFAR-10和PathMNIST上的实验表明，DP-FedSOFIM在广泛的隐私预算范围内比几种有竞争力的差分隐私联邦学习基线方法收敛更快，并持续获得更高的精度。
英文摘要：
Differentially private federated learning (DP-FL) often suffers from slow convergence under tight privacy budgets because the noise required for privacy preservation degrades gradient quality. Although second-order optimization can accelerate training, existing approaches for DP-FL face significant scalability limitations: Newton-type methods require clients to compute Hessians, while feature covariance methods scale poorly with model dimension. We propose DP-FedSOFIM, a simple and scalable Hessian approximation-based second-order optimization method for DP-FL. The method constructs a regularized proxy for the Fisher information matrix at the server using only privatized aggregated gradients, capturing useful curvature information without requiring full Hessian computations or feature covariance estimation. Efficient rank-one updates based on the Sherman-Morrison formula enable communication costs proportional to the model size and require only O(d) client-side memory. Because all curvature and preconditioning operations are performed at the server on already privatized gradients, DP-FedSOFIM introduces no additional privacy cost beyond the underlying privatized gradient release mechanism. Experiments on CIFAR-10 and PathMNIST demonstrate that DP-FedSOFIM converges faster and consistently achieves higher accuracy than several competitive differentially private federated learning baselines across a wide range of privacy budgets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
