差分隐私联邦学习的自适应梯度裁剪与噪声注入机制
An Adaptive Gradient Clipping and Noise Injection Mechanism for Differentially Private Federated Learning
浏览论文内容
中文总结 AI 辅助
本文提出DDP-SA-adaptive自适应梯度裁剪与噪声注入机制，可改善差分隐私联邦学习的隐私-精度-效率权衡，在联邦回归任务上较静态基线提升训练效率与模型精度，且隐私预算更低。
中文摘要 AI 辅助
差分隐私联邦学习必须在隐私保护、模型精度与训练效率之间取得平衡。静态梯度裁剪在整个训练过程及各模型层使用固定阈值，若阈值过小会导致过度裁剪，若阈值过大则会引入不必要的噪声。本文提出DDP-SA-adaptive，一种结合安全聚合的差分隐私联邦学习自适应梯度裁剪与噪声添加机制。在每一轮通信中，每个客户端会基于其每个样本梯度范数的中位数，为每个模型层确定独立的裁剪阈值。由此得到的层间阈值可适应不断变化的梯度分布，并在校准拉普拉斯噪声后，再将更新内容编码并秘密共享至中间聚合服务器。我们在联邦回归任务上，从效率、精度、隐私、收敛性、裁剪范数及噪声幅度等维度对所提机制进行评估。与静态DDP-SA基线相比，DDP-SA-adaptive将通信轮次减少6.81%，总训练时间减少19.21%，平均每轮训练时间减少13.33%，提升了训练效率；同时将测试损失降低98.74%，测试R2提升3.41%，提高了模型精度。为达到R2=0.99，该自适应机制所需的隐私预算约为ε=0.1，而静态DDP-SA则需ε=0.4，因此其提供了更强的隐私保护，实现了更优的隐私保障。上述结果表明，轮次与层间的自适应机制可改善差分隐私联邦学习的隐私-精度-效率权衡。
英文摘要
Differentially private federated learning must balance privacy protection against model accuracy and training efficiency. Static gradient clipping applies a fixed threshold throughout training and across model layers, which can cause excessive clipping when the threshold is too small or unnecessarily large noise when it is too large. This paper presents DDP-SA-adaptive, an adaptive gradient clipping and noise adding mechanism for differentially private federated learning with secure aggregation. At each communication round, every client determines a separate clipping threshold for each model layer from the median of its per-sample gradient norms. The resulting layer-wise thresholds adapt to the evolving gradient distributions and calibrate the Laplace noise added before the updates are encoded and secret-shared among intermediate aggregation servers. We evaluate the proposed mechanism on a federated regression task in terms of efficiency, accuracy, privacy, convergence, clipping norm, and noise magnitude. Compared with the static DDP-SA baseline, DDP-SA-adaptive reduces the number of communication rounds by 6.81%, total training time by 19.21%, and average per-round training time by 13.33%, leading to improved training efficiency. It also reduces test loss by 98.74% and increases test R2 by 3.41%, leading to improved model accuracy. To attain R2 = 0.99, the adaptive mechanism operates with a privacy budget of approximately epsilon = 0.1, compared with epsilon = 0.4 for static DDP-SA, thus providing stronger privacy protection and achieving stronger privacy guarantees. These results demonstrate that round-wise, layer-wise adaptation can improve the privacy-accuracy-efficiency trade-off of differentially private federated learning.
发表机构
- Université Paris Cité（巴黎大学）
机构由 AI 辅助整理，请以论文原文为准。
