DP-IVON-Gradsq：差分隐私平方梯度改进变分在线牛顿法
DP-IVON-Gradsq: Differentially Private Squared-Gradient Improved Variational Online Newton
浏览论文内容
中文总结 AI 辅助
研究结合差分隐私与贝叶斯深度学习的挑战，提出DP-IVON-Gradsq方法，通过噪声校正平方梯度估计器构建曲率估计，在CIFAR-10上评估，该方法在弱到中等隐私约束下有竞争力，强隐私下性能下降。
中文摘要 AI 辅助
差分隐私为在敏感数据上训练神经网络提供了正式的隐私保证，而贝叶斯深度学习为不确定性感知预测提供了一个有原则的框架。将这两个目标结合起来仍然具有挑战性，因为隐私噪声会与贝叶斯后验采样引入的随机性相互作用。在这项工作中，我们通过改进的变分在线牛顿（IVON）优化器研究差分隐私变分贝叶斯学习。我们引入了DP-IVON-Gradsq，它是IVON的一个隐私变体。该方法使用噪声校正的平方梯度估计器从私有化梯度构建曲率估计，减少了后验采样噪声和隐私噪声之间的直接相互作用，同时保持了IVON类似Adam的计算效率。我们在一系列隐私预算下，在CIFAR-10上对DP-IVON-Gradsq与标准的隐私优化器DP-SGD和DP-Adam进行了评估。结果表明，DP-IVON-Gradsq在弱到中等隐私约束下具有竞争力，即在较大到中等的ε值下，而在强隐私下性能会下降。代码可在该https URL获取。
英文摘要
Differential privacy provides formal privacy guarantees for training neural networks on sensitive data, while Bayesian deep learning offers a principled framework for uncertainty-aware prediction. Combining these two objectives remains challenging, as privacy noise can interact with the stochasticity introduced by Bayesian posterior sampling. In this work, we investigate differentially private variational Bayesian learning through the Improved Variational Online Newton (IVON) optimizer. We introduce DP-IVON-Gradsq, a private variant of IVON. The proposed method constructs its curvature estimate from the privatized gradient using a noise-corrected squared-gradient estimator, reducing the direct interaction between posterior-sampling noise and privacy noise while preserving the Adam-like computational efficiency of IVON. We evaluate DP-IVON-Gradsq on CIFAR-10 against the standard private optimizers DP-SGD and DP-Adam over a range of privacy budgets. The results show that DP-IVON-Gradsq is competitive under weak-to-moderate privacy constraints, i.e., large-to-moderate values of $\varepsilon$, while degrading under strong privacy. Code is available at https://github.com/NourJamoussi/DP-IVON-Gradsq.git.
发表机构
- EURECOM（欧洲电信研究院）
- Concordia University（康考迪亚大学）
- University of Granada（格拉纳达大学）
机构由 AI 辅助整理，请以论文原文为准。
