具有时变样本量的差分隐私分布式随机优化
Differentially Private Distributed Stochastic Optimization with Time-Varying Sample Sizes
AI总结：
本文针对时变样本量的差分隐私分布式随机优化，提出两时间尺度随机逼近算法，通过梯度与输出扰动实现有限隐私预算下的无限迭代差分隐私，并保证收敛性，数值实验验证了其有效性。
摘要：
差分隐私分布式随机优化由于分布式随机优化中对隐私保护的迫切需求而成为热门话题。本文针对具有时变样本量的差分隐私分布式随机优化，利用梯度和输出扰动方法提出了两时间尺度随机逼近型算法。对于梯度和输出扰动两种情况，同时建立了算法的收敛性和在无限次迭代下具有有限累积隐私预算 $\varepsilon$ 的差分隐私，这与现有工作有显著不同。通过时变样本量方法，隐私水平得到增强，并建立了在无限次迭代下具有有限累积隐私预算 $\varepsilon$ 的差分隐私。通过适当选择李雅普诺夫函数，即使添加的隐私噪声具有递增方差，算法也能实现几乎必然收敛和均方收敛。此外，我们严格给出了算法的均方收敛速率，并展示了添加的隐私噪声如何影响算法的收敛速率。最后，给出了包括在基准机器学习数据集上的分布式训练在内的数值示例，以证明算法的效率和优势。
英文摘要：
Differentially private distributed stochastic optimization has become a hot topic due to the urgent need of privacy protection in distributed stochastic optimization. In this paper, two-time scale stochastic approximation-type algorithms for differentially private distributed stochastic optimization with time-varying sample sizes are proposed using gradient- and output-perturbation methods. For both gradient- and output-perturbation cases, the convergence of the algorithm and differential privacy with a finite cumulative privacy budget $\varepsilon$ for an infinite number of iterations are simultaneously established, which is substantially different from the existing works. By a time-varying sample sizes method, the privacy level is enhanced, and differential privacy with a finite cumulative privacy budget $\varepsilon$ for an infinite number of iterations is established. By properly choosing a Lyapunov function, the algorithm achieves almost-sure and mean-square convergence even when the added privacy noises have an increasing variance. Furthermore, we rigorously provide the mean-square convergence rates of the algorithm and show how the added privacy noise affects the convergence rate of the algorithm. Finally, numerical examples including distributed training on a benchmark machine learning dataset are presented to demonstrate the efficiency and advantages of the algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
