arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2008.13468eess.SP
收藏

保护隐私的分布式零阶优化算法

Privacy-Preserving Distributed Zeroth-Order Optimization

C. Gratton, N. K. D. Venkategowda, R. Arablouei, S. Werner

摘要:

我们开发了一个隐私保护的分布式算法,以最小化正则化的经验风险函数时,一阶信息是不可用的,数据分布在一个多智能体网络。我们采用零阶方法,以最小化相关的增广拉格朗日函数在原域使用交替方向的乘法器(ADMM)。我们表明,该算法,命名为分布式零阶ADMM(D-ZOA),具有内在的隐私保护特性。与现有的基于ADMM的隐私保护方法不同,其中原始或对偶变量被噪声扰动,由于使用零阶方法而产生的固有随机性赋予D-ZOA固有的差分隐私。通过分析原始变量的扰动,我们证明了所提出的D-ZOA算法的隐私泄漏是有界的。此外,我们采用的时刻会计方法表明,总的隐私泄漏增长的ADMM迭代次数呈次线性。D-ZOA在准确性方面优于现有的差异隐私方法,同时产生相同的隐私保证。我们证明了D-ZOA以$\mathcal{O}(1/M)$的速度收敛到最优解,其中$M$是ADMM迭代的次数。收敛性分析还揭示了隐私和准确性之间的实际重要权衡。仿真结果验证了D-ZOA的理想的隐私保护性能和它的优越性比一个国家的最先进的算法,以及它的网络范围内收敛到最优解。

英文摘要:

We develop a privacy-preserving distributed algorithm to minimize a regularized empirical risk function when the first-order information is not available and data is distributed over a multi-agent network. We employ a zeroth-order method to minimize the associated augmented Lagrangian function in the primal domain using the alternating direction method of multipliers (ADMM). We show that the proposed algorithm, named distributed zeroth-order ADMM (D-ZOA), has intrinsic privacy-preserving properties. Unlike the existing privacy-preserving methods based on the ADMM where the primal or the dual variables are perturbed with noise, the inherent randomness due to the use of a zeroth-order method endows D-ZOA with intrinsic differential privacy. By analyzing the perturbation of the primal variable, we show that the privacy leakage of the proposed D-ZOA algorithm is bounded. In addition, we employ the moments accountant method to show that the total privacy leakage grows sublinearly with the number of ADMM iterations. D-ZOA outperforms the existing differentially private approaches in terms of accuracy while yielding the same privacy guarantee. We prove that D-ZOA converges to the optimal solution at a rate of $\mathcal{O}(1/M)$ where $M$ is the number of ADMM iterations. The convergence analysis also reveals a practically important trade-off between privacy and accuracy. Simulation results verify the desirable privacy-preserving properties of D-ZOA and its superiority over a state-of-the-art algorithm as well as its network-wide convergence to the optimal solution.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑