利用坐标间差异实现更高实用性的差分隐私:拉普拉斯机制在高维空间中可超越高斯机制
Differential Privacy with Higher Utility by Exploiting Coordinate-wise Disparity: Laplace Mechanism Can Beat Gaussian in High Dimensions
AI总结:
本文提出跨坐标独立非同分布(i.n.i.d.)噪声机制,利用坐标间敏感度差异优化隐私-准确性权衡,理论分析和实验证明其在高维下优于i.i.d.机制,且拉普拉斯机制可超越高斯机制。
摘要:
传统上,在差分隐私加性噪声机制中,独立同分布(i.i.d.)的噪声样本被添加到响应的每个坐标上。在本工作中,我们正式提出了添加跨坐标独立但非同分布(i.n.i.d.)噪声的方法,通过利用隐私泄露中的坐标间差异来实现更紧密的隐私-准确性权衡。具体而言,我们研究了i.n.i.d.高斯和拉普拉斯机制,并获得了这些机制保证隐私的条件。推导了确保这些条件的最优参数选择,考虑(加权)均方误差和$\ell_{p}^{p}$误差作为准确性度量。理论分析和数值模拟表明,与i.i.d.对应机制相比,i.n.i.d.机制在给定隐私要求下实现了更高的实用性。一个有趣的观察是,如果利用坐标间敏感度的不规则性,拉普拉斯机制即使在较高维度上也优于高斯机制,这与普遍看法相反。我们还演示了i.n.i.d.噪声如何在私有(a)坐标下降法、(b)主成分分析以及(c)带组裁剪的深度学习中提高性能。
英文摘要:
Conventionally, in a differentially private additive noise mechanism, independent and identically distributed (i.i.d.) noise samples are added to each coordinate of the response. In this work, we formally present the addition of noise that is independent but not identically distributed (i.n.i.d.) across the coordinates to achieve tighter privacy-accuracy trade-off by exploiting coordinate-wise disparity in privacy leakage. In particular, we study the i.n.i.d. Gaussian and Laplace mechanisms and obtain the conditions under which these mechanisms guarantee privacy. The optimal choice of parameters that ensure these conditions are derived considering (weighted) mean squared and $\ell_{p}^{p}$-errors as measures of accuracy. Theoretical analyses and numerical simulations demonstrate that the i.n.i.d. mechanisms achieve higher utility for the given privacy requirements compared to their i.i.d. counterparts. One of the interesting observations is that the Laplace mechanism outperforms Gaussian even in high dimensions, as opposed to the popular belief, if the irregularity in coordinate-wise sensitivities is exploited. We also demonstrate how the i.n.i.d. noise can improve the performance in private (a) coordinate descent, (b) principal component analysis, and (c) deep learning with group clipping.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。