arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.02672cs.CRcs.ITmath.IT
收藏

敏感性的几何:差分隐私中的二次采样与混合裁剪、最优高斯噪声及其在深度学习中的应用

Geometry of Sensitivity: Twice Sampling and Hybrid Clipping in Differential Privacy with Optimal Gaussian Noise and Application to Deep Learning

Hanshen Xiao, Jun Wan, Srinivas Devadas

AI总结:

本文研究差分隐私高维敏感性集合的最优高斯噪声几何,提出混合裁剪下的样本级与坐标级二次采样方法,并通过闭式RDP分析证明其隐私放大改进。

摘要:

我们研究差分隐私中构造最优随机化机制的基础性问题。根据裁剪策略或处理函数的附加属性,相应的敏感性集合在理论上决定了产生所需安全参数所必需的随机化。为实现最优效用-隐私权衡,为适当选择的敏感性集合寻找最小扰动是差分隐私研究中的核心问题。在实践中,采用高斯/拉普拉斯噪声机制的 l_2/l_1 范数裁剪属于最常见的配置。然而,它们也受到维数灾难的困扰。对于更通用的裁剪策略,人们对高维敏感性集合的最优噪声理解仍然有限。在本文中,我们重新审视高维敏感性集合的几何结构,并给出一系列结果,以刻画 Rényi DP(RDP)的非渐近最优高斯噪声。我们的结果兼具负面与正面两方面:一方面,我们证明对于满足特定对称性的一大类敏感性集合,维数灾难是紧的;但如果幸运的是,敏感性集合在某组正交基上的表示是非对称的,我们证明最优噪声界无需显式依赖于维数或秩。我们还重新审视高维场景下的采样,这是大规模数据处理中隐私放大和计算效率的关键。我们提出一种称为二次采样的新方法,同时实施样本级采样和坐标级采样,使高斯噪声能够更紧密地适配敏感性几何。借助闭式 RDP 分析,我们证明在附加无穷范数限制的条件下,二次采样可带来隐私放大的渐近改进,尤其适用于小采样率情形。

英文摘要:

We study the fundamental problem of the construction of optimal randomization in Differential Privacy. Depending on the clipping strategy or additional properties of the processing function, the corresponding sensitivity set theoretically determines the necessary randomization to produce the required security parameters. Towards the optimal utility-privacy tradeoff, finding the minimal perturbation for properly-selected sensitivity sets stands as a central problem in DP research. In practice, l_2/l_1-norm clippings with Gaussian/Laplace noise mechanisms are among the most common setups. However, they also suffer from the curse of dimensionality. For more generic clipping strategies, the understanding of the optimal noise for a high-dimensional sensitivity set remains limited. In this paper, we revisit the geometry of high-dimensional sensitivity sets and present a series of results to characterize the non-asymptotically optimal Gaussian noise for Rényi DP (RDP). Our results are both negative and positive: on one hand, we show the curse of dimensionality is tight for a broad class of sensitivity sets satisfying certain symmetry properties; but if, fortunately, the representation of the sensitivity set is asymmetric on some group of orthogonal bases, we show the optimal noise bounds need not be explicitly dependent on either dimension or rank. We also revisit sampling in the high-dimensional scenario, which is the key for both privacy amplification and computation efficiency in large-scale data processing. We propose a novel method, termed twice sampling, which implements both sample-wise and coordinate-wise sampling, to enable Gaussian noises to fit the sensitivity geometry more closely. With closed-form RDP analysis, we prove twice sampling produces asymptotic improvement of the privacy amplification given an additional infinity-norm restriction, especially for small sampling rate.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑