arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.01243cs.CRcs.LG
收藏

正态分布不可区分光谱及其在隐私保护机器学习中的应用

The Normal Distributions Indistinguishability Spectrum and its Application to Privacy-Preserving Machine Learning

Georgia Institute of Technology(佐治亚理工学院);University of Victoria(维多利亚大学);Rensselaer Polytechnic Institute(拉特格斯理工学院)

Yu Wei, Yun Lu, Malik Magdon-Ismail, Vassilis Zikas

AI总结:

本文研究了任何输出具有高斯分布的算法的隐私性,提出了一种名为正态分布不可区分光谱(NDIS)的通用引理,用于计算任意两个多元高斯分布之间的hockey-stick散度,并将其应用于随机投影等算法的隐私证明,从而实现更高效的隐私保护机制。

摘要:

我们研究了任何输出具有高斯分布的算法的隐私性。这项工作受到在多个有用(ML)应用中广泛使用此类算法,以及在数据上添加高斯噪声(如DP-SGD)之外相对较少关注隐私保护学习的启发。什么是任何具有多元高斯输出的算法的DP?我们通过一个通用引理来回答这个问题,该引理称为正态分布不可区分光谱(NDIS),用于计算任意两个多元高斯分布之间的hockey-stick散度δ,参数化为隐私参数ε。为了展示其实际影响,我们证明了NDIS引理的几个性质。这些性质形成了一组结果工具,可用于为任何高斯输出算法提供更简单的隐私证明。作为我们工具包的一个应用示例,我们证明了随机投影(RP)隐私的更紧参数化,并由此获得一个更节省噪声的DP机制。除了随机投影,NDIS可以用于将任何具有`sensitivity`(我们定义)的高斯输出算法提升为高斯输出的DP机制。该机制增强了现有算法中的随机性,使得可以将机制的隐私描述为单对高斯分布之间的IS,然后通过NDIS进行分析。最后,我们利用NDIS与广义χ²分布CDF之间的联系(具有高效的实证估计器)来提出一个用于高斯输出算法白盒审计的工具。

英文摘要:

We investigate the privacy of {\em any} algorithm whose outputs have Gaussian distribution. This work is motivated by the prevalence of such algorithms in several useful (ML) applications, and the comparatively little research that focuses on privacy-preserving learning outside of adding Gaussian noise to the data (such as DP-SGD). {\em What is the DP of any algorithm with multivariate Gaussian output?} We answer the above research question with a general lemma which we call {\em Normal Distributions Indistinguishability Spectrum} (NDIS), a closed-form analytic computation of the hockey-stick divergence $δ$ between an arbitrary pair of multivariate Gaussians, parameterized by privacy parameter $ε$. To show its practical implications, we prove several properties of our NDIS lemma. These properties form a {\em toolbox} of results which lead to potentially {\em easier} privacy proofs for any Gaussian-output algorithm. As an example application of our toolbox, we prove a tighter parametrisation of the privacy of {\em random projection (RP)}, and obtaining from it a more noise-frugal DP mechanism. Beyond random projection, NDIS can be used to lift {\em any} Gaussian-output algorithm with a `sensitivity' (which we define) to a Gaussian-output DP mechanism. The mechanism boosts the existing randomness in the algorithm, so that one can describe the mechanism's privacy as the IS between a single pair of Gaussians, which can then be analyzed via NDIS. Lastly, we leverage the connections between NDIS and the CDF of the generalized $χ^2$ distribution (which have efficient empirical estimators) to present a tool for white-box auditing of Gaussian-output algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑