arXiv 2205.030142022-05cs.LGstat.ML

重新审视差分隐私广义线性模型

Differentially Private Generalized Linear Models Revisited

The Johns Hopkins University（约翰斯·霍普金斯大学）；The Ohio State University（俄亥俄州立大学）；University of Twente（特文特大学）；Pontificia Universidad Católica de Chile（智利天主教大学）

Raman Arora, Raef Bassily, Cristóbal Guzmán, Michael Menart, Enayat Ullah

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该研究重新审视差分隐私广义线性模型，针对两类凸损失线性预测器的隐私学习问题，推导了紧的上下界与最优速率，提出无需先验知识的私有模型选择方法实现最优性能。

摘要：

我们研究带凸损失的线性预测器的$(ε,δ)$-差分隐私学习问题，给出两类损失函数子类的研究结果。第一类是损失平滑且非负、但不一定满足Lipschitz条件的情况（例如平方损失），针对该情况我们建立了过剩总体风险的上界$\tilde{O}\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^* \Vert^2}{(nε)^{2/3}},\frac{\sqrt{d}\Vert w^*\Vert^2}{nε}\right\}\right)$，其中$n$为样本数量，$d$为问题维度，$w^*$是总体风险的最小化器。除了对$\Vert w^\ast\Vert$的依赖外，我们的界在所有参数上基本是紧的，我们还给出了下界$\tildeΩ\left(\frac{1}{\sqrt{n}} + {\min\left\{\frac{\Vert w^*\Vert^{4/3}}{(nε)^{2/3}}, \frac{\sqrt{d}\Vert w^*\Vert}{nε}\right\}}\right)$。我们还重新研究了此前已被探讨的Lipschitz损失情况[SSTT20]，填补了现有研究的差距，证明最优速率（至多相差对数因子）为$Θ\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^*\Vert}{\sqrt{nε}},\frac{\sqrt{\text{rank}}\Vert w^*\Vert}{nε}\right\}\right)$，其中$\text{rank}$是设计矩阵的秩，该结果在高隐私 regime 下优于现有研究。最后，我们的算法包含一种我们提出的私有模型选择方法，可在无需预先知晓$\Vert w^*\Vert$的情况下达到上述速率。

英文摘要：

We study the problem of $(ε,δ)$-differentially private learning of linear predictors with convex losses. We provide results for two subclasses of loss functions. The first case is when the loss is smooth and non-negative but not necessarily Lipschitz (such as the squared loss). For this case, we establish an upper bound on the excess population risk of $\tilde{O}\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^* \Vert^2}{(nε)^{2/3}},\frac{\sqrt{d}\Vert w^*\Vert^2}{nε}\right\}\right)$, where $n$ is the number of samples, $d$ is the dimension of the problem, and $w^*$ is the minimizer of the population risk. Apart from the dependence on $\Vert w^\ast\Vert$, our bound is essentially tight in all parameters. In particular, we show a lower bound of $\tildeΩ\left(\frac{1}{\sqrt{n}} + {\min\left\{\frac{\Vert w^*\Vert^{4/3}}{(nε)^{2/3}}, \frac{\sqrt{d}\Vert w^*\Vert}{nε}\right\}}\right)$. We also revisit the previously studied case of Lipschitz losses [SSTT20]. For this case, we close the gap in the existing work and show that the optimal rate is (up to log factors) $Θ\left(\frac{\Vert w^*\Vert}{\sqrt{n}} + \min\left\{\frac{\Vert w^*\Vert}{\sqrt{nε}},\frac{\sqrt{\text{rank}}\Vert w^*\Vert}{nε}\right\}\right)$, where $\text{rank}$ is the rank of the design matrix. This improves over existing work in the high privacy regime. Finally, our algorithms involve a private model selection approach that we develop to enable attaining the stated rates without a-priori knowledge of $\Vert w^*\Vert$.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文