arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.06465cs.LGcs.CRcs.DS
收藏

无分布假设下的差分隐私子空间估计

On Differentially Private Subspace Estimation in a Distribution-Free Setting

Georgetown University(乔治城大学)

Eliad Tsfadia

AI总结:

本文研究差分隐私子空间估计的样本复杂度,提出以乘性奇异值间隙刻画实例“容易程度”的新度量,给出与维度无关估计的充分必要间隙类型及上下界,并用实用算法验证高维优势。

摘要:

隐私数据分析面临一个被称为维度灾难的重大挑战,会导致成本上升。然而,许多数据集具有内在的低维结构。例如,在通过梯度下降进行优化时,梯度通常位于一个低维子空间附近。如果能够利用少量数据点以隐私方式识别这种低维结构,我们就可以避免为高环境维度付出代价。在负面结果方面,Dwork、Talwar、Thakurta 和 Zhang(STOC 2014)证明,一般而言,隐私估计子空间所需的数据点数量对维度具有多项式依赖。然而,他们的界并未排除针对“容易”实例减少数据点数量的可能性。尽管如此,提出一种能够衡量给定数据集在该任务上“容易”程度的度量颇具挑战,且以往工作未对此作出恰当处理。受 Singhal 和 Steinke(NeurIPS 2021)工作的启发,我们首次给出了将“容易程度”量化为输入数据集中乘性奇异值间隙之函数的度量,并以新的上界和下界为其提供支持。特别是,我们的结果确定了首批这样的间隙类型:它们对于使用与维度无关的数据点数量来估计子空间既是充分的,也是必要的。此外,我们通过一种实用算法实现了这些上界,并展示了其在高维情形下相较以往方法的优势。

英文摘要:

Private data analysis faces a significant challenge known as the curse of dimensionality, leading to increased costs. However, many datasets possess an inherent low-dimensional structure. For instance, during optimization via gradient descent, the gradients frequently reside near a low-dimensional subspace. If the low-dimensional structure could be privately identified using a small amount of points, we could avoid paying for the high ambient dimension. On the negative side, Dwork, Talwar, Thakurta, and Zhang (STOC 2014) proved that privately estimating subspaces, in general, requires an amount of points that has a polynomial dependency on the dimension. However, their bounds do not rule out the possibility to reduce the number of points for "easy" instances. Yet, providing a measure that captures how much a given dataset is "easy" for this task turns out to be challenging, and was not properly addressed in prior works. Inspired by the work of Singhal and Steinke (NeurIPS 2021), we provide the first measures that quantify "easiness" as a function of multiplicative singular-value gaps in the input dataset, and support them with new upper and lower bounds. In particular, our results determine the first types of gaps that are sufficient and necessary for estimating a subspace with an amount of points that is independent of the dimension. Furthermore, we realize our upper bounds using a practical algorithm and demonstrate its advantage in high-dimensional regimes compared to prior approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑