扰动目标上梯度下降的差分隐私
Differential Privacy of Gradient Descent on Perturbed Objectives
浏览论文内容
中文总结 AI 辅助
本文研究扰动目标上梯度下降的有限迭代隐私，证明其微分同胚条件并给出无维度因子的隐私界，同时以几何递减修正恢复精确最小化器隐私证书。
中文摘要 AI 辅助
目标扰动向正则化经验风险添加一个随机线性项，并释放精确的扰动最小化器。我们研究在$w\mapsto F(w;S)+\langle z,w\rangle$上确定性梯度下降的第$N$次迭代所获得的有限计算，其中$z\sim\mathcal N(0,\sigma^2I_d)$在优化前仅抽取一次。对于具有Lipschitz Hessian的强凸且光滑目标，我们证明了一个显式条件，在该条件下，映射$z\mapsto w_N$在隐私论证所用的有界域上是$C^1$-微分同胚，并给出其Jacobian最小奇异值的定量下界。这允许对有限迭代进行直接的变量替换分析。对于广义线性模型，一旦迭代条件成立，所得的隐私轮廓界没有显式的环境维度因子，且其有限迭代修正呈几何级数递减。通过让自由截断参数随$N$缓慢增长，我们在极限情况下恢复了相应的精确最小化器证书。我们还以$d\sigma^2/(2\mu)$加上一个几何递减的优化项来界定期望超额经验风险，并将结果转移到总体风险，而无需在前导统计项中引入额外的乘法条件数因子。
英文摘要
Objective perturbation adds a random linear term to a regularized empirical risk and releases the exact perturbed minimizer. We study the finite computation obtained by releasing the $N$-th iterate of deterministic gradient descent on $w\mapsto F(w;S)+\langle z,w\rangle$, where $z\sim\mathcal N(0,σ^2I_d)$ is drawn once before optimization. For strongly convex and smooth objectives with Lipschitz Hessian, we prove an explicit condition under which the map $z\mapsto w_N$ is a $C^1$-diffeomorphism on the bounded domains used in the privacy argument, with a quantitative lower bound on the smallest singular value of its Jacobian. This permits a direct change-of-variables analysis of the finite iterate. For generalized linear models, the resulting privacy-profile bound has no explicit ambient-dimension factor once the iteration condition holds, and its finite-iteration correction decreases geometrically. By letting the free truncation parameter grow slowly with $N$, we recover the corresponding exact-minimizer certificate in the limit. We also bound the expected excess empirical risk by $dσ^2/(2μ)$ plus a geometrically decreasing optimization term, and transfer the result to population risk without an additional multiplicative condition-number factor in the leading statistical terms.
发表机构
- Johns Hopkins University（约翰霍普金斯大学）
机构由 AI 辅助整理，请以论文原文为准。
