发表机构
Imperial College London（帝国理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究将形式化方法（抽象解释）推广至差分隐私的预测与学习，提出抽象梯度采样算法，提供更严格的隐私保证并首次在无界灵敏度场景给出有限隐私界限。
AI 中文摘要
机器学习中的差分隐私（DP）通常通过向模型参数（私有学习）或模型输出（私有预测）添加噪声来实现。近期工作使用形式化方法，即抽象解释，来提供更严格的隐私保证，但仅限于分类设置中的私有预测。在本工作中，我们研究将形式化方法作为更严格隐私分析的通用工具。首先，我们将抽象梯度训练（AGT）框架推广到连续、无界回归中的私有预测。其次，通过将参数化模型中的学习问题归结为参数空间上的回归问题，我们引入了抽象梯度采样（AGS），一种使得基于可达性的分析能够为私有学习提供保证的算法。在私有预测和私有学习中，我们为AGT框架提供了更严格的隐私核算，并进行了理论分析，展示了我们的平滑灵敏度上界何时能产生有利的隐私-效用权衡。在实践中，我们验证了我们的回归界限在回归基准上比全局灵敏度基线更严格，并且值得注意的是，在全局预测灵敏度先验无界的设置中，我们首次提供了有限的隐私保证。我们还发现，在匹配条件下，我们的私有学习算法可以优于标准的私有学习器。
英文摘要
Differential privacy (DP) in machine learning is typically achieved by adding noise to model parameters (private learning) or to model outputs (private prediction). Recent work uses formal methods, namely abstract interpretation, to provide tighter privacy guarantees, but only for private prediction in classification settings. In this work, we investigate the use of formal methods as a general tool for tighter privacy analysis. First, we generalize the abstract gradient training (AGT) framework to private prediction in continuous, unbounded regression. Second, by reducing learning in parameterized models to a regression problem over the parameter space, we introduce Abstract Gradient Sampling (AGS), an algorithm that enables reachability-based analysis to provide guarantees for private learning. In both private prediction and private learning, we provide tightened privacy accounting for the AGT framework and a theoretical analysis demonstrating when our smooth sensitivity upper-bounds yield favourable privacy-utility trade-off. In practice, we validate that our regression bounds are tighter than global-sensitivity baselines on regression benchmarks, and, notably, yield the first finite privacy guarantees in settings where global prediction sensitivity is a priori unbounded. We also find that under matched conditions, our private learning algorithm can outperform standard private learners.
