DPlis:通过随机平滑提升差异私有深度学习的效用
DPlis: Boosting Utility of Differentially Private Deep Learning via Randomized Smoothing
摘要:
深度学习技术在广泛的任务中取得了卓越的性能。然而,当在隐私敏感数据集上训练时,模型参数可能暴露训练数据中的隐私信息。之前的差异化私有训练尝试,虽然提供了严格的隐私保证,但导致模型性能比非私有训练低得多。此外,同一训练算法的不同运行产生的模型具有很大的性能差异。为了解决这些问题,我们提出了DPlis--Differentially Private Learning wIth Smoothing。DPlis的核心思想是构造一个平滑的损失函数,该函数有利于位于损失景观的大平坦区域中的抗噪声模型。我们提供了理论依据的实用程序的改进DPlis。大量的实验也表明,DPlis可以有效地提高模型质量和训练稳定性在给定的隐私预算。
英文摘要:
Deep learning techniques have achieved remarkable performance in wide-ranging tasks. However, when trained on privacy-sensitive datasets, the model parameters may expose private information in training data. Prior attempts for differentially private training, although offering rigorous privacy guarantees, lead to much lower model performance than the non-private ones. Besides, different runs of the same training algorithm produce models with large performance variance. To address these issues, we propose DPlis--Differentially Private Learning wIth Smoothing. The core idea of DPlis is to construct a smooth loss function that favors noise-resilient models lying in large flat regions of the loss landscape. We provide theoretical justification for the utility improvements of DPlis. Extensive experiments also demonstrate that DPlis can effectively boost model quality and training stability under a given privacy budget.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。