arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.08557cs.LGcs.AIcs.CR
收藏

DPNAS:基于差分隐私的深度学习神经架构搜索

DPNAS: Neural Architecture Search for Deep Learning with Differential Privacy

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)

Anda Cheng, Jiaxing Wang, Xi Sheryl Zhang, Qiang Chen, Peisong Wang, Jian Cheng

摘要:

针对有意义的差分隐私(DP)训练深度神经网络(DNN)保证严重降低模型效用。在本文中,我们证明了DNN的架构在私有深度学习的背景下对模型效用有重大影响,而其影响在以前的研究中基本上未被探索。鉴于这种缺失,我们提出了第一个框架,该框架采用神经架构搜索来自动设计私人深度学习的模型,称为DPNAS。为了将私有学习与架构搜索相结合,我们精心设计了一个新的搜索空间,并提出了一种DP感知的候选模型训练方法。我们经验证明所提出的框架的有效性。所搜索的模型DPNASNet实现了最先进的隐私/效用权衡,例如,对于$(ε, δ)=(3, 1\times10^{-5})$的隐私预算,我们的模型获得了$98.57\%$在MNIST上、$88.09\%$在FashionMNIST上和$68.33\%$在CIFAR-10上的测试准确度。此外,通过研究生成的架构,我们提供了设计私有学习友好的DNN的几个有趣的发现,这可以为具有差分隐私的深度学习模型设计提供新的思路。

英文摘要:

Training deep neural networks (DNNs) for meaningful differential privacy (DP) guarantees severely degrades model utility. In this paper, we demonstrate that the architecture of DNNs has a significant impact on model utility in the context of private deep learning, whereas its effect is largely unexplored in previous studies. In light of this missing, we propose the very first framework that employs neural architecture search to automatic model design for private deep learning, dubbed as DPNAS. To integrate private learning with architecture search, we delicately design a novel search space and propose a DP-aware method for training candidate models. We empirically certify the effectiveness of the proposed framework. The searched model DPNASNet achieves state-of-the-art privacy/utility trade-offs, e.g., for the privacy budget of $(ε, δ)=(3, 1\times10^{-5})$, our model obtains test accuracy of $98.57\%$ on MNIST, $88.09\%$ on FashionMNIST, and $68.33\%$ on CIFAR-10. Furthermore, by studying the generated architectures, we provide several intriguing findings of designing private-learning-friendly DNNs, which can shed new light on model design for deep learning with differential privacy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑