arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1902.01870cs.LGcs.CRstat.ML
收藏

深度神经网络中用同态加密稳定输入到近似非线性函数的推理

Stabilizing Inputs to Approximated Nonlinear Functions for Inference with Homomorphic Encryption in Deep Neural Networks

Technische Universität München(慕尼黑工业大学);German Center for Vertigo and Balance Disorders(德国眩晕与平衡障碍中心);Ludwig-Maximilians Universität München(慕尼黑大学)

Moustafa AboulAtta, Matthias Ossadnik, Seyed-Ahmad Ahmadi

摘要:

分层同态加密(LHE)提供了一种潜在的解决方案,可以允许具有敏感数据的部门利用云,并安全地部署其模型,以便使用深度神经网络(DNN)进行远程推理。然而,由于LHE的限制,这种应用面临着几个障碍。其中一个主要问题是DNN中常用的非线性函数与LHE支持的运算(即加法和乘法)不兼容。正如LHE方法中常见的那样,我们用非线性函数训练模型,并在推理时用私人数据的低次多项式近似代替它。虽然这通常会导致近似误差和预测精度的损失,但我们提出了一种方法,可以根据简单的超参数将这种损失减少到很小的值或完全消除它。这是通过引入一种新颖且优雅简单的Min-Max归一化方案来实现的,该方案将非线性函数的输入缩放到具有低近似误差的范围内。虽然它的概念很直观,实现起来也很简单,但我们的经验表明,它为DNN中的非线性函数提供了一个稳定有效的近似解决方案。反过来,这可以通过LHE实现更深入的网络,并促进安全和隐私感知分析应用程序的开发。

英文摘要:

Leveled Homomorphic Encryption (LHE) offers a potential solution that could allow sectors with sensitive data to utilize the cloud and securely deploy their models for remote inference with Deep Neural Networks (DNN). However, this application faces several obstacles due to the limitations of LHE. One of the main problems is the incompatibility of commonly used nonlinear functions in DNN with the operations supported by LHE, i.e. addition and multiplication. As common in LHE approaches, we train a model with a nonlinear function, and replace it with a low-degree polynomial approximation at inference time on private data. While this typically leads to approximation errors and loss in prediction accuracy, we propose a method that reduces this loss to small values or eliminates it entirely, depending on simple hyper-parameters. This is achieved by the introduction of a novel and elegantly simple Min-Max normalization scheme, which scales inputs to nonlinear functions into ranges with low approximation error. While being intuitive in its concept and trivial to implement, we empirically show that it offers a stable and effective approximation solution to nonlinear functions in DNN. In return, this can enable deeper networks with LHE, and facilitate the development of security- and privacy-aware analytics applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑