PolyMPCNet：面向ReLU-free的两方计算私有推理神经结构搜索
PolyMPCNet: Towards ReLU-free Neural Architecture Search in Two-party Computation Based Private Inference
摘要：
深度学习（DL）的快速发展和部署见证了新出现的隐私和安全问题。为了缓解这些问题，已经讨论了安全多方计算（MPC），以实现隐私保护DL计算。在实践中，它们通常会带来非常高的计算和通信开销，并可能阻碍它们在大规模系统中的普及。两个正交的研究趋势在解决安全深度学习中的能量效率方面吸引了巨大的兴趣，即，MPC比较协议的开销减少和硬件加速。然而，由于计算和通信节省有限，它们要么实现低缩减率并遭受高延迟，要么由于现有工作主要集中在CPU和GPU等通用计算平台上而耗电。在这项工作中，作为第一次尝试，我们开发了一个系统的框架，PolyMPCNet，联合开销减少MPC比较协议和硬件加速，通过集成硬件延迟的密码构建块到DNN损失函数，以实现高能效，准确性和安全性保证。我们的关键设计原则不是在DNN经过良好训练后（通过删除或丢弃一些非多项式运算符）验证性地检查模型敏感性，而是严格执行DNN设计中的假设-训练一个既硬件高效又安全的DNN，同时逃避局部最小值和鞍点并保持高精度。更具体地说，我们提出了一种直通多项式激活初始化方法，用于加密硬件友好的可训练多项式激活函数，以取代昂贵的2 P-ReLU运算符。针对现场可编程门阵列（FPGA）平台，设计了一种密码硬件调度器，并建立了相应的性能模型。
英文摘要：
The rapid growth and deployment of deep learning (DL) has witnessed emerging privacy and security concerns. To mitigate these issues, secure multi-party computation (MPC) has been discussed, to enable the privacy-preserving DL computation. In practice, they often come at very high computation and communication overhead, and potentially prohibit their popularity in large scale systems. Two orthogonal research trends have attracted enormous interests in addressing the energy efficiency in secure deep learning, i.e., overhead reduction of MPC comparison protocol, and hardware acceleration. However, they either achieve a low reduction ratio and suffer from high latency due to limited computation and communication saving, or are power-hungry as existing works mainly focus on general computing platforms such as CPUs and GPUs. In this work, as the first attempt, we develop a systematic framework, PolyMPCNet, of joint overhead reduction of MPC comparison protocol and hardware acceleration, by integrating hardware latency of the cryptographic building block into the DNN loss function to achieve high energy efficiency, accuracy, and security guarantee. Instead of heuristically checking the model sensitivity after a DNN is well-trained (through deleting or dropping some non-polynomial operators), our key design principle is to em enforce exactly what is assumed in the DNN design -- training a DNN that is both hardware efficient and secure, while escaping the local minima and saddle points and maintaining high accuracy. More specifically, we propose a straight through polynomial activation initialization method for cryptographic hardware friendly trainable polynomial activation function to replace the expensive 2P-ReLU operator. We develop a cryptographic hardware scheduler and the corresponding performance model for Field Programmable Gate Arrays (FPGA) platform.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
