Hybrid private inference (PI) protocol, which synergistically utilizes both multi-party computation (MPC) and homomorphic encryption, is one of the most prominent techniques for PI. However, even the state-of-the-art PI protocols are bottlenecked by the non-linear layers, especially the activation functions. Although a standard non-linear activation function can generate higher model accuracy, it must be processed via a costly garbled-circuit MPC primitive. A polynomial activation can be processed via Beaver's multiplication triples MPC primitive but has been incurring severe accuracy drops so far.
In this paper, we propose an accuracy preserving low-degree polynomial activation function (AESPA) that exploits the Hermite expansion of the ReLU and basis-wise normalization. We apply AESPA to popular ML models, such as VGGNet, ResNet, and pre-activation ResNet, to show an inference accuracy comparable to those of the standard models with ReLU activation, achieving superior accuracy over prior low-degree polynomial studies. When applied to the all-RELU baseline on the state-of-the-art Delphi PI protocol, AESPA shows up to 42.1x and 28.3x lower online latency and communication cost.
Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity
State Key Laboratory of Cyberspace Security Defense, Institute of Information Engineering, CAS and University of Chinese Academy of Sciences（国家密码安全防御重点实验室，信息工程研究所，中国科学院及中国科学院大学）；EIRI, NELBDRC, Tsinghua University（EIRI，NELBDRC，清华大学）
CipherPrune: Efficient and Scalable Private Transformer Inference
University of Central Florida（中佛罗里达大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；Ant Research（蚂蚁集团研究院）；Indiana University Bloomington（印第安纳大学伯明顿分校）
Optimized Layerwise Approximation for Efficient Private Inference on Fully Homomorphic Encryption
Department of Electrical and Computer Engineering, INMC, Seoul National University（电子与计算机工程系，首尔国立大学）；School of Computer Science and Engineering, Chung-Ang University（计算机科学与工程学院， Chung-Ang 大学）；Department of Software, Sejong University（软件系，世宗大学）；Department of Electrical Engineering and Computer Science, DGIST（电子与计算机科学系，DGIST）；Department of Electrical and Electronic Engineering, Inha University（电子与电气工程系，Inha 大学）；Department of Electrical Engineering, POSTECH（电子工程系，POSTECH）
Differentially Private Pre-Trained Model Fusion using Decentralized Federated Graph Matching
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Changsha Aier Eye Hospital（长沙爱尔眼科医院）；Nanchang Aier Eye Hospital（南昌爱尔眼科医院）
CryptoGRU: Low Latency Privacy-Preserving Text Analysis With GRU
Intelligent Systems Engineering（智能系统工程系）；Luddy School of Informatics, Computing, and Engineering（拉迪信息、计算与工程学院）；Indiana University Bloomington（印第安纳大学伯明顿分校）
PrivSpike: Employing Homomorphic Encryption for Private Inference of Deep Spiking Neural Networks
STAM Center, Ira A. Fulton Schools of Engineering, Arizona State University（STAM中心、伊拉·A·福鲁顿工程学院、亚利桑那州立大学）；Faculty of Technical Sciences, University of Novi Sad（技术科学学院、诺维萨德大学）
