arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2207.03384cs.CRcs.LG
收藏

同态加密下机器学习的高效剪枝

Efficient Pruning for Machine Learning Under Homomorphic Encryption

IBM Research – Israel(IBM以色列研究院);IBM T. J. Watson Research Center(IBM T. J. 沃森研究中心)

Ehud Aharoni, Moran Baruch, Pradip Bose, Alper Buyuktosunoglu, Nir Drucker, Subhankar Pal, Tomer Pelleg, Kanthi Sarpatwar, Hayim Shaul, Omri Soceanu, Roman Vaculin

AI总结:

提出HE-PEx框架,结合瓦片张量打包、置换剪枝和扩展恢复,在HElayers上实现PPML推理加速10-35%、内存降低17-35%,密文减少33-65%,精度损失小于2.5%。

摘要:

隐私保护机器学习(PPML)解决方案正日益普及。其中,许多方案依赖于同态加密(HE),该技术能保障模型和数据机密性,但代价是较大的延迟和内存需求。剪枝神经网络(NN)参数可改善明文机器学习中的延迟和内存,但直接应用于基于HE的PPML时效果甚微。我们提出一个名为HE-PEx的框架,该框架在称为瓦片张量的打包技术之上,包含新的剪枝方法,旨在降低PPML推理的延迟和内存。HE-PEx使用置换来剪除额外的密文,并使用扩展来恢复推理损失。我们展示了我们的方法在PPML任务(即图像压缩、去噪和分类)中剪枝NN中全连接层和卷积层的有效性,这些任务使用自编码器、多层感知器(MLP)和卷积神经网络(CNN)实现。我们在名为HElayers的框架之上实现并部署了我们的网络,结果显示,与未剪枝网络相比,推理速度提升了10-35%,内存需求降低了17-35%,对应密文数量减少了33-65%,而推理精度相对于未剪枝网络的下降在2.5%以内。与最先进的PPML剪枝技术相比,在相同的精度下降限制下,我们的技术生成的网络平均密文数量减少了70%。

英文摘要:

Privacy-preserving machine learning (PPML) solutions are gaining widespread popularity. Among these, many rely on homomorphic encryption (HE) that offers confidentiality of the model and the data, but at the cost of large latency and memory requirements. Pruning neural network (NN) parameters improves latency and memory in plaintext ML but has little impact if directly applied to HE-based PPML. We introduce a framework called HE-PEx that comprises new pruning methods, on top of a packing technique called tile tensors, for reducing the latency and memory of PPML inference. HE-PEx uses permutations to prune additional ciphertexts, and expansion to recover inference loss. We demonstrate the effectiveness of our methods for pruning fully-connected and convolutional layers in NNs on PPML tasks, namely, image compression, denoising, and classification, with autoencoders, multilayer perceptrons (MLPs) and convolutional neural networks (CNNs). We implement and deploy our networks atop a framework called HElayers, which shows a 10-35% improvement in inference speed and a 17-35% decrease in memory requirement over the unpruned network, corresponding to 33-65% fewer ciphertexts, within a 2.5% degradation in inference accuracy over the unpruned network. Compared to the state-of-the-art pruning technique for PPML, our techniques generate networks with 70% fewer ciphertexts, on average, for the same degradation limit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑