同态 WiSARD：加密数据上的高效无权神经网络训练
Homomorphic WiSARDs: Efficient Weightless Neural Network training over encrypted data
AI总结：
本文提出对 WiSARD 及无权神经网络进行同态评估，实现加密数据上的高效训练与推理，在 MNIST 和 HAM10000 上相比现有方法 Glyph 实现最高 1200 倍加速且精度损失极小。
摘要：
机器学习算法的广泛应用日益引起数据隐私研究界的关注，许多研究者致力于开发隐私保护技术。在现有方法中，机器学习算法的同态评估通过对加密数据直接执行运算而脱颖而出，能够提供强有力的机密性保障。推理算法的同态评估即使对于相对较深的卷积神经网络（CNNs）也是实用的。然而，训练仍然是一个重大挑战，目前的解决方案往往退而采用轻量级算法，这些算法可能不适合解决更复杂的问题，如图像识别。本文介绍了 Wilkie、Stonham 和 Aleksander 识别设备（WiSARD）及后续无权神经网络（WNNs）的同态评估，用于在加密数据上进行训练和推理。与 CNN 相比，WNN 在精度下降相对较小的情况下提供了更好的性能。我们为此开发了一个完整框架，其中包括若干可能具有独立价值的构建模块。我们的框架在 MNIST 数据集上仅经过 3.5 分钟的加密训练（多线程）即达到 91.7% 的准确率，在 3.5 小时内可提升至 93.8%。对于 HAM10000 数据集，我们在仅 1.5 分钟内达到 67.9% 的准确率，1 小时后提升至 69.9%。与 CNN 训练同态评估的最新成果 Glyph（Lou 等人，NeurIPS 2020）相比，这些结果代表了最高达 1200 倍的加速，而准确率损失最多仅为 5.4%。对于 HAM10000，我们甚至实现了 0.65% 的准确率提升，同时比 Glyph 快 60 倍。我们还提供了小规模加密训练的解决方案。在一台内存使用量低于 200MB 的台式机上以单线程运行，我们在 12 分钟内训练了超过 1000 张 MNIST 图像，或仅用 11 秒即可完成对整个 Wisconsin Breast Cancer 数据集的训练。
英文摘要：
The widespread application of machine learning algorithms is a matter of increasing concern for the data privacy research community, and many have sought to develop privacy-preserving techniques for it. Among existing approaches, the homomorphic evaluation of ML algorithms stands out by performing operations directly over encrypted data, enabling strong guarantees of confidentiality. The homomorphic evaluation of inference algorithms is practical even for relatively deep Convolution Neural Networks (CNNs). However, training is still a major challenge, with current solutions often resorting to lightweight algorithms that can be unfit for solving more complex problems, such as image recognition. This work introduces the homomorphic evaluation of Wilkie, Stonham, and Aleksander's Recognition Device (WiSARD) and subsequent Weightless Neural Networks (WNNs) for training and inference on encrypted data. Compared to CNNs, WNNs offer better performance with a relatively small accuracy drop. We develop a complete framework for it, including several building blocks that can be of independent interest. Our framework achieves 91.7% accuracy on the MNIST dataset after only 3.5 minutes of encrypted training (multi-threaded), going up to 93.8% in 3.5 hours. For the HAM10000 dataset, we achieve 67.9% accuracy in just 1.5 minutes, going up to 69.9% after 1 hour. Compared to the state of the art on the HE evaluation of CNN training, Glyph (Lou et al., NeurIPS 2020), these results represent a speedup of up to 1200 times with an accuracy loss of at most 5.4%. For HAM10000, we even achieved a 0.65% accuracy improvement while being 60 times faster than Glyph. We also provide solutions for small-scale encrypted training. In a single thread on a desktop machine using less than 200MB of memory, we train over 1000 MNIST images in 12 minutes or over the entire Wisconsin Breast Cancer dataset in just 11 seconds.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
