xMLP：利用独占平方激活实现私有推理的革命性变革
xMLP: Revolutionizing Private Inference with Exclusive Square Activation
AI总结：
本文提出xMLP架构，仅用平方激活替代ReLU，解决私有推理延迟问题，在CIFAR-100和ImageNet上实现与ReLU相当精度，且PI速度提升7倍，GPU卸载时快700倍。
摘要：
私有推理（PI）通过利用多方计算（MPC）和同态加密（HE）等密码学原语，使深度神经网络（DNN）能够在私有数据上工作而不泄露敏感信息。然而，在现有PI系统中，DNN中使用ReLU等非线性激活函数会导致不切实际的高PI延迟，因为ReLU需要使用昂贵的MPC计算，如混淆电路。由于平方激活可以通过Beaver三元组处理，其速度比ReLU快数百倍，因此它们对PI任务更友好，但使用它们会导致模型准确率显著下降。本文首先探讨了使用平方激活后准确率下降的原因，并得出结论，这是由于“信息复合”效应所致。基于这一见解，我们提出了xMLP，一种新颖的DNN架构，它专门使用平方激活，同时在准确率和效率上与基于ReLU的DNN保持相当。我们在CIFAR-100和ImageNet上的实验表明，xMLP模型在激活层和参数更少的情况下，始终优于ResNet模型，同时与其基于ReLU的变体保持一致的性能。值得注意的是，与最先进的PI模型相比，xMLP表现出优越的性能，准确率提高了0.58%，PI速度提高了7倍。此外，在保持相同PI延迟的同时，它实现了4.96%的显著准确率提升。当将PI卸载到GPU时，xMLP比之前最先进的PI模型快700倍，同时具有相当的准确率。
英文摘要：
Private Inference (PI) enables deep neural networks (DNNs) to work on private data without leaking sensitive information by exploiting cryptographic primitives such as multi-party computation (MPC) and homomorphic encryption (HE). However, the use of non-linear activations such as ReLU in DNNs can lead to impractically high PI latency in existing PI systems, as ReLU requires the use of costly MPC computations, such as Garbled Circuits. Since square activations can be processed by Beaver's triples hundreds of times faster compared to ReLU, they are more friendly to PI tasks, but using them leads to a notable drop in model accuracy. This paper starts by exploring the reason for such an accuracy drop after using square activations, and concludes that this is due to an "information compounding" effect. Leveraging this insight, we propose xMLP, a novel DNN architecture that uses square activations exclusively while maintaining parity in both accuracy and efficiency with ReLU-based DNNs. Our experiments on CIFAR-100 and ImageNet show that xMLP models consistently achieve better performance than ResNet models with fewer activation layers and parameters while maintaining consistent performance with its ReLU-based variants. Remarkably, when compared to state-of-the-art PI Models, xMLP demonstrates superior performance, achieving a 0.58% increase in accuracy with 7x faster PI speed. Moreover, it delivers a significant accuracy improvement of 4.96% while maintaining the same PI latency. When offloading PI to the GPU, xMLP is up to 700x faster than the previous state-of-the-art PI model with comparable accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
