选择性网络线性化实现高效的私有推理
Selective Network Linearization for Efficient Private Inference
摘要：
私有推理（PI）允许直接对加密安全数据进行推理。虽然有望解决许多隐私问题，但由于极端的运行时间，它的使用受到限制。与延迟由FLOP主导的明文推理不同，在PI中，非线性函数（即ReLU）是瓶颈。因此，实际的PI需要新的ReLU感知优化。为了减少PI延迟，我们提出了一种基于梯度的算法，该算法选择性地线性化ReLU，同时保持预测精度。我们评估我们的算法在几个标准的PI基准。结果表明，与当前技术水平相比，$4.25\%$具有更高的准确性（50 K时的iso-ReLU计数）或$2.2\times$具有更少的延迟（70\%时的iso-accuracy），并在延迟-准确性空间上推进了Pareto边界。为了补充实证结果，我们提出了一个“没有免费的午餐”定理，揭示了如何以及何时网络线性化是可能的，同时保持预测精度。公共代码可在\url{https://github.com/NYU-DICE-Lab/selective_network_linearization}获得。
英文摘要：
Private inference (PI) enables inference directly on cryptographically secure data.While promising to address many privacy issues, it has seen limited use due to extreme runtimes. Unlike plaintext inference, where latency is dominated by FLOPs, in PI non-linear functions (namely ReLU) are the bottleneck. Thus, practical PI demands novel ReLU-aware optimizations. To reduce PI latency we propose a gradient-based algorithm that selectively linearizes ReLUs while maintaining prediction accuracy. We evaluate our algorithm on several standard PI benchmarks. The results demonstrate up to $4.25\%$ more accuracy (iso-ReLU count at 50K) or $2.2\times$ less latency (iso-accuracy at 70\%) than the current state of the art and advance the Pareto frontier across the latency-accuracy space. To complement empirical results, we present a "no free lunch" theorem that sheds light on how and when network linearization is possible while maintaining prediction accuracy. Public code is available at \url{https://github.com/NYU-DICE-Lab/selective_network_linearization}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
