arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.13679cs.CR
收藏

Primer:对加密数据的快速私有Transformer推理

Primer: Fast Private Transformer Inference on Encrypted Data

Mengxin Zheng, Qian Lou, Lei Jiang

摘要:

为基于Transformers的云服务启用隐私保护推理越来越重要。后量子密码技术,例如,全同态加密(FHE)和多方计算(MPC)是支持私有Transformer推理的流行方法。然而,现有的作品仍然遭受禁止计算和通信开销。在这项工作中,我们提出,引物,使一个快速,准确的Transformer加密数据的自然语言处理任务。特别是,引物是由一个混合密码协议优化的注意力为基础的Transformer模型,以及包括计算合并和令牌优先密文包装的技术。在加密语言建模上的综合实验表明,Primer算法达到了最高的准确率,推理延迟比已有方法降低了90.6%~97.5%。

英文摘要:

It is increasingly important to enable privacy-preserving inference for cloud services based on Transformers. Post-quantum cryptographic techniques, e.g., fully homomorphic encryption (FHE), and multi-party computation (MPC), are popular methods to support private Transformer inference. However, existing works still suffer from prohibitively computational and communicational overhead. In this work, we present, Primer, to enable a fast and accurate Transformer over encrypted data for natural language processing tasks. In particular, Primer is constructed by a hybrid cryptographic protocol optimized for attention-based Transformer models, as well as techniques including computation merge and tokens-first ciphertext packing. Comprehensive experiments on encrypted language modeling show that Primer achieves state-of-the-art accuracy and reduces the inference latency by 90.6% ~ 97.5% over previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑