arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.16675cs.CRcs.DBcs.LG
收藏

CryptoTrain:在加密数据集上的快速安全训练

CryptoTrain: Fast Secure Training on Encrypted Dataset

University of Central Florida(中佛罗里达大学);University of Pittsburgh(匹兹堡大学)

Jiaqi Xue, Yancheng Zhang, Yanshan Wang, Xueqiang Wang, Hao Zheng, Qian Lou

AI总结:

针对安全训练中FHE自举操作的高昂开销,本文提出CryptoTrain框架,结合FHE与OT协议,并引入CCMul-Precompute和相关多项式卷积技术,使训练时间相比先前方法减少约5.3倍。

摘要:

安全训练在保护数据和模型权重机密性的同时,通常会产生显著的训练开销。传统的基于全同态加密(FHE)的非交互式训练模型受制于计算要求极高的自举操作而负担沉重。为开发高效的安全训练系统,我们建立了一个基础框架 CryptoTrain-B,利用混合密码协议将 FHE 与不经意传输(OT)结合,分别处理线性和非线性操作。这一整合消除了高成本的自举需求。尽管 CryptoTrain-B 创立了新的性能基准,但降低其训练开销仍至关重要。我们发现密文-密文乘法(CCMul)是涉及加密输入和模型操作中的关键瓶颈。我们的解决方案 CCMul-Precompute 技术在离线阶段预计算 CCMul,并在私有训练期间转而使用资源消耗较少的密文-明文乘法(CPMul)。此外,FHE 系统中传统的多项式卷积倾向于将无关和冗余的值编码到多项式槽中,需要额外的多项式和密文进行输入表示,从而导致额外的乘法。针对此问题,我们引入了相关多项式卷积,仅将相关输入值编码到多项式中,从而大幅减少了计算量和开销。通过将 CCMul-Precompute 和相关多项式卷积集成到 CryptoTrain-B 中,我们构建了一个快速高效的安全训练框架 CryptoTrain。大量实验表明,与先前方法相比,CryptoTrain 实现了约 5.3 倍的训练时间缩减。

英文摘要:

Secure training, while protecting the confidentiality of both data and model weights, typically incurs significant training overhead. Traditional Fully Homomorphic Encryption (FHE)-based non-inter-active training models are heavily burdened by computationally demanding bootstrapping. To develop an efficient secure training system, we established a foundational framework, CryptoTrain-B, utilizing a hybrid cryptographic protocol that merges FHE with Oblivious Transfer (OT) for handling linear and non-linear operations, respectively. This integration eliminates the need for costly bootstrapping. Although CryptoTrain-B sets a new baseline in performance, reducing its training overhead remains essential. We found that ciphertext-ciphertext multiplication (CCMul) is a critical bottleneck in operations involving encrypted inputs and models. Our solution, the CCMul-Precompute technique, involves precomputing CCMul offline and resorting to the less resource-intensive ciphertext-plaintext multiplication (CPMul) during private training. Furthermore, conventional polynomial convolution in FHE systems tends to encode irrelevant and redundant values into polynomial slots, necessitating additional polynomials and ciphertexts for input representation and leading to extra multiplications. Addressing this, we introduce correlated polynomial convolution, which encodes only related input values into polynomials, thus drastically reducing the number of computations and overheads. By integrating CCMul-Precompute and correlated polynomial convolution into CryptoTrain-B, we facilitate a rapid and efficient secure training framework, CryptoTrain. Extensive experiments demonstrate that CryptoTrain achieves a ~5.3X training time reduction compared to prior methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑