HETAL：基于同态加密的高效隐私保护迁移学习
HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption
- University of California, Berkeley（加州大学伯克利分校）
- CryptoLab Inc.（CryptoLab公司）
- Inha University（仁荷大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究针对迁移学习在机器学习即服务场景下的隐私保护问题，提出基于CKKS同态加密的HETAL算法，实现了与非加密训练精度相当的加密训练，训练效率大幅提升。
AI中文摘要：
迁移学习是针对数据稀缺问题高效训练机器学习模型的事实标准方法，其通过在大型数据集上预训练的模型中添加并微调新的分类层来实现。尽管此前大量研究提出使用同态加密解决机器学习即服务场景下迁移学习中的数据隐私问题，但大多仅聚焦于加密推理。本研究提出HETAL——一种高效的基于同态加密的迁移学习算法，该算法采用CKKS同态加密方案对客户端数据进行加密，从而在训练任务中保护客户端隐私。HETAL是首个严格实现加密训练的实用方案，采用基于验证的早停机制，且能达到非加密训练的准确率。我们提出了一种高效的加密矩阵乘法算法，其速度比现有方法快1.8至323倍，还提出了一种覆盖范围更广的高精度softmax近似算法。在五个知名基准数据集上的实验结果显示，总训练时长为567-3442秒，不足一小时。
英文摘要：
Transfer learning is a de facto standard method for efficiently training machine learning models for data-scarce problems by adding and fine-tuning new classification layers to a model pre-trained on large datasets. Although numerous previous studies proposed to use homomorphic encryption to resolve the data privacy issue in transfer learning in the machine learning as a service setting, most of them only focused on encrypted inference. In this study, we present HETAL, an efficient Homomorphic Encryption based Transfer Learning algorithm, that protects the client's privacy in training tasks by encrypting the client data using the CKKS homomorphic encryption scheme. HETAL is the first practical scheme that strictly provides encrypted training, adopting validation-based early stopping and achieving the accuracy of nonencrypted training. We propose an efficient encrypted matrix multiplication algorithm, which is 1.8 to 323 times faster than prior methods, and a highly precise softmax approximation algorithm with increased coverage. The experimental results for five well-known benchmark datasets show total training times of 567-3442 seconds, which is less than an hour.
