arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2110.13499cs.CR
收藏

SEDML:在机器学习中安全有效地利用分布式知识

SEDML: Securely and Efficiently Harnessing Distributed Knowledge in Machine Learning

Yansong Gao, Qun Li, Yifeng Zheng, Guohong Wang, Jiannan Wei, Mang Su

摘要:

训练高性能的深度学习模型需要大量的数据,这些数据在实践中通常分布在多个数据源中。简单地集中这些多源数据进行培训将引发关键的安全和隐私问题,并且由于越来越严格的数据法规可能会被禁止。为了解决分布式学习中隐私和数据利用之间的紧张关系,最近提出了一种称为教师集合私有聚合(PATE)的机器学习框架。PATE利用来自分布式教师模型的知识(未标记数据集的标记预测)来训练学生模型,从而避免访问分布式数据集。尽管很诱人,但PATE并不为教师模型的个人标签预测提供保护,这仍然会带来隐私风险。在本文中,我们提出了SEDML,一个新的协议,允许安全和有效地利用分布式知识的机器学习。SEDML建立在轻量级加密技术的基础上,为单个标签预测提供强大的保护,并为聚合结果提供差分隐私保证。广泛的评估表明,在提供隐私保护的同时,SEDML保留了明文基线的准确性。同时,SEDML在计算和通信方面的性能分别是最新技术的43倍和1.23倍。

英文摘要:

Training high-performing deep learning models require a rich amount of data which is usually distributed among multiple data sources in practice. Simply centralizing these multi-sourced data for training would raise critical security and privacy concerns, and might be prohibited given the increasingly strict data regulations. To resolve the tension between privacy and data utilization in distributed learning, a machine learning framework called private aggregation of teacher ensembles(PATE) has been recently proposed. PATE harnesses the knowledge (label predictions for an unlabeled dataset) from distributed teacher models to train a student model, obviating access to distributed datasets. Despite being enticing, PATE does not offer protection for the individual label predictions from teacher models, which still entails privacy risks. In this paper, we propose SEDML, a new protocol which allows to securely and efficiently harness the distributed knowledge in machine learning. SEDML builds on lightweight cryptography and provides strong protection for the individual label predictions, as well as differential privacy guarantees on the aggregation results. Extensive evaluations show that while providing privacy protection, SEDML preserves the accuracy as in the plaintext baseline. Meanwhile, SEDML's performance in computing and communication is 43 times and 1.23 times higher than the latest technology, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑