arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.11795cs.LGcs.DC
收藏

低成本的隐私保护去中心化学习

Low-Cost Privacy-Preserving Decentralized Learning

EPFL, Switzerland(苏黎世联邦理工学院,瑞士);Univ Rennes, Inria, CNRS, IRISA, France(里昂大学,法国)

Sayan Biswas, Davide Frey, Romaric Gaudel, Anne-Marie Kermarrec, Dimitri Lerévérend, Rafael Pires, Rishi Sharma, François Taïani

AI总结:

本文提出了一种名为Zip-DL的去中心化学习算法,利用相关噪声在低通信成本下实现隐私保护与高效收敛,在精度与漏洞的权衡上优于现有方法。

摘要:

去中心化学习(DL)是一种新兴的协作机器学习范式,它使网络中的节点能够共同训练模型,而无需共享原始数据或依赖中央服务器。本文提出了Zip-DL,一种隐私感知的DL算法,该算法利用相关噪声来抵御局部对手以实现强大的隐私保护,同时确保以低通信成本实现高效收敛。通过逐步中和分布式平均过程中添加的噪声,Zip-DL将强大的隐私保证与高模型精度相结合。其设计在每次梯度下降迭代中仅需一轮通信,与竞争方法相比显著降低了通信开销。我们建立了关于收敛速度和隐私保证的理论界限。此外,大量实验证明了Zip-DL的实用适用性,使其在精度与漏洞的权衡中优于现有方法。具体而言,与基线DL相比,Zip-DL(i)将成员推理攻击成功率降低了高达35%;与提供相似效用的竞争方法相比,(ii)将攻击效力降低了高达13%;并且在相同威胁模型下,与最先进的隐私保护方法相比,(iii)实现了高达59%的更高精度,从而完全消除了基本的攻击场景。这些结果使Zip-DL成为现实应用中隐私保护去中心化学习的实用且高效的解决方案。

英文摘要:

Decentralized learning (DL) is an emerging paradigm of collaborative machine learning that enables nodes in a network to train models collectively without sharing their raw data or relying on a central server. This paper introduces Zip-DL, a privacy-aware DL algorithm that leverages correlated noise to achieve robust privacy against local adversaries while ensuring efficient convergence at low communication costs. By progressively neutralizing the noise added during distributed averaging, Zip-DL combines strong privacy guarantees with high model accuracy. Its design requires only one communication round per gradient descent iteration, significantly reducing communication overhead compared to competitors. We establish theoretical bounds on both convergence speed and privacy guarantees. Moreover, extensive experiments demonstrating Zip-DL's practical applicability make it outperform state-of-the-art methods in the accuracy vs. vulnerability trade-off. Specifically, Zip-DL (i) reduces membership-inference attack success rates by up to 35% compared to baseline DL, (ii) decreases attack efficacy by up to 13% compared to competitors offering similar utility, and (iii) achieves up to 59% higher accuracy to completely nullify a basic attack scenario, compared to a state-of-the-art privacy-preserving approach under the same threat model. These results position Zip-DL as a practical and efficient solution for privacy-preserving decentralized learning in real-world applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑