联邦学习的通用协调差分隐私机制：提升准确性与收敛性
Universally Harmonizing Differential Privacy Mechanisms for Federated Learning: Boosting Accuracy and Convergence
AI总结：
提出首个通用协调任意随机化机制与DP-SGD的差分隐私联邦学习框架UDP-FL，基于Rényi差分隐私协调预算并利用模式连接性分析收敛性，显著提升了模型准确率、收敛性及抵御推理攻击的能力。
摘要：
差分隐私联邦学习（DP-FL）是一种很有前景的技术，能够在确保客户端可证明隐私的同时进行协作模型训练。然而，优化隐私与准确性之间的权衡仍然是一个关键挑战。据我们所知，我们提出了首个DP-FL框架（即UDP-FL），该框架将任何随机化机制（例如最优机制）与Gaussian Moments Accountant（即DP-SGD）通用协调，以显著提升准确性和收敛性。具体而言，UDP-FL通过减轻对Gaussian噪声的依赖来展现增强的模型性能。这一转换中的关键中介变量是Rényi Differential Privacy概念，它被精心用于协调隐私预算。我们还提出了一种创新方法，基于模式连接性分析对DP-FL（包括我们的UDP-FL）的收敛性进行理论分析。此外，我们通过与最先进（SOTA）方法进行基准对比的大量实验评估了我们的UDP-FL，证明了其在隐私保证和模型性能方面均表现出卓越的性能。值得注意的是，UDP-FL对不同的推理攻击展现出强大的抵抗力，表明在联邦学习环境中保护敏感数据方面取得了重大进展。
英文摘要：
Differentially private federated learning (DP-FL) is a promising technique for collaborative model training while ensuring provable privacy for clients. However, optimizing the tradeoff between privacy and accuracy remains a critical challenge. To our best knowledge, we propose the first DP-FL framework (namely UDP-FL), which universally harmonizes any randomization mechanism (e.g., an optimal one) with the Gaussian Moments Accountant (viz. DP-SGD) to significantly boost accuracy and convergence. Specifically, UDP-FL demonstrates enhanced model performance by mitigating the reliance on Gaussian noise. The key mediator variable in this transformation is the Rényi Differential Privacy notion, which is carefully used to harmonize privacy budgets. We also propose an innovative method to theoretically analyze the convergence for DP-FL (including our UDP-FL ) based on mode connectivity analysis. Moreover, we evaluate our UDP-FL through extensive experiments benchmarked against state-of-the-art (SOTA) methods, demonstrating superior performance on both privacy guarantees and model performance. Notably, UDP-FL exhibits substantial resilience against different inference attacks, indicating a significant advance in safeguarding sensitive data in federated learning environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
