在同态加密与差分隐私的联邦学习中结合模型检查与可用性

中文摘要 AI 辅助

去中心化数据的日益普及引发了对联邦学习的兴趣，联邦学习使得协作模型训练成为可能，而无需客户端共享其敏感的本地数据。然而，仅靠联邦学习不足以充分保护敏感训练数据，通常需要结合隐私保护技术，如差分隐私和同态加密。尽管这些技术功能强大，但它们通过不同的机制解决各自关注的问题，因此仅依赖其中一种可能不足以应对联邦学习相关的挑战，或者在实际中不可行。在本工作中，我们提出了一种隐私保护的联邦学习框架，该框架将基于同态加密的训练与基于差分隐私的模型检查和发布相结合。我们采用基于马尔可夫链蒙特卡洛的贝叶斯隐私估计方法来估计所提出框架的隐私。我们的结果表明，与仅依赖差分隐私进行训练的基线方法相比，该方法提高了模型效用和估计隐私。在我们使用FEMNIST数据集的实验中，训练结束时，我们的方法达到了测试损失$1.09$，而仅差分隐私方法为$2.37$，同时提供了更强的估计隐私保护，隐私参数$\u03b5$的估计后验均值为$4.32$，而仅差分隐私方法为$7.26$。我们还表明，间歇性模型监控可以保留加密的训练轨迹，同时在我们评估的实验设置下，提供与仅差分隐私方法相当或更强的估计隐私。

英文摘要

The increasing prevalence of decentralized data has led to a growing interest in federated learning, which enables collaborative model training without clients sharing their sensitive local data. However, FL alone does not sufficiently protect sensitive training data and is generally coupled with privacy-preserving techniques, such as differential privacy and homomorphic encryption. Although powerful, these techniques address separate concerns via different mechanisms, so relying on just one might prove insufficient or impractical for addressing challenges associated with federated learning. In this work, we propose a privacy-preserving federated learning framework that combines homomorphic encryption-based training with differential privacy-based model inspection and release. We adopt a Markov chain Monte Carlo-based Bayesian privacy estimation method to estimate the privacy of our proposed framework. Our results show that this method improves both model utility and estimated privacy over the baseline method that relies solely on differential privacy for training. In our experiments with the FEMNIST dataset, by the end of training, our method reaches a test loss of $1.09$, compared to $2.37$ for the differential privacy-only approach, while providing stronger estimated privacy protection, with the estimated posterior mean of the privacy parameter $ε$ of $4.32$, compared to $7.26$ for the differential privacy-only approach. We also show that intermittent model monitoring can preserve the encrypted training trajectory while, under our evaluated experimental setting, providing estimated privacy comparable to or stronger than the differential privacy-only approach.

发表机构

• Sabancı University（萨班吉大学）

机构由 AI 辅助整理，请以论文原文为准。