arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.13071cs.CR
收藏

LOCKS:用户差异私有和联邦最优客户端采样

LOCKS: User Differentially Private and Federated Optimal Client Sampling

Ajinkya K Mulay

摘要:

随着隐私法的变化,客户端数据通常需要保留在设备上,而不是发送到服务器。因此,大多数处理发生在设备上,只有更改的元素被发送到服务器。这种机制是通过利用差分隐私和联邦学习开发的。差分隐私向客户端输出添加了噪声,因此降低了每次迭代的质量。这种分布式设置增加了一层复杂性以及额外的通信和性能开销。这些成本是每轮的附加成本,因此我们需要减少迭代次数。在这项工作中,我们提供了一个分析框架,研究基于梯度的分布式算法的收敛保证。我们表明,我们的私人算法最大限度地减少了预期的梯度方差约$d^2$轮,其中d是模型的维数。我们讨论并提出了新的方法来提高收敛速度,以尽量减少使用重要性采样(IS)和梯度多样性的开销。最后,我们提供了替代框架,可能更适合利用客户端采样技术,如IS和梯度多样性。

英文摘要:

With changes in privacy laws, there is often a hard requirement for client data to remain on the device rather than being sent to the server. Therefore, most processing happens on the device, and only an altered element is sent to the server. Such mechanisms are developed by leveraging differential privacy and federated learning. Differential privacy adds noise to the client outputs and thus deteriorates the quality of each iteration. This distributed setting adds a layer of complexity and additional communication and performance overhead. These costs are additive per round, so we need to reduce the number of iterations. In this work, we provide an analytical framework for studying the convergence guarantees of gradient-based distributed algorithms. We show that our private algorithm minimizes the expected gradient variance by approximately $d^2$ rounds, where d is the dimensionality of the model. We discuss and suggest novel ways to improve the convergence rate to minimize the overhead using Importance Sampling (IS) and gradient diversity. Finally, we provide alternative frameworks that might be better suited to exploit client sampling techniques like IS and gradient diversity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑