SPriFed-OMP:一种用于稀疏基恢复的差分隐私联邦学习算法
SPriFed-OMP: A Differentially Private Federated Learning Algorithm for Sparse Basis Recovery
AI总结:
针对高维联邦场景下稀疏基恢复的隐私保护难题,本文将 OMP 改造为结合 SMPC 与 DP 的 SPriFed-OMP,并提出梯度私有化增强版,在 n=O(√p) 时实现更优准确性—隐私权衡。
摘要:
当模型维度数 p 远大于样本数 n 时,稀疏基恢复是一个经典且重要的统计学习问题。然而,在联邦学习(Federated Learning,FL)场景下研究稀疏基恢复的工作仍然很少,而在该场景中还必须同时保护客户端数据的差分隐私(DP)。特别是,现有 DP-FL 算法(例如 DP-SGD)的性能保证在 p ≫ n 时会显著下降,因此它们无法准确学习真实的底层稀疏模型。在本工作中,我们为 FL 场景开发了一种新的差分隐私稀疏基恢复算法,称为 SPriFed-OMP。SPriFed-OMP 将 OMP(正交匹配追踪)转换到 FL 场景。此外,它结合了 SMPC(安全多方计算)和 DP,以确保只需添加少量噪声即可实现差分隐私。因此,SPriFed-OMP 仅用 n = O(√p) 个样本就能高效恢复线性模型的真实稀疏基。我们进一步提出了方法的增强版本 SPriFed-OMP-GRAD,它基于梯度私有化,可提升 SPriFed-OMP 的性能。我们的理论分析和实证结果表明,SPriFed-OMP 与 SPriFed-OMP-GRAD 都能在很少的步骤内终止,并且在准确性—隐私权衡方面显著优于此前最先进的 DP-FL 解决方案。
英文摘要:
Sparse basis recovery is a classical and important statistical learning problem when the number of model dimensions $p$ is much larger than the number of samples $n$. However, there has been little work that studies sparse basis recovery in the Federated Learning (FL) setting, where the client data's differential privacy (DP) must also be simultaneously protected. In particular, the performance guarantees of existing DP-FL algorithms (such as DP-SGD) will degrade significantly when $p \gg n$, and thus, they will fail to learn the true underlying sparse model accurately. In this work, we develop a new differentially private sparse basis recovery algorithm for the FL setting, called SPriFed-OMP. SPriFed-OMP converts OMP (Orthogonal Matching Pursuit) to the FL setting. Further, it combines SMPC (secure multi-party computation) and DP to ensure that only a small amount of noise needs to be added in order to achieve differential privacy. As a result, SPriFed-OMP can efficiently recover the true sparse basis for a linear model with only $n = O(\sqrt{p})$ samples. We further present an enhanced version of our approach, SPriFed-OMP-GRAD based on gradient privatization, that improves the performance of SPriFed-OMP. Our theoretical analysis and empirical results demonstrate that both SPriFed-OMP and SPriFed-OMP-GRAD terminate in a small number of steps, and they significantly outperform the previous state-of-the-art DP-FL solutions in terms of the accuracy-privacy trade-off.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。