arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.11707cs.LGcs.CRcs.ITmath.ITstat.ML
收藏

打破通信-隐私-准确性三难困境

Breaking the Communication-Privacy-Accuracy Trilemma

Stanford University(斯坦福大学);Google(谷歌)

Wei-Ning Chen, Peter Kairouz, Ayfer Özgür

摘要:

分布式学习和估计中的两个主要挑战是:1)保护本地样本的隐私; 2)将它们有效地传输到中央服务器,同时实现端到端任务的高精度。虽然在最近的文献中,人们对分别解决这些挑战中的每一个都有很大的兴趣,但同时解决这两个挑战的治疗方法在很大程度上仍然缺失。在本文中,我们开发了新的编码和解码机制,同时实现最佳的隐私和通信效率在各种规范设置。特别是,我们考虑的问题下的平均值估计和频率估计$\varepsilon$ -本地差分隐私和$b$位通信的限制。对于均值估计,我们提出了一个基于Kashin表示和随机抽样的方案,在这两个约束下具有阶最优估计误差。对于频率估计,我们提出了一种机制,利用沃尔什-哈达玛矩阵的递归结构,并实现了所有隐私级别和通信预算的顺序最优估计误差。作为一个副产品,我们还构建了一个分布估计机制,是所有的隐私制度和通信约束的速率最优,扩展最近的工作,仅限于$b=1$和$\varepsilon=O(1)$。我们的研究结果表明,联合隐私和通信约束下的智能编码可以产生的性能,匹配的最佳精度下实现单独的约束。

英文摘要:

Two major challenges in distributed learning and estimation are 1) preserving the privacy of the local samples; and 2) communicating them efficiently to a central server, while achieving high accuracy for the end-to-end task. While there has been significant interest in addressing each of these challenges separately in the recent literature, treatments that simultaneously address both challenges are still largely missing. In this paper, we develop novel encoding and decoding mechanisms that simultaneously achieve optimal privacy and communication efficiency in various canonical settings. In particular, we consider the problems of mean estimation and frequency estimation under $\varepsilon$-local differential privacy and $b$-bit communication constraints. For mean estimation, we propose a scheme based on Kashin's representation and random sampling, with order-optimal estimation error under both constraints. For frequency estimation, we present a mechanism that leverages the recursive structure of Walsh-Hadamard matrices and achieves order-optimal estimation error for all privacy levels and communication budgets. As a by-product, we also construct a distribution estimation mechanism that is rate-optimal for all privacy regimes and communication constraints, extending recent work that is limited to $b=1$ and $\varepsilon=O(1)$. Our results demonstrate that intelligent encoding under joint privacy and communication constraints can yield a performance that matches the optimal accuracy achievable under either constraint alone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑