Two major challenges in distributed learning and estimation are 1) preserving the privacy of the local samples; and 2) communicating them efficiently to a central server, while achieving high accuracy for the end-to-end task. While there has been significant interest in addressing each of these challenges separately in the recent literature, treatments that simultaneously address both challenges are still largely missing. In this paper, we develop novel encoding and decoding mechanisms that simultaneously achieve optimal privacy and communication efficiency in various canonical settings.
In particular, we consider the problems of mean estimation and frequency estimation under $\varepsilon$-local differential privacy and $b$-bit communication constraints. For mean estimation, we propose a scheme based on Kashin's representation and random sampling, with order-optimal estimation error under both constraints. For frequency estimation, we present a mechanism that leverages the recursive structure of Walsh-Hadamard matrices and achieves order-optimal estimation error for all privacy levels and communication budgets. As a by-product, we also construct a distribution estimation mechanism that is rate-optimal for all privacy regimes and communication constraints, extending recent work that is limited to $b=1$ and $\varepsilon=O(1)$. Our results demonstrate that intelligent encoding under joint privacy and communication constraints can yield a performance that matches the optimal accuracy achievable under either constraint alone.
Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearning
University of Notre Dame(圣母大学);University of Pennsylvania(宾夕法尼亚大学);Dartmouth College(达特茅斯学院);Georgia Institute of Technology(佐治亚理工学院);George Mason University(乔治梅森大学)
OSNIP: Breaking the Privacy-Utility-Efficiency Trilemma in LLM Inference via Obfuscated Semantic Null Space
Shanghai Key Laboratory of Computer Software Testing and Evaluating(上海计算机软件测试与评估重点实验室);Shanghai Normal University(上海师范大学);Shanghai Polytechnic University(上海理工大学);TrustAI Pte. Ltd.(TrustAI有限公司)
Breaking the MoE LLM Trilemma: Dynamic Expert Clustering with Structured Compression
Guangzhou University(广州大学);Institue of Automation Chinese Academy of Sciences(中国科学院自动化研究所);ByteDance Inc.(字节跳动公司);University of Science and Technology Beijing(北京科技大学)
Can We Predict Your Next Move Without Breaking Your Privacy?
Eudoxia Research University(欧多克斯研究大学);Jamia Hamdard(贾迈亚·哈姆达德大学);Macquarie University(麦考瑞大学);TERI School of Advanced Studies(TERI高级研究学校);Cornell University(康奈尔大学);Stanford University(斯坦福大学)
Extracting Spatiotemporal Data from Gradients with Large Language Models
Tokyo Institute of Technology(东京工业大学);The University of Tokyo(东京大学);Xidian University(西安电子科技大学);National Institute of Information and Communications Technology(日本信息通信研究机构);Osaka Seikei University(大阪成蹊大学)
Mjolnir: Breaking the Shield of Perturbation-Protected Gradients via Adaptive Diffusion
The Hong Kong Polytechnic University(香港理工大学);Wuhan University of Technology(武汉理工大学);Shenzhen University(深圳大学);The Hong Kong University of Science and Technology(香港科技大学)
DiffMI: Breaking Face Recognition Privacy via Diffusion-Driven Training-Free Model Inversion
Echizen Lab, National Institute of Informatics (NII)(筑前实验室,日本国立信息学研究所);Shanghai Jiao Tong University(上海交通大学);Institute of Information Science, Academia Sinica(学术院信息科学研究所)
Privacy-Preserving EHR Data Transformation via Geometric Operators: A Human-AI Co-Design Technical Report
Hong Kong Institute of AI for Science (HKAI-Sci), City University of Hong Kong(香港人工智能科学研究院(HKAI-Sci),香港城市大学);Department of Biostatistics, City University of Hong Kong(香港城市大学生物统计学系);Institute of Digital Medicine, City University of Hong Kong(香港城市大学数字医学研究院);YIDU TECH co. Ltd.(YIDU科技有限公司)