机器学习模型通常在分布于不同“孤岛”（如医院）的敏感数据（如医疗记录和种族/性别）上进行训练。这些联邦学习模型随后可能被用于做出具有重大影响的决策，例如分配医疗资源。在此背景下出现了两个关键挑战：（i）保护每个人的数据隐私，即使其他孤岛或具有访问中央服务器权限的攻击者试图推断这些数据；（ii）确保决策对不同人口群体（如种族/性别）是公平的。在本文中，我们开发了一种用于隐私与公平联邦学习（FL）的新颖算法。我们的算法满足孤岛间记录级差分隐私（ISRL-DP），这是一种隐私 FL 的强概念，要求孤岛 i 发送的消息对所有 i 满足记录级差分隐私。我们的框架可用于促进不同的公平性概念，包括人口统计均等和均等机会。我们证明了我们的算法在损失函数的温和平滑假设下收敛，而先前的工作需要强凸性才能收敛。作为我们分析的副产品，我们获得了 ISRL-DP 非凸-强凹极小极大 FL 的首个收敛保证。实验表明，我们的算法在不同隐私级别下实现了最先进的公平性-准确性权衡。
英文摘要：
Machine learning models are often trained on sensitive data (e.g., medical records and race/gender) that is distributed across different "silos" (e.g., hospitals). These federated learning models may then be used to make consequential decisions, such as allocating healthcare resources. Two key challenges emerge in this setting: (i) maintaining the privacy of each person's data, even if other silos or an adversary with access to the central server tries to infer this data; (ii) ensuring that decisions are fair to different demographic groups (e.g., race/gender). In this paper, we develop a novel algorithm for private and fair federated learning (FL). Our algorithm satisfies inter-silo record-level differential privacy (ISRL-DP), a strong notion of private FL requiring that silo i's sent messages satisfy record-level differential privacy for all i. Our framework can be used to promote different fairness notions, including demographic parity and equalized odds. We prove that our algorithm converges under mild smoothness assumptions on the loss function, whereas prior work required strong convexity for convergence. As a byproduct of our analysis, we obtain the first convergence guarantee for ISRL-DP nonconvex-strongly concave min-max FL. Experiments demonstrate the state-of-the-art fairness-accuracy tradeoffs of our algorithm across different privacy levels.
This paper presents a differentially private algorithm for linear regression learning in a decentralized fashion. Under this algorithm, privacy budget is theoretically derived, in addition to that the solution error is shown to be bounded by $O(t)$ for $O(1/t)$ descent step size and $O(\exp(t^{1-e}))$ for $O(t^{-e})$ descent step size.
Differentially private (DP) synthetic data is a promising approach to maximizing the utility of data containing sensitive information. Due to the suppression of underrepresented classes that is often required to achieve privacy, however, it may be in conflict with fairness. We evaluate four DP synthesizers and present empirical results indicating that three of these models frequently degrade fairness outcomes on downstream binary classification tasks. We draw a connection between fairness and the proportion of minority groups present in the generated synthetic data, and find that training synt…
The Hong Kong Polytechnic University（香港理工大学）；University of Toronto（多伦多大学）；City University of Hong Kong（香港城市大学）
Many real-world networks are inherently decentralized. For example, in social networks, each user maintains a local view of a social graph, such as a list of friends and her profile. It is typical to collect these local views of social graphs and conduct graph learning tasks. However, learning over graphs can raise privacy concerns as these local views often contain sensitive information. In this paper, we seek to ensure private graph learning on a decentralized network graph. Towards this objective, we propose {\em Solitude}, a new privacy-preserving learning framework based on graph neural n…
Duke University（杜克大学）；Hong Kong University of Science and Technology（香港科技大学）；Texas A&M University（得克萨斯农工大学）
Applying machine learning (ML) in design flow is a popular trend in EDA with various applications from design quality predictions to optimizations. Despite its promise, which has been demonstrated in both academic researches and industrial tools, its effectiveness largely hinges on the availability of a large amount of high-quality training data. In reality, EDA developers have very limited access to the latest design data, which is owned by design companies and mostly confidential. Although one can commission ML model training to a design company, the data of a single company might be still i…
Recently, lots of algorithms have been proposed for learning a fair classifier from decentralized data. However, many theoretical and algorithmic questions remain open. First, is federated learning necessary, i.e., can we simply train locally fair classifiers and aggregate them? In this work, we first propose a new theoretical framework, with which we demonstrate that federated learning can strictly boost model fairness compared with such non-federated algorithms. We then theoretically and empirically show that the performance tradeoff of FedAvg-based fair learning algorithms is strictly worse…
Encoded text representations often capture sensitive attributes about individuals (e.g., race or gender), which raise privacy concerns and can make downstream models unfair to certain groups. In this work, we propose FEDERATE, an approach that combines ideas from differential privacy and adversarial training to learn private text representations which also induces fairer models. We empirically evaluate the trade-off between the privacy of the representations and the fairness and accuracy of the downstream model on four NLP datasets. Our results show that FEDERATE consistently improves upon pre…
CSIRO Data61（联邦科学与工业研究组织数据61所）；University of Technology Sydney（悉尼科技大学）；Harbin University of Technology（哈尔滨工业大学）；Zhejiang Lab（之江实验室）
Federated learning (FL) provides an effective machine learning (ML) architecture to protect data privacy in a distributed manner. However, the inevitable network asynchrony, the over-dependence on a central coordinator, and the lack of an open and fair incentive mechanism collectively hinder its further development. We propose \textsc{IronForge}, a new generation of FL framework, that features a Directed Acyclic Graph (DAG)-based data structure and eliminates the need for central coordinators to achieve fully decentralized operations. \textsc{IronForge} runs in a public and open network, and l…
To improve real-world applications of machine learning, experienced modelers develop intuition about their datasets, their models, and how the two interact. Manual inspection of raw data - of representative samples, of outliers, of misclassifications - is an essential tool in a) identifying and fixing problems in the data, b) generating new modeling hypotheses, and c) assigning or refining human-provided labels. However, manual data inspection is problematic for privacy sensitive datasets, such as those representing the behavior of real-world individuals. Furthermore, manual data inspection is…
Towards Personalized Differentially Private Learning for Decentralized Local Graphs
School of Computer Science, Beijing University of Posts and Telecommunications（北京邮电大学计算机学院）；State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（北京邮电大学网络与交换技术国家重点实验室）；School of Cyber Science and Technology, Shandong University（山东大学网络空间安全学院）；School of Cyberspace Security, Beijing University of Posts and Telecommunications（北京邮电大学网络空间安全学院）；Department of Computer Science, University of Illinois at Chicago（伊利诺伊大学芝加哥分校计算机科学系）
Rutgers, The State University of New Jersey（罗格斯，新泽西州立大学）；Nokia Bell Labs（诺基亚贝尔实验室）
Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differentially private decentralized algorithms can provide strong privacy guarantees. However, the accuracy of the joint estimates may be poor when the datasets at each site are small. This paper proposes a new framework, Correlation Assisted Private Estimation (CAPE), for designing pr…
KTH Royal Institute of Technology（KTH皇家理工学院）；Apple（苹果公司）
Federated learning with differential privacy, or private federated learning, provides a strategy to train machine learning models while respecting users' privacy. However, differential privacy can disproportionately degrade the performance of the models on under-represented groups, as these parts of the distribution are difficult to learn in the presence of noise. Existing approaches for enforcing fairness in machine learning models have considered the centralized setting, in which the algorithm has access to the users' data. This paper introduces an algorithm to enforce group fairness in priv…
Differentially Private Decentralized Dataset Synthesis Through Randomized Mixing with Correlated Noise
School of Data and Sciences（数据与科学学院）；BRAC University（布拉克大学）；Electrical and Computer Engineering（电气与计算机工程）；University of California, Santa Barbara（加州大学圣巴巴拉分校）；Electrical and Electronic Engineering（电气与电子工程）；Bangladesh University Engineering and Technology（孟加拉工程与技术大学）
Bangladesh University of Engineering and Technology（孟加拉工程技术大学）；Rutgers University（罗格斯大学）；Nokia Bell Labs（诺基亚贝尔实验室）；Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)（神经成像与数据科学转化研究三机构中心（TReNDS））；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. In order to leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, precluding centralized analyses that pool the data at a single site. In this work, we propose a differentially private algorithm for performing ICA in a decentralized data setting. Conventional approaches to decentralized differentially private algorithms may introduce too much noise…
KTH Royal Institute of Technology（瑞典皇家理工学院）；Elekta AB（医科达公司）
When it comes to preserving privacy in medical machine learning, two important considerations are (1) keeping data local to the institution and (2) avoiding inference of sensitive information from the trained model. These are often addressed using federated learning and differential privacy, respectively. However, the commonly used Federated Averaging algorithm requires a high degree of synchronization between participating institutions. For this reason, we turn our attention to Private Aggregation of Teacher Ensembles (PATE), where all local models can be trained independently without inter-i…
University of Houston（休斯顿大学）；Guilin University of Electronic Technology（桂林电子科技大学）；Guangdong University of Technology（广东工业大学）
Machine learning is increasingly becoming a powerful tool to make decisions in a wide variety of applications, such as medical diagnosis and autonomous driving. Privacy concerns related to the training data and unfair behaviors of some decisions with regard to certain attributes (e.g., sex, race) are becoming more critical. Thus, constructing a fair machine learning model while simultaneously providing privacy protection becomes a challenging problem. In this paper, we focus on the design of classification model with fairness and differential privacy guarantees by jointly combining functional …
National University of Singapore（新加坡国立大学）；The University of Melbourne（墨尔本大学）；IBM Singapore Lab（IBM新加坡实验室）；Monash University（莫纳什大学）；Deakin University（迪肯大学）
This paper firstly considers the research problem of fairness in collaborative deep learning, while ensuring privacy. A novel reputation system is proposed through digital tokens and local credibility to ensure fairness, in combination with differential privacy to guarantee privacy. In particular, we build a fair and differentially private decentralised deep learning framework called FDPDDL, which enables parties to derive more accurate local models in a fair and private manner by using our developed two-stage scheme: during the initialisation stage, artificial samples generated by Differentia…
Achieving Hilbert-Schmidt Independence Under Rényi Differential Privacy for Fair and Private Data Generation
Department of Mathematics and Computer Science University of Southern Denmark（丹麦南部大学数学与计算机科学系）；Department of Mathematics and Computer Science Freie Universität Berlin（柏林自由大学数学与计算机科学系）；Faculty for Informatik Universität der Bundeswehr München（联邦国防军大学慕尼黑信息学院）
Seldon（塞尔顿）；University of California San Diego（加州大学圣地亚哥分校）；Ford R&A（福特研发中心）；University of Oxford（牛津大学）；OpenMined（开放挖掘）
Deployment of deep learning in different fields and industries is growing day by day due to its performance, which relies on the availability of data and compute. Data is often crowd-sourced and contains sensitive information about its contributors, which leaks into models that are trained on it. To achieve rigorous privacy guarantees, differentially private training mechanisms are used. However, it has recently been shown that differential privacy can exacerbate existing biases in the data and have disparate impacts on the accuracy of different subgroups of data. In this paper, we aim to stud…
Etsy Inc（Etsy公司）；Rutgers University（罗格斯大学）；University at Albany, SUNY（纽约州立大学奥尔巴尼分校）
Recently, there has been an increasing adoption of differential privacy guided algorithms for privacy-preserving machine learning tasks. However, the use of such algorithms comes with trade-offs in terms of algorithmic fairness, which has been widely acknowledged. Specifically, we have empirically observed that the classical collaborative filtering method, trained by differentially private stochastic gradient descent (DP-SGD), results in a disparate impact on user groups with respect to different user engagement levels. This, in turn, causes the original unfair model to become even more biased…
The Private Aggregation of Teacher Ensembles (PATE) is an important private machine learning framework. It combines multiple learning models used as teachers for a student model that learns to predict an output chosen by noisy voting among the teachers. The resulting model satisfies differential privacy and has been shown effective in learning high-quality private models in semisupervised settings or when one wishes to protect the data labels. This paper asks whether this privacy-preserving framework introduces or exacerbates bias and unfairness and shows that PATE can introduce accuracy dispa…
National University of Singapore（新加坡国立大学）；Monash University（莫纳什大学）；The University of Melbourne（墨尔本大学）
It has been demonstrated that hidden representation learned by a deep model can encode private information of the input, hence can be exploited to recover such information with reasonable accuracy. To address this issue, we propose a novel approach called Differentially Private Neural Representation (DPNR) to preserve the privacy of the extracted representation from text. DPNR utilises Differential Privacy (DP) to provide a formal privacy guarantee. Further, we show that masking words via dropout can further enhance privacy. To maintain utility of the learned representation, we integrate DP-no…
Media Lab（媒体实验室）；Institute for Data, Systems and Society（数据、系统与社会研究所）；Massachusetts Institute of Technology（麻省理工学院）
The rapid proliferation of decentralized learning systems mandates the need for differentially-private cooperative learning. In this paper, we study this in context of the contextual linear bandit: we consider a collection of agents cooperating to solve a common contextual bandit, while ensuring that their communication remains private. For this problem, we devise \textsc{FedUCB}, a multiagent private algorithm for both centralized and decentralized (peer-to-peer) federated learning. We provide a rigorous technical analysis of its utility in terms of regret, improving several results in cooper…
Differentially Private Pre-Trained Model Fusion using Decentralized Federated Graph Matching
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Changsha Aier Eye Hospital（长沙爱尔眼科医院）；Nanchang Aier Eye Hospital（南昌爱尔眼科医院）
Northeastern University（东北大学）；University of Pennsylvania（宾夕法尼亚大学）
Motivated by settings in which predictive models may be required to be non-discriminatory with respect to certain attributes (such as race), but even collecting the sensitive attribute may be forbidden or restricted, we initiate the study of fair learning under the constraint of differential privacy. We design two learning algorithms that simultaneously promise differential privacy and equalized odds, a 'fairness' condition that corresponds to equalizing false positive and negative rates across protected groups. Our first algorithm is a private implementation of the equalized odds post-process…
FAIRPLAI: A Human-in-the-Loop Approach to Fair and Private Machine Learning
Dept. of Computer Science, The University of Texas at El Paso（得克萨斯大学埃尔帕索分校计算机科学系）；Dept. of Mathematics, Mountain View High School（山景高中数学系）；Dept. of Mathematics, El Paso Independent School District（埃尔帕索独立学区数学系）
More data is almost always beneficial for analysis and machine learning tasks. In many realistic situations however, an enterprise cannot share its data, either to keep a competitive advantage or to protect the privacy of the data sources, the enterprise's clients for example. We propose a method for data owners to share synthetic or fake versions of their data without sharing the actual data, nor the parameters of models that have direct access to the data. The method proposed is based on the privGAN architecture where local GANs are trained on their respective data subsets with an extra pena…
Syracuse University（雪城大学）；Georgia Institute of Technology（佐治亚理工学院）
A critical concern in data-driven decision making is to build models whose outcomes do not discriminate against some demographic groups, including gender, ethnicity, or age. To ensure non-discrimination in learning tasks, knowledge of the sensitive attributes is essential, while, in practice, these attributes may not be available due to legal and ethical requirements. To address this challenge, this paper studies a model that protects the privacy of the individuals sensitive information while also allowing it to learn non-discriminatory predictors. The method relies on the notion of differenti…
