Nowadays, gathering high-quality training data from multiple data sources with privacy preservation is a crucial challenge to training high-performance machine learning models. The potential solutions could break the barriers among isolated data corpus, and consequently enlarge the range of data available for processing. To this end, both academic researchers and industrial vendors are recently strongly motivated to propose two main-stream folders of solutions mainly based on software constructions: 1) Secure Multi-party Learning (MPL for short); and 2) Federated Learning (FL for short). The above two technical folders have their advantages and limitations when we evaluate them according to the following five criteria: security, efficiency, data distribution, the accuracy of trained models, and application scenarios.
Motivated to demonstrate the research progress and discuss the insights on the future directions, we thoroughly investigate these protocols and frameworks of both MPL and FL. At first, we define the problem of Training machine learning Models over Multiple data sources with Privacy Preservation (TMMPP for short). Then, we compare the recent studies of TMMPP from the aspects of the technical routes, the number of parties supported, data partitioning, threat model, and machine learning models supported, to show their advantages and limitations. Next, we investigate and evaluate five popular FL platforms. Finally, we discuss the potential directions to resolve the problem of TMMPP in the future.
Technical University of Munich（慕尼黑工业大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；Imperial College London（伦敦帝国理工学院）
In this work, we study the applications of differential privacy (DP) in the context of graph-structured data. We discuss the formulations of DP applicable to the publication of graphs and their associated statistics as well as machine learning on graph-based data, including graph neural networks (GNNs). The formulation of DP in the context of graph-structured data is difficult, as individual data points are interconnected (often non-linearly or sparsely). This connectivity complicates the computation of individual privacy loss in differentially private learning. The problem is exacerbated by a…
Cornell University（康奈尔大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
Local differential privacy (LDP) can be adopted to anonymize richer user data attributes that will be input to sophisticated machine learning (ML) tasks. However, today's LDP approaches are largely task-agnostic and often lead to severe performance loss -- they simply inject noise to all data attributes according to a given privacy budget, regardless of what features are most relevant for the ultimate task. In this paper, we address how to significantly improve the ultimate task performance with multi-dimensional user data by considering a task-aware privacy preservation problem. The key idea …
Efficiently Achieving Secure Model Training and Secure Aggregation to Ensure Bidirectional Privacy-Preservation in Federated Learning
School of Information Science and Technology, Southwest Jiaotong University（西南交通大学信息科学与技术学院）；Meituan Group（美团集团）；Shandong University（山东大学）；Zhejiang Gongshang University（浙江工商大学）
Process mining is rapidly growing in the industry. Consequently, privacy concerns regarding sensitive and private information included in event data, used by process mining algorithms, are becoming increasingly relevant. State-of-the-art research mainly focuses on providing privacy guarantees, e.g., differential privacy, for trace variants that are used by the main process mining techniques, e.g., process discovery. However, privacy preservation techniques for releasing trace variants still do not fulfill all the requirements of industry-scale usage. Moreover, providing privacy guarantees when…
University of Quebec（魁北克大学）；ETS（高等技术学院）；Princeton University（普林斯顿大学）
Recent advances in electronic devices and communication infrastructure have revolutionized the traditional healthcare system into a smart healthcare system by using IoMT devices. However, due to the centralized training approach of artificial intelligence (AI), the use of mobile and wearable IoMT devices raises privacy concerns with respect to the information that has been communicated between hospitals and end users. The information conveyed by the IoMT devices is highly confidential and can be exposed to adversaries. In this regard, federated learning (FL), a distributive AI paradigm has ope…
Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation
Division of Information Science and Engineering, School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology（信息科学与工程系，电气工程与计算机科学学院，皇家理工学院）
New Jersey Institute of Technology（新泽西理工学院）；University of Arkansas（阿肯色大学）；University of Oregon（俄勒冈大学）
In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks. To achieve this, we figure out a way to perturb affine transformations of neurons, and loss functions used in deep neural networks. In addition, our mechanism intentionally adds "more noise" into…
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；Case Western Reserve University（凯斯西储大学）
Despite the impressive performance of random forests (RF), its theoretical properties have not been thoroughly understood. In this paper, we propose a novel RF framework, dubbed multinomial random forest (MRF), to analyze the \emph{consistency} and \emph{privacy-preservation}. Instead of deterministic greedy split rule or with simple randomness, the MRF adopts two impurity-based multinomial distributions to randomly select a split feature and a split value respectively. Theoretically, we prove the consistency of the proposed MRF and analyze its privacy-preservation within the framework of diff…
Harnessing Inherent Noises for Privacy Preservation in Quantum Machine Learning
State Key Laboratory of Networking and Switching Technology（网络与交换技术国家重点实验室）；Beijing University of Posts and Telecommunications（北京邮电大学）；University of Houston（休斯顿大学）；Kennesaw State University（肯尼索州立大学）
UBTECH Sydney AI Centre, School of Computer Science, Faculty of Engineering, the University of Sydney（悉尼大学工程学院计算机科学学院优必选悉尼人工智能中心）；University of Science and Technology of China（中国科学技术大学）
This paper studies the relationship between generalization and privacy preservation in iterative learning algorithms by two sequential steps. We first establish an alignment between generalization and privacy preservation for any learning algorithm. We prove that $(\varepsilon, δ)$-differential privacy implies an on-average generalization bound for multi-database learning algorithms which further leads to a high-probability bound for any learning algorithm. This high-probability bound also implies a PAC-learnable guarantee for differentially private learning algorithms. We then investigate how…
TinyML NLP Scheme for Semantic Wireless Sentiment Classification with Privacy Preservation
Department of Electrical Engineering and Computer Science, York University（电气工程与计算机科学系，约克大学）；CEMSE Division, King Abdullah University of Science and Technology (KAUST)（CEMSE部门，国王 Abdullah 科学与技术大学（KAUST））
Georgia State University（佐治亚州立大学）；Rochester Institute of Technology（罗切斯特理工学院）
Outsourced computation for neural networks allows users access to state of the art models without needing to invest in specialized hardware and know-how. The problem is that the users lose control over potentially privacy sensitive data. With homomorphic encryption (HE) computation can be performed on encrypted data without revealing its content. In this systematization of knowledge, we take an in-depth look at approaches that combine neural networks with HE for privacy preservation. We categorize the changes to neural network models and architectures to make them computable over HE and how th…
Laboratory for Data Security -- EPFL（数据安全实验室 -- 洛桑联邦理工学院）
Tree-based models are among the most efficient machine learning techniques for data mining nowadays due to their accuracy, interpretability, and simplicity. The recent orthogonal needs for more data and privacy protection call for collaborative privacy-preserving solutions. In this work, we survey the literature on distributed and privacy-preserving training of tree-based models and we systematize its knowledge based on four axes: the learning algorithm, the collaborative model, the protection mechanism, and the threat model. We use this to identify the strengths and limitations of these works…
Histograms and synthetic data are of key importance in data analysis. However, researchers have shown that even aggregated data such as histograms, containing no obvious sensitive attributes, can result in privacy leakage. To enable data analysis, a strong notion of privacy is required to avoid risking unintended privacy violations. Such a strong notion of privacy is differential privacy, a statistical notion of privacy that makes privacy leakage quantifiable. The caveat regarding differential privacy is that while it has strong guarantees for privacy, privacy comes at a cost of accuracy. Desp…
The online privacy and security of the disabled community is a complex field that has implications for every user who navigates web services. While many disciplines have separately researched the disabled population and their online privacy and security concerns, the overlap between the two is very high but under-researched. Moreover, a complex relationship exists between the disabled population and web services where the interaction depends on several web service developmental factors, including usability and accessibility. To this aid, we explored this intersection of privacy and security of…
Shortly after it was first introduced in 2006, differential privacy became the flagship data privacy definition. Since then, numerous variants and extensions were proposed to adapt it to different scenarios and attacker models. In this work, we propose a systematic taxonomy of these variants and extensions. We list all data privacy definitions based on differential privacy, and partition them into seven categories, depending on which aspect of the original definition is modified. These categories act like dimensions: variants from the same category cannot be combined, but variants from differe…
Vienna University of Technology（维也纳技术大学）；University of Alicante（阿利坎特大学）
As the privacy risks posed by camera surveillance and facial recognition have grown, so has the research into privacy preservation algorithms. Among these, visual privacy preservation algorithms attempt to impart bodily privacy to subjects in visuals by obfuscating privacy-sensitive areas. While disparate performances of facial recognition systems across phenotypes are the subject of much study, its counterpart, privacy preservation, is not commonly analysed from a fairness perspective. In this paper, the fairness of commonly used visual privacy preservation algorithms is investigated through …
Blockchain, also coined as decentralized AI, has the potential to empower AI to be more trustworthy by creating a decentralized trust of privacy, security, and audibility. However, systematic studies on the design principle of blockchain as a trust engine for an integrated society of cyber-physical-social-system (CPSS) are still absent. In this article, we provide an initiative for seeking the design principle of blockchain for a better digital world. Using a hybrid method of qualitative and quantitative studies, we examine the past origin, the current development, and the future directions of…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；UC Berkeley（加州大学伯克利分校）；Technische Universität München（慕尼黑工业大学）；University of Minnesota（明尼苏达大学）；Alibaba Group（阿里巴巴集团）
This work tackles the Pixel Privacy task put forth by MediaEval 2019. Our goal is to manipulate images in a way that conceals them from automatic scene classifiers while preserving the original image quality. We use the fast gradient sign method, which normally has a corrupting influence on image appeal, and devise two methods to minimize the damage. The first approach uses a map of pixel locations that are either salient or flat, and directs perturbations away from them. The second approach subtracts the gradient of an aesthetics evaluation model from the gradient of the attack model to guide…
Digital identity is a multidimensional, multidisciplinary, and a complex concept. As a result, it is difficult to apprehend. Many contributions have proposed definitions and representations of digital identity. However, lots of them are either very generic and difficult to implement or do not take into account privacy issues. Seeing how important privacy master is, it becomes a necessity to rethink digital identity in order to take into account privacy issues. So, this paper aims at proposing an attribute-based digital identity vision for privacy preservation purposes. The proposed model takes…
