Tree-based models are among the most efficient machine learning techniques for data mining nowadays due to their accuracy, interpretability, and simplicity. The recent orthogonal needs for more data and privacy protection call for collaborative privacy-preserving solutions. In this work, we survey the literature on distributed and privacy-preserving training of tree-based models and we systematize its knowledge based on four axes: the learning algorithm, the collaborative model, the protection mechanism, and the threat model. We use this to identify the strengths and limitations of these works and provide for the first time a framework analyzing the information leakage occurring in distributed tree-based model learning.
Georgia State University（佐治亚州立大学）；Rochester Institute of Technology（罗切斯特理工学院）
Outsourced computation for neural networks allows users access to state of the art models without needing to invest in specialized hardware and know-how. The problem is that the users lose control over potentially privacy sensitive data. With homomorphic encryption (HE) computation can be performed on encrypted data without revealing its content. In this systematization of knowledge, we take an in-depth look at approaches that combine neural networks with HE for privacy preservation. We categorize the changes to neural network models and architectures to make them computable over HE and how th…
Nowadays, gathering high-quality training data from multiple data sources with privacy preservation is a crucial challenge to training high-performance machine learning models. The potential solutions could break the barriers among isolated data corpus, and consequently enlarge the range of data available for processing. To this end, both academic researchers and industrial vendors are recently strongly motivated to propose two main-stream folders of solutions mainly based on software constructions: 1) Secure Multi-party Learning (MPL for short); and 2) Federated Learning (FL for short). The a…
Technical University of Munich（慕尼黑工业大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；Imperial College London（伦敦帝国理工学院）
In this work, we study the applications of differential privacy (DP) in the context of graph-structured data. We discuss the formulations of DP applicable to the publication of graphs and their associated statistics as well as machine learning on graph-based data, including graph neural networks (GNNs). The formulation of DP in the context of graph-structured data is difficult, as individual data points are interconnected (often non-linearly or sparsely). This connectivity complicates the computation of individual privacy loss in differentially private learning. The problem is exacerbated by a…
Histograms and synthetic data are of key importance in data analysis. However, researchers have shown that even aggregated data such as histograms, containing no obvious sensitive attributes, can result in privacy leakage. To enable data analysis, a strong notion of privacy is required to avoid risking unintended privacy violations. Such a strong notion of privacy is differential privacy, a statistical notion of privacy that makes privacy leakage quantifiable. The caveat regarding differential privacy is that while it has strong guarantees for privacy, privacy comes at a cost of accuracy. Desp…
The online privacy and security of the disabled community is a complex field that has implications for every user who navigates web services. While many disciplines have separately researched the disabled population and their online privacy and security concerns, the overlap between the two is very high but under-researched. Moreover, a complex relationship exists between the disabled population and web services where the interaction depends on several web service developmental factors, including usability and accessibility. To this aid, we explored this intersection of privacy and security of…
Shortly after it was first introduced in 2006, differential privacy became the flagship data privacy definition. Since then, numerous variants and extensions were proposed to adapt it to different scenarios and attacker models. In this work, we propose a systematic taxonomy of these variants and extensions. We list all data privacy definitions based on differential privacy, and partition them into seven categories, depending on which aspect of the original definition is modified. These categories act like dimensions: variants from the same category cannot be combined, but variants from differe…
Blockchain, also coined as decentralized AI, has the potential to empower AI to be more trustworthy by creating a decentralized trust of privacy, security, and audibility. However, systematic studies on the design principle of blockchain as a trust engine for an integrated society of cyber-physical-social-system (CPSS) are still absent. In this article, we provide an initiative for seeking the design principle of blockchain for a better digital world. Using a hybrid method of qualitative and quantitative studies, we examine the past origin, the current development, and the future directions of…
Distributed data analysis without revealing the individual data has recently attracted significant attention in several applications. A collaborative data analysis through sharing dimensionality reduced representations of data has been proposed as a non-model sharing-type federated learning. This paper analyzes the accuracy and privacy evaluations of this novel framework. In the accuracy analysis, we provided sufficient conditions for the equivalence of the collaborative data analysis and the centralized analysis with dimensionality reduction. In the privacy analysis, we proved that collaborat…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；UC Berkeley（加州大学伯克利分校）；Technische Universität München（慕尼黑工业大学）；University of Minnesota（明尼苏达大学）；Alibaba Group（阿里巴巴集团）
Los Andes University（洛斯安第斯大学）；Rappi（Rappi公司）；Amazon（亚马逊）
Privacy-preserving machine learning in data-sharing processes is an ever-critical task that enables collaborative training of Machine Learning (ML) models without the need to share the original data sources. It is especially relevant when an organization must assure that sensitive data remains private throughout the whole ML pipeline, i.e., training and inference phases. This paper presents an innovative framework that uses Representation Learning via autoencoders to generate privacy-preserving embedded data. Thus, organizations can share the data representation to increase machine learning mo…
Communicating anonymously comes at a cost - and large communities have been in a constant tug-of-war between the development of faster protocols, and the improvement of security analyses. Thereby more intricate privacy goals emerged and more detailed bounds on the minimum overhead necessary to achieve them were proven. The entanglement of requirements, scenarios, and protocols complicates analysis, and the published results are hardly comparable, due to deviating, yet specific choices of assumptions and goals (some explicit, most implicit). In this paper, we systematize the field by harmonizin…
The Web public key infrastructure is essential to providing secure communication on the Internet today, and certificate authorities play a crucial role in this ecosystem by issuing certificates. These authorities may misissue certificates or suffer misuse attacks, however, which has given rise to the Certificate Transparency (CT) project. The goal of CT is to store all issued certificates in public logs, which can then be checked for the presence of potentially misissued certificates. Thus, the requirement that a given certificate is indeed in one (or several) of these logs lies at the core of…
Machine learning is promising, but it often needs to process vast amounts of sensitive data which raises concerns about privacy. In this white-paper, we introduce Substra, a distributed framework for privacy-preserving, traceable and collaborative Machine Learning. Substra gathers data providers and algorithm designers into a network of nodes that can train models on demand but under advanced permission regimes. To guarantee data privacy, Substra implements distributed learning: the data never leave their nodes; only algorithms, predictive models and non-sensitive metadata are exchanged on the…
When multiple parties that deal with private data aim for a collaborative prediction task such as medical image classification, they are often constrained by data protection regulations and lack of trust among collaborating parties. If done in a privacy-preserving manner, predictive analytics can benefit from the collective prediction capability of multiple parties holding complementary datasets on the same machine learning task. This paper presents PRICURE, a system that combines complementary strengths of secure multi-party computation (SMPC) and differential privacy (DP) to enable privacy-p…
The evolution of drone technology in the past nine years since the first commercial drone was introduced at CES 2010 has caused many individuals and businesses to adopt drones for various purposes. We are currently living in an era in which drones are being used for pizza delivery, the shipment of goods, and filming, and they are likely to provide an alternative for transportation in the near future. However, drones also pose a significant challenge in terms of security and privacy within society (for both individuals and organizations), and many drone related incidents are reported on a daily…
BME（布达佩斯技术与经济大学）；University of Cambridge（剑桥大学）；KUL（鲁汶大学）；Owkin（Owkin公司）
Pharmaceutical industry can better leverage its data assets to virtualize drug discovery through a collaborative machine learning platform. On the other hand, there are non-negligible risks stemming from the unintended leakage of participants' training data, hence, it is essential for such a platform to be secure and privacy-preserving. This paper describes a privacy risk assessment for collaborative modeling in the preclinical phase of drug discovery to accelerate the selection of promising drug candidates. After a short taxonomy of state-of-the-art inference attacks we adopt and customize se…
Missouri University of Science and Technology（密苏里科技大学）；University of Missouri-Columbia（密苏里大学哥伦比亚分校）；Applied Computational Intelligence Laboratory, Missouri University of Science and Technology（密苏里科技大学应用计算智能实验室）
Collaborative filtering recommendation systems provide recommendations to users based on their own past preferences, as well as those of other users who share similar interests. The use of recommendation systems has grown widely in recent years, helping people choose which movies to watch, books to read, and items to buy. However, users are often concerned about their privacy when using such systems, and many users are reluctant to provide accurate information to most online services. Privacy-preserving collaborative filtering recommendation systems aim to provide users with accurate recommend…
Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院（LIST））
Collaborative filtering recommenders provide effective personalization services at the cost of sacrificing the privacy of their end users. Due to the increasing concerns from the society and stricter privacy regulations, it is an urgent research challenge to design privacy-preserving and yet robust recommenders which offer recommendation services to privacy-aware users. Our analysis shows that existing solutions fall short in several aspects, including lacking attention to the precise output to end users and ignoring the correlated robustness issues. In this paper, we provide a general system …
Wuhan University（武汉大学）；The State Key Laboratory of Cryptography（密码学国家重点实验室）；Huawei Technologies Co., Ltd（华为技术有限公司）
With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them …
Cloud-edge collaborative inference approach splits deep neural networks (DNNs) into two parts that run collaboratively on resource-constrained edge devices and cloud servers, aiming at minimizing inference latency and protecting data privacy. However, even if the raw input data from edge devices is not directly exposed to the cloud, state-of-the-art attacks targeting collaborative inference are still able to reconstruct the raw private data from the intermediate outputs of the exposed local models, introducing serious privacy risks. In this paper, a secure privacy inference framework for cloud…
Decentralised, Collaborative, and Privacy-preserving Machine Learning for Multi-Hospital Data
University of Toronto（多伦多大学）；University Health Network（大学健康网络）；Vector Institute（矢量研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Simon Fraser University（西蒙菲莎大学）；St. Michael’s Hospital（圣迈克尔医院）；Unity Health Toronto（多伦多团结医疗）；Institute of Health Policy, Management and Evaluation, University of Toronto（多伦多大学卫生政策、管理与评价研究所）；Temerty Faculty of Medicine, University of Toronto（多伦多大学特梅蒂医学院）
