PRICURE: Privacy-Preserving Collaborative Inference in a Multi-Party Setting
University of Michigan, Dearborn（密歇根大学迪尔伯恩分校）
Ismat Jarin, Birhanu Eshete
备注：
12 pages, 9 figures, to appear in the proceedings of the 7th ACM International Workshop on Security and Privacy Analytics (IWSPA'21) co-located with ACM CODASPY'21
When multiple parties that deal with private data aim for a collaborative prediction task such as medical image classification, they are often constrained by data protection regulations and lack of trust among collaborating parties. If done in a privacy-preserving manner, predictive analytics can benefit from the collective prediction capability of multiple parties holding complementary datasets on the same machine learning task. This paper presents PRICURE, a system that combines complementary strengths of secure multi-party computation (SMPC) and differential privacy (DP) to enable privacy-preserving collaborative prediction among multiple model owners. SMPC enables secret-sharing of private models and client inputs with non-colluding secure servers to compute predictions without leaking model parameters and inputs. DP masks true prediction results via noisy aggregation so as to deter a semi-honest client who may mount membership inference attacks. We evaluate PRICURE on neural networks across four datasets including benchmark medical image classification datasets. Our results suggest PRICURE guarantees privacy for tens of model owners and clients with acceptable accuracy loss. We also show that DP reduces membership inference attack exposure without hurting accuracy.
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；The Chinese University of Hong Kong（香港中文大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Northwestern University（西北大学）
Machine learning benefits from large training datasets, which may not always be possible to collect by any single entity, especially when using privacy-sensitive data. In many contexts, such as healthcare and finance, separate parties may wish to collaborate and learn from each other's data but are prevented from doing so due to privacy regulations. Some regulations prevent explicit sharing of data between parties by joining datasets in a central location (confidentiality). Others also limit implicit sharing of data, e.g., through model predictions (privacy). There is currently no method that …
JD Finance America Corporation（京东金融美国公司）；Department of Electrical & Computer Engineering, University of Pittsburgh（匹兹堡大学电气与计算机工程系）；University of Pittsburgh（匹兹堡大学）
The privacy-preserving federated learning for vertically partitioned data has shown promising results as the solution of the emerging multi-party joint modeling application, in which the data holders (such as government branches, private finance and e-business companies) collaborate throughout the learning process rather than relying on a trusted third party to hold data. However, existing federated learning algorithms for vertically partitioned data are limited to synchronous computation. To improve the efficiency when the unbalanced computation/communication resources are common among the pa…
Machine learning is promising, but it often needs to process vast amounts of sensitive data which raises concerns about privacy. In this white-paper, we introduce Substra, a distributed framework for privacy-preserving, traceable and collaborative Machine Learning. Substra gathers data providers and algorithm designers into a network of nodes that can train models on demand but under advanced permission regimes. To guarantee data privacy, Substra implements distributed learning: the data never leave their nodes; only algorithms, predictive models and non-sensitive metadata are exchanged on the…
Recommendation and collaborative filtering systems are important in modern information and e-commerce applications. As these systems are becoming increasingly popular in the industry, their outputs could affect business decision making, introducing incentives for an adversarial party to compromise the availability or integrity of such systems. We introduce a data poisoning attack on collaborative filtering systems. We demonstrate how a powerful attacker with full knowledge of the learner can generate malicious data so as to maximize his/her malicious objectives, while at the same time mimickin…
This paper proposes an interpretable non-model sharing collaborative data analysis method as one of the federated learning systems, which is an emerging technology to analyze distributed data. Analyzing distributed data is essential in many applications such as medical, financial, and manufacturing data analyses due to privacy, and confidentiality concerns. In addition, interpretability of the obtained model has an important role for practical applications of the federated learning systems. By centralizing intermediate representations, which are individually constructed in each party, the prop…
Institute for AI Industry Research, Tsinghua University（清华大学智能产业研究院）
Vertical Federated Learning (VFL) is widely utilized in real-world applications to enable collaborative learning while protecting data privacy and safety. However, previous works show that parties without labels (passive parties) in VFL can infer the sensitive label information owned by the party with labels (active party) or execute backdoor attacks to VFL. Meanwhile, active party can also infer sensitive feature information from passive party. All these pose new privacy and security challenges to VFL systems. We propose a new general defense method which limits the mutual information between…
Cooperative learning, that enables two or more data owners to jointly train a model, has been widely adopted to solve the problem of insufficient training data in machine learning. Nowadays, there is an urgent need for institutions and organizations to train a model cooperatively while keeping each other's data privately. To address the issue of privacy-preserving in collaborative learning, secure outsourced computation and federated learning are two typical methods. Nevertheless, there are many drawbacks for these two methods when they are leveraged in cooperative learning. For secure outsour…
Johannes Gutenberg University Mainz（约翰内斯·古腾堡-美因茨大学）
Deep learning (DL) approaches are achieving extraordinary results in a wide range of domains, but often require a massive collection of private data. Hence, methods for training neural networks on the joint data of different data owners, that keep each party's input confidential, are called for. We address a specific setting in federated learning, namely that of deep learning from horizontally distributed data with a limited number of parties, where their vulnerable intermediate results have to be processed in a privacy-preserving manner. This setting can be found in medical and healthcare as …
Federated Learning (FL) is emerging as a promising technology to build machine learning models in a decentralized, privacy-preserving fashion. Indeed, FL enables local training on user devices, avoiding user data to be transferred to centralized servers, and can be enhanced with differential privacy mechanisms. Although FL has been recently deployed in real systems, the possibility of collaborative modeling across different 3rd-party applications has not yet been explored. In this paper, we tackle this problem and present Federated Learning as a Service (FLaaS), a system enabling different sce…
University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）；Mayo Clinic（梅奥诊所）
Federated learning (FL) is a decentralized method enabling hospitals to collaboratively learn a model without sharing private patient data for training. In FL, participant hospitals periodically exchange training results rather than training samples with a central server. However, having access to model parameters or gradients can expose private training data samples. To address this challenge, we adopt secure multiparty computation (SMC) to establish a privacy-preserving federated learning framework. In our proposed method, the hospitals are divided into clusters. After local training, each h…
National University of Singapore（新加坡国立大学）；The University of Melbourne（墨尔本大学）；IBM Singapore Lab（IBM新加坡实验室）；Monash University（莫纳什大学）；Deakin University（迪肯大学）
This paper firstly considers the research problem of fairness in collaborative deep learning, while ensuring privacy. A novel reputation system is proposed through digital tokens and local credibility to ensure fairness, in combination with differential privacy to guarantee privacy. In particular, we build a fair and differentially private decentralised deep learning framework called FDPDDL, which enables parties to derive more accurate local models in a fair and private manner by using our developed two-stage scheme: during the initialisation stage, artificial samples generated by Differentia…
Tsinghua-Berkeley Shenzhen Institute（清华-伯克利深圳学院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua University（清华大学）
With the deployment of smart sensors and advancements in communication technologies, big data analytics have become vastly popular in the smart grid domain, informing stakeholders of the best power utilization strategy. However, these power-related data are stored and owned by different parties. For example, power consumption data are stored in numerous transformer stations across cities; mobility data of the population, which are important indicators of power consumption, are held by mobile companies. Direct data sharing might compromise party benefits, individual privacy and even national se…
FedPall: Prototype-based Adversarial and Collaborative Learning for Federated Learning with Feature Drift
Artificial Intelligence Research Institute, Shenzhen MSU-BIT University（深圳MSU-BIT大学人工智能研究院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）
Data privacy has become an increasingly important issue in Machine Learning (ML), where many approaches have been developed to tackle this challenge, e.g. cryptography (Homomorphic Encryption (HE), Differential Privacy (DP), etc.) and collaborative training (Secure Multi-Party Computation (MPC), Distributed Learning and Federated Learning (FL)). These techniques have a particular focus on data encryption or secure local computation. They transfer the intermediate information to the third party to compute the final result. Gradient exchanging is commonly considered to be a secure way of trainin…
IBM Research（IBM研究院）；University of Pittsburgh（匹兹堡大学）
Federated learning (FL) has been proposed to allow collaborative training of machine learning (ML) models among multiple parties where each party can keep its data private. In this paradigm, only model updates, such as model weights or gradients, are shared. Many existing approaches have focused on horizontal FL, where each party has the entire feature set and labels in the training data set. However, many real scenarios follow a vertically-partitioned FL setup, where a complete feature set is formed only when all the datasets from the parties are combined, and the labels are only available to…
The Hong Kong University of Science and Technology（香港科技大学）；University of Toronto（多伦多大学）
Federated learning (FL) has increasingly been deployed, in its vertical form, among organizations to facilitate secure collaborative training over siloed data. In vertical FL (VFL), participants hold disjoint features of the same set of sample instances. Among them, only one has labels. This participant, known as the active party, initiates the training and interacts with the other participants, known as the passive parties. Despite the increasing adoption of VFL, it remains largely unknown if and how the active party can extract feature data from the passive party, especially when training de…
EASTER: Embedding Aggregation-based Heterogeneous Models Training in Vertical Federated Learning
Beijing Institute of Technology（北京理工大学）；School of Cyberspace Science and Technology（信息空间科学与技术学院）；Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE（教育部民族语言智能分析与安全治理重点实验室）；Minzu University of China（中央民族大学）；School of Information Engineering（信息工程学院）；School of Computing and Communications（计算与通信学院）
Federated learning (FL) is recently surging as a promising decentralized deep learning (DL) framework that enables DL-based approaches trained collaboratively across clients without sharing private data. However, in the context of the central party being active and dishonest, the data of individual clients might be perfectly reconstructed, leading to the high possibility of sensitive information being leaked. Moreover, FL also suffers from the nonindependent and identically distributed (non-IID) data among clients, resulting in the degradation in the inference performance on local clients' dat…
Data61, CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）；Cyber Security CRC（网络安全合作研究中心）；Illinois Institute of Technology（伊利诺伊理工学院）；Concordia University（康考迪亚大学）；Hong Kong Polytechnic University（香港理工大学）；Ericsson Security Research, Ericsson Canada（加拿大爱立信公司爱立信安全研究院）；University of Connecticut（康涅狄格大学）
Outsourcing anomaly detection to third-parties can allow data owners to overcome resource constraints (e.g., in lightweight IoT devices), facilitate collaborative analysis (e.g., under distributed or multi-party scenarios), and benefit from lower costs and specialized expertise (e.g., of Managed Security Service Providers). Despite such benefits, a data owner may feel reluctant to outsource anomaly detection without sufficient privacy protection. To that end, most existing privacy solutions would face a novel challenge, i.e., preserving privacy usually requires the difference between data entr…
Xidian University（西安电子科技大学）；JD Tech（京东科技）；JD Finance America Corporation（京东金融美国公司）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Pittsburgh（匹兹堡大学）
Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage. In the real VFL applications, usually only one or partial parties hold labels, which makes it challenging for all parties to collaboratively learn the model without privacy leakage. Meanwhile, most existing VFL algorithms are trapped in the synchronous computations, which leads to inefficiency in their real-world applications. To address these challenging problems, we propose a novel {\bf VF}L framework integrated with new {\bf b}ack…
Federated Learning (FL) is an approach to collaboratively train a model across multiple parties without sharing data between parties or an aggregator. It is used both in the consumer domain to protect personal data as well as in enterprise settings, where dealing with data domicile regulation and the pragmatics of data silos are the main drivers. While gradient boosted tree implementations such as XGBoost have been very successful for many use cases, its federated learning adaptations tend to be very slow due to using cryptographic and privacy methods and have not experienced widespread use. W…
We propose an efficient framework for enabling secure multi-party numerical computations in a Peer-to-Peer network. This problem arises in a range of applications such as collaborative filtering, distributed computation of trust and reputation, monitoring and numerous other tasks, where the computing nodes would like to preserve the privacy of their inputs while performing a joint computation of a certain function. Although there is a rich literature in the field of distributed systems security concerning secure multi-party computation, in practice it is hard to deploy those methods in very la…
Proto-EVFL: Enhanced Vertical Federated Learning via Dual Prototype with Extremely Unaligned Data
School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）；School of Computer Science and Technology, Shandong University（山东大学计算机科学与技术学院）；Center for Applied Statistics, School of Statistics, Renmin University of China（中国人民大学统计学院应用统计中心）；Beijing Academy of Blockchain and Edge Computing（北京区块链与边缘计算研究院）；Department of Statistics, Fudan University（复旦大学统计系）；Department of Computer Science and Engineering, Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）
Delft University of Technology（代尔夫特理工大学）；University of Greenwich（格林威治大学）
In this work, we propose FLVoogd, an updated federated learning method in which servers and clients collaboratively eliminate Byzantine attacks while preserving privacy. In particular, servers use automatic Density-based Spatial Clustering of Applications with Noise (DBSCAN) combined with S2PC to cluster the benign majority without acquiring sensitive personal information. Meanwhile, clients build dual models and perform test-based distance controlling to adjust their local models toward the global one to achieve personalizing. Our framework is automatic and adaptive that servers/clients don't…
University of Connecticut（康涅狄格大学）；Northeastern University（东北大学）；Duke University（杜克大学）；Amazon Web Services, Inc.（亚马逊云科技公司）；Stevens Institute of Technology（史蒂文斯理工学院）
Distributed learning such as federated learning or collaborative learning enables model training on decentralized data from users and only collects local gradients, where data is processed close to its sources for data privacy. The nature of not centralizing the training data addresses the privacy issue of privacy-sensitive data. Recent studies show that a third party can reconstruct the true training data in the distributed machine learning system through the publicly-shared gradients. However, existing reconstruction attack frameworks lack generalizability on different Deep Neural Network (D…
The performance of deep learning models highly depends on the amount of training data. It is common practice for today's data holders to merge their datasets and train models collaboratively, which yet poses a threat to data privacy. Different from existing methods such as secure multi-party computation (MPC) and federated learning (FL), we find representation learning has unique advantages in collaborative learning due to the lower communication overhead and task-independency. However, data representations face the threat of model inversion attacks. In this article, we formally define the col…
FLEDGE: Ledger-based Federated Learning Resilient to Inference and Backdoor Attacks
The University of Texas Rio Grande Valley（德克萨斯大学里奥格兰德谷分校）；Technical University of Darmstadt（达姆施塔特工业大学）；KOBIL GmbH（KOBIL有限公司）；The University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Toronto Metropolitan University（多伦多都会大学）；Royal Military College（皇家军事学院）
As the most significant data source in smart mobility systems, GPS trajectories can help identify user travel mode. However, these GPS datasets may contain users' private information (e.g., home location), preventing many users from sharing their private information with a third party. Hence, identifying travel modes while protecting users' privacy is a significant issue. To address this challenge, we use federated learning (FL), a privacy-preserving machine learning technique that aims at collaboratively training a robust global model by accessing users' locally trained models but not their r…
Preservation of privacy has been a serious concern with the increasing use of IoT-assisted smart systems and their ubiquitous smart sensors. To solve the issue, the smart systems are being trained to depend more on aggregated data instead of directly using raw data. However, most of the existing strategies for privacy-preserving data aggregation, either depend on computation-intensive Homomorphic Encryption based operations or communication-intensive collaborative mechanisms. Unfortunately, none of the approaches are directly suitable for a resource-constrained IoT system. In this work, we lev…
We present a relational MPC framework for secure collaborative analytics on private data with no information leakage. Our work targets challenging use cases where data owners may not have private resources to participate in the computation, thus, they need to securely outsource the data analysis to untrusted third parties. We define a set of oblivious operators, explain the secure primitives they rely on, and analyze their costs in terms of operations and inter-party communication. We show how these operators can be composed to form end-to-end oblivious queries, and we introduce logical and ph…
Federated Learning in Genetics: Extended Analysis of Accuracy, Performance and Privacy Trade-offs
Dept. of Computer Science（计算机科学系）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig（可扩展数据分析与人工智能中心（ScaDS.AI）德累斯顿/莱比锡）；Leipzig University（莱比锡大学）
Tsinghua-Berkeley Shenzhen Institute, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院清华-伯克利深圳学院）；Tsinghua University（清华大学）
In a modern power system, real-time data on power generation/consumption and its relevant features are stored in various distributed parties, including household meters, transformer stations and external organizations. To fully exploit the underlying patterns of these distributed data for accurate power prediction, federated learning is needed as a collaborative but privacy-preserving training scheme. However, current federated learning frameworks are polarized towards addressing either the horizontal or vertical separation of data, and tend to overlook the case where both are present. Further…
IBM Research（IBM研究院）；IBM Cognitive Systems（IBM认知系统公司）；ByteDance（字节跳动）
Distributed collaborative learning (DCL) paradigms enable building joint machine learning models from distrusting multi-party participants. Data confidentiality is guaranteed by retaining private training data on each participant's local infrastructure. However, this approach to achieving data confidentiality makes today's DCL designs fundamentally vulnerable to data poisoning and backdoor attacks. It also limits DCL's model accountability, which is key to backtracking the responsible "bad" training data instances/contributors. In this paper, we introduce CALTRAIN, a Trusted Execution Environm…
HybridFL: A Federated Learning Approach for Financial Crime Detection
Department of Advanced Computing Sciences, Maastricht University（马斯特里赫特大学高级计算科学系）；Department of Cognitive Science and Artificial Intelligence, Tilburg University（蒂尔堡大学认知科学与人工智能系）
In recent years, gradient boosted decision tree learning has proven to be an effective method of training robust models. Moreover, collaborative learning among multiple parties has the potential to greatly benefit all parties involved, but organizations have also encountered obstacles in sharing sensitive data due to business, regulatory, and liability concerns. We propose Secure XGBoost, a privacy-preserving system that enables multiparty training and inference of XGBoost models. Secure XGBoost protects the privacy of each party's data as well as the integrity of the computation with the help…
Secure multi-party computation (MPC) is a broad cryptographic concept that can be adopted for privacy-preserving computation. With MPC, a number of parties can collaboratively compute a function, without revealing the actual input or output of the plaintext to others. The applications of MPC range from privacy-preserving voting, arithmetic calculation, and large-scale data analysis. From the system perspective, each party in MPC can run on one compute node. The compute nodes of multiple parties could be either homogeneous or heterogeneous; however, the distributed workloads from the MPC protoc…
Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, by design, MPC protocols faithfully compute the training functionality, which the adversarial ML community has shown to leak private information and can be tampered with in poisoning attacks. In this work, we argue that model ensembles, implemented in our framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks. The natural partitioning of data amongst owners in MPC training al…
Georgia Institute of Technology（佐治亚理工学院）；J.P. Morgan AI Research（摩根大通人工智能研究院）
Federated Learning enables a population of clients, working with a trusted server, to collaboratively learn a shared machine learning model while keeping each client's data within its own local systems. This reduces the risk of exposing sensitive data, but it is still possible to reverse engineer information about a client's private data set from communicated model parameters. Most federated learning systems therefore use differential privacy to introduce noise to the parameters. This adds uncertainty to any attempt to reveal private client data, but also reduces the accuracy of the shared mod…
The current business model for existing recommender services is centered around the availability of users' personal data at their side whereas consumers have to trust that the recommender service providers will not use their data in a malicious way. With the increasing number of cases for privacy breaches, different countries and corporations have issued privacy laws and regulations to define the best practices for the protection of personal information. The data protection directive 95/46/EC and the privacy principles established by the Organization for Economic Cooperation and Development (O…
We propose a framework in which multiple entities collaborate to build a machine learning model while preserving privacy of their data. The approach utilizes feature embeddings from shared/per-entity feature extractors transforming data into a feature space for cooperation between entities. We propose two specific methods and compare them with a baseline method. In Shared Feature Extractor (SFE) Learning, the entities use a shared feature extractor to compute feature embeddings of samples. In Locally Trained Feature Extractor (LTFE) Learning, each entity uses a separate feature extractor and m…
Mining the spatial and temporal correlation of wind farm output data is beneficial for enhancing the precision of ultra-short-term wind power prediction. However, if the wind farms are owned by separate entities, they may be reluctant to share their data directly due to privacy concerns as well as business management regulation policies. Although cryptographic approaches have been designed to protect privacy in the process of data sharing, it is still a challenging problem to encrypt the original data while extracting the nonlinear relationship among multiple wind farms in the machine learning…
