Collaborative learning enables two or more participants, each with their own training dataset, to collaboratively learn a joint model. It is desirable that the collaboration should not cause the disclosure of either the raw datasets of each individual owner or the local model parameters trained on them. This privacy-preservation requirement has been approached through differential privacy mechanisms, homomorphic encryption (HE) and secure multiparty computation (MPC), but existing attempts may either introduce the loss of model accuracy or imply significant computational and/or communicational overhead. In this work, we address this problem with the lightweight additive secret sharing technique. We propose PrivColl, a framework for protecting local data and local models while ensuring the correctness of training processes. PrivColl employs secret sharing technique for securely evaluating addition operations in a multiparty computation environment, and achieves practicability by employing only the homomorphic addition operations. We formally prove that it guarantees privacy preservation even though the majority (n-2 out of n) of participants are corrupted. With experiments on real-world datasets, we further demonstrate that PrivColl retains high efficiency. It achieves a speedup of more than 45X over the state-of-the-art MPC/HE based schemes for training linear/logistic regression, and 216X faster for training neural network.
Cloud-edge collaborative inference approach splits deep neural networks (DNNs) into two parts that run collaboratively on resource-constrained edge devices and cloud servers, aiming at minimizing inference latency and protecting data privacy. However, even if the raw input data from edge devices is not directly exposed to the cloud, state-of-the-art attacks targeting collaborative inference are still able to reconstruct the raw private data from the intermediate outputs of the exposed local models, introducing serious privacy risks. In this paper, a secure privacy inference framework for cloud…
With the rapid demand of data and computational resources in deep learning systems, a growing number of algorithms to utilize collaborative machine learning techniques, for example, federated learning, to train a shared deep model across multiple participants. It could effectively take advantage of the resources of each participant and obtain a more powerful learning system. However, integrity and privacy threats in such systems have greatly obstructed the applications of collaborative learning. And a large amount of works have been proposed to maintain the model integrity and mitigate the pri…
Collaboration among multiple organizations is imperative for contemporary intrusion detection. As modern threats become well sophisticated it is difficult for organizations to defend with threat context local to their networks alone. Availability of global \emph{threat intelligence} is must for organizations to defend against modern advanced persistent threats (APTs). In order to benefit from such global context of attacks, privacy concerns continue to be of major hindrance. In this position paper we identify real world privacy problems as precise use cases, relevant cryptographic technologies…
Collaborative inference has recently emerged as an attractive framework for applying deep learning to Internet of Things (IoT) applications by splitting a DNN model into several subpart models among resource-constrained IoT devices and the cloud. However, the reconstruction attack was proposed recently to recover the original input image from intermediate outputs that can be collected from local models in collaborative inference. For addressing such privacy issues, a promising technique is to adopt differential privacy so that the intermediate outputs are protected with a small accuracy loss. …
Spam phone calls have been rapidly growing from nuisance to an increasingly effective scam delivery tool. To counter this increasingly successful attack vector, a number of commercial smartphone apps that promise to block spam phone calls have appeared on app stores, and are now used by hundreds of thousands or even millions of users. However, following a business model similar to some online social network services, these apps often collect call records or other potentially sensitive information from users' phones with little or no formal privacy guarantees. In this paper, we study whether it…
The rogue DHCP is unauthorized server that releases the incorrect IP address to users and sniffs the traffic illegally. The contribution specially provides privacy to users and enhances the security aspects of mobile supported collaborative framework (MSCF) explained in [24].The paper introduces multi-frame signature-cum anomaly-based intrusion detection systems (MSAIDS) supported with novel algorithms and inclusion of new rules in existing IDS. The major target of contribution is to detect the malicious attacks and blocks the illegal activities of rogue DHCP server. This innovative security m…
Distributed data analysis without revealing the individual data has recently attracted significant attention in several applications. A collaborative data analysis through sharing dimensionality reduced representations of data has been proposed as a non-model sharing-type federated learning. This paper analyzes the accuracy and privacy evaluations of this novel framework. In the accuracy analysis, we provided sufficient conditions for the equivalence of the collaborative data analysis and the centralized analysis with dimensionality reduction. In the privacy analysis, we proved that collaborat…
With the rise of online lenders, the loan stacking problem has become a significant issue in the financial industry. One of the key steps in the fight against it is the querying of the loan history of a borrower from peer lenders. This is especially important in markets without a trusted credit bureau. To protect participants privacy and business interests, we want to hide borrower identities and lenders data from the loan originator, while simultaneously verifying that the borrower authorizes the query. In this paper, we propose Octopus, a distributed system to execute the query while meeting…
Real-time, online-editing web apps provide free and convenient services for collaboratively editing, sharing and storing files. The benefits of these web applications do not come for free: not only do service providers have full access to the users' files, but they also control access, transmission, and storage mechanisms for them. As a result, user data may be at risk of data mining, third-party interception, or even manipulation. To combat this, we propose a new system for helping to preserve the privacy of user data within collaborative environments. There are several distinct challenges in…
In collaborative recommendation systems, privacy may be compromised, as users' opinions are used to generate recommendations for others. In this paper, we consider an online collaborative recommendation system, and we measure users' privacy in terms of the standard differential privacy. We give the first quantitative analysis of the trade-offs between recommendation quality and users' privacy in such a system by showing a lower bound on the best achievable privacy for any non-trivial algorithm, and proposing a near-optimal algorithm. From our results, we find that there is actually little trad…
The neighbourhood-based Collaborative Filtering is a widely used method in recommender systems. However, the risks of revealing customers' privacy during the process of filtering have attracted noticeable public concern recently. Specifically, $k$NN attack discloses the target user's sensitive information by creating $k$ fake nearest neighbours by non-sensitive information. Among the current solutions against $k$NN attack, the probabilistic methods showed a powerful privacy preserving effect. However, the existing probabilistic methods neither guarantee enough prediction accuracy due to the gl…
Location-based Services (LBSs) provide valuable services, with convenient features for users. However, the information disclosed through each request harms user privacy. This is a concern particularly with honest-but-curious LBS servers, which could, by collecting requests, track users and infer additional sensitive user data. This is the motivation of both centralized and decentralized location privacy protection schemes for LBSs: anonymizing and obfuscating LBS queries to not disclose exact information, while still getting useful responses. Decentralized schemes overcome the disadvantages of…
Laboratory for Data Security -- EPFL（数据安全实验室 -- 洛桑联邦理工学院）
Tree-based models are among the most efficient machine learning techniques for data mining nowadays due to their accuracy, interpretability, and simplicity. The recent orthogonal needs for more data and privacy protection call for collaborative privacy-preserving solutions. In this work, we survey the literature on distributed and privacy-preserving training of tree-based models and we systematize its knowledge based on four axes: the learning algorithm, the collaborative model, the protection mechanism, and the threat model. We use this to identify the strengths and limitations of these works…
Inverting the Hidden: Unveiling Multimodal Privacy Leakage in Collaborative LVLM Inference
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Department of Statistics, Texas A&M University（德克萨斯农工大学统计系）；Nanchang University（南昌大学）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全国家重点实验室）；Hong Kong University of Science and Technology（香港科技大学）
Los Andes University（洛斯安第斯大学）；Rappi（Rappi公司）；Amazon（亚马逊）
Privacy-preserving machine learning in data-sharing processes is an ever-critical task that enables collaborative training of Machine Learning (ML) models without the need to share the original data sources. It is especially relevant when an organization must assure that sensitive data remains private throughout the whole ML pipeline, i.e., training and inference phases. This paper presents an innovative framework that uses Representation Learning via autoencoders to generate privacy-preserving embedded data. Thus, organizations can share the data representation to increase machine learning mo…
Collaborative filtering (CF) recommendation algorithms are well-known for their outstanding recommendation performances, but previous researches showed that they could cause privacy leakage for users due to k-nearest neighboring (KNN) attacks. Recently, the notion of differential privacy (DP) has been applied to privacy preservation for collaborative filtering recommendation algorithms. However, as far as we know, existing differentially private CF recommendation schemes degraded the recommendation performance (such as recall and precision) to an unacceptable level. In this paper, in order to …
Missouri University of Science and Technology（密苏里科技大学）；University of Missouri-Columbia（密苏里大学哥伦比亚分校）；Applied Computational Intelligence Laboratory, Missouri University of Science and Technology（密苏里科技大学应用计算智能实验室）
Collaborative filtering recommendation systems provide recommendations to users based on their own past preferences, as well as those of other users who share similar interests. The use of recommendation systems has grown widely in recent years, helping people choose which movies to watch, books to read, and items to buy. However, users are often concerned about their privacy when using such systems, and many users are reluctant to provide accurate information to most online services. Privacy-preserving collaborative filtering recommendation systems aim to provide users with accurate recommend…
To date publish of a giant social network jointly from different parties is an easier collaborative approach. Agencies and researchers who collect such social network data often have a compelling interest in allowing others to analyze the data. In many cases the data describes relationships that are private and sharing the data in full can result in unacceptable disclosures. Thus, preserving privacy without revealing sensitive information in the social network is a serious concern. Recent developments for preserving privacy using anonymization techniques are focused on relational data only. Pr…
Machine learning is promising, but it often needs to process vast amounts of sensitive data which raises concerns about privacy. In this white-paper, we introduce Substra, a distributed framework for privacy-preserving, traceable and collaborative Machine Learning. Substra gathers data providers and algorithm designers into a network of nodes that can train models on demand but under advanced permission regimes. To guarantee data privacy, Substra implements distributed learning: the data never leave their nodes; only algorithms, predictive models and non-sensitive metadata are exchanged on the…
When multiple parties that deal with private data aim for a collaborative prediction task such as medical image classification, they are often constrained by data protection regulations and lack of trust among collaborating parties. If done in a privacy-preserving manner, predictive analytics can benefit from the collective prediction capability of multiple parties holding complementary datasets on the same machine learning task. This paper presents PRICURE, a system that combines complementary strengths of secure multi-party computation (SMPC) and differential privacy (DP) to enable privacy-p…
Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院（LIST））
Collaborative filtering recommenders provide effective personalization services at the cost of sacrificing the privacy of their end users. Due to the increasing concerns from the society and stricter privacy regulations, it is an urgent research challenge to design privacy-preserving and yet robust recommenders which offer recommendation services to privacy-aware users. Our analysis shows that existing solutions fall short in several aspects, including lacking attention to the precise output to end users and ignoring the correlated robustness issues. In this paper, we provide a general system …
With increasing usage of deep learning algorithms in many application, new research questions related to privacy and adversarial attacks are emerging. However, the deep learning algorithm improvement needs more and more data to be shared within research community. Methodologies like federated learning, differential privacy, additive secret sharing provides a way to train machine learning models on edge without moving the data from the edge. However, it is very computationally intensive and prone to adversarial attacks. Therefore, this work introduces a privacy preserving FedCollabNN framework …
Vehicular ad hoc network (VANET) is an enabling technology in modern transportation systems for providing safety and valuable information, and yet vulnerable to a number of attacks from passive eavesdropping to active interfering. Intrusion detection systems (IDSs) are important devices that can mitigate the threats by detecting malicious behaviors. Furthermore, the collaborations among vehicles in VANETs can improve the detection accuracy by communicating their experiences between nodes. To this end, distributed machine learning is a suitable framework for the design of scalable and implement…
Emory University（埃默里大学）；UT Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）
Tensor factorization has been demonstrated as an efficient approach for computational phenotyping, where massive electronic health records (EHRs) are converted to concise and meaningful clinical concepts. While distributing the tensor factorization tasks to local sites can avoid direct data sharing, it still requires the exchange of intermediary results which could reveal sensitive patient information. Therefore, the challenge is how to jointly decompose the tensor under rigorous and principled privacy constraints, while still support the model's interpretability. We propose DPFact, a privacy-…
Machine learning (ML) methods have been widely used in genomic studies. However, genomic data are often held by different stakeholders (e.g. hospitals, universities, and healthcare companies) who consider the data as sensitive information, even though they desire to collaborate. To address this issue, recent works have proposed solutions using Secure Multi-party Computation (MPC), which train on the decentralized data in a way that the participants could learn nothing from each other beyond the final trained model. We design and implement several MPC-friendly ML primitives, including class wei…
The current business model for existing recommender services is centered around the availability of users' personal data at their side whereas consumers have to trust that the recommender service providers will not use their data in a malicious way. With the increasing number of cases for privacy breaches, different countries and corporations have issued privacy laws and regulations to define the best practices for the protection of personal information. The data protection directive 95/46/EC and the privacy principles established by the Organization for Economic Cooperation and Development (O…
Collaborative predictive blacklisting (CPB) allows to forecast future attack sources based on logs and alerts contributed by multiple organizations. Unfortunately, however, research on CPB has only focused on increasing the number of predicted attacks but has not considered the impact on false positives and false negatives. Moreover, sharing alerts is often hindered by confidentiality, trust, and liability issues, which motivates the need for privacy-preserving approaches to the problem. In this paper, we present a measurement study of state-of-the-art CPB techniques, aiming to shed light on t…
In recent years, gradient boosted decision tree learning has proven to be an effective method of training robust models. Moreover, collaborative learning among multiple parties has the potential to greatly benefit all parties involved, but organizations have also encountered obstacles in sharing sensitive data due to business, regulatory, and liability concerns. We propose Secure XGBoost, a privacy-preserving system that enables multiparty training and inference of XGBoost models. Secure XGBoost protects the privacy of each party's data as well as the integrity of the computation with the help…
Cloud computing has been a dominant paradigm for a variety of information processing platforms, particularly for enabling various popular applications of sharing economy. However, there is a major concern regarding data privacy on these cloud-based platforms. This work presents novel cloud-based privacy-preserving solutions to support collaborative consumption applications for sharing economy. In typical collaborative consumption, information processing platforms need to enable fair cost-sharing among multiple users for utilizing certain shared facilities and communal services. Our cloud-based…
Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding
The Hong Kong Polytechnic University（香港理工大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；School of Software, Tsinghua University（清华大学软件学院）
BME（布达佩斯技术与经济大学）；University of Cambridge（剑桥大学）；KUL（鲁汶大学）；Owkin（Owkin公司）
Pharmaceutical industry can better leverage its data assets to virtualize drug discovery through a collaborative machine learning platform. On the other hand, there are non-negligible risks stemming from the unintended leakage of participants' training data, hence, it is essential for such a platform to be secure and privacy-preserving. This paper describes a privacy risk assessment for collaborative modeling in the preclinical phase of drug discovery to accelerate the selection of promising drug candidates. After a short taxonomy of state-of-the-art inference attacks we adopt and customize se…
Machine learning algorithms have reached mainstream status and are widely deployed in many applications. The accuracy of such algorithms depends significantly on the size of the underlying training dataset; in reality a small or medium sized organization often does not have the necessary data to train a reasonably accurate model. For such organizations, a realistic solution is to train their machine learning models based on their joint dataset (which is a union of the individual ones). Unfortunately, privacy concerns prevent them from straightforwardly doing so. While a number of privacy-prese…
This work considers the problem of Distributed Mean Estimation (DME) over networks with intermittent connectivity, where the goal is to learn a global statistic over the data samples localized across distributed nodes with the help of a central server. To mitigate the impact of intermittent links, nodes can collaborate with their neighbors to compute local consensus which they forward to the central server. In such a setup, the communications between any pair of nodes must satisfy local differential privacy constraints. We study the tradeoff between collaborative relaying and privacy leakage d…
Hong Kong University of Science and Technology（香港科技大学）；Nokia Bell Labs（诺基亚贝尔实验室）；University of Nevada, Reno（内华达大学里诺分校）
With the advancement of machine learning (ML) and its growing awareness, many organizations who own data but not ML expertise (data owner) would like to pool their data and collaborate with those who have expertise but need data from diverse sources to train truly generalizable models (model owner). In such collaborative ML, the data owner wants to protect the privacy of its training data, while the model owner desires the confidentiality of the model and the training method which may contain intellectual properties. However, existing private ML solutions, such as federated learning and split …
Federated Inference: Toward Privacy-Preserving Collaborative and Incentivized Model Serving
Department of Electrical Engineering and Computer Science, University of Stavanger（电子工程与计算机科学系，斯塔万格大学）；Department of Data Science, The Catholic University of Korea（数据科学系，韩国天主大学）
The exponential growth of Internet of Things (IoT) has become a transcending force in creating innovative smart devices and connected domains including smart homes, healthcare, transportation and manufacturing. With billions of IoT devices, there is a huge amount of data continuously being generated, transmitted, and stored at various points in the IoT architecture. Deep learning is widely being used in IoT applications to extract useful insights from IoT data. However, IoT users have security and privacy concerns and prefer not to share their personal data with third party applications or sta…
University of Southern California (USC)（南加利福尼亚大学）；University of California, Riverside（加州大学河滨分校）
We consider a collaborative learning scenario in which multiple data-owners wish to jointly train a logistic regression model, while keeping their individual datasets private from the other parties. We propose COPML, a fully-decentralized training framework that achieves scalability and privacy-protection simultaneously. The key idea of COPML is to securely encode the individual datasets to distribute the computation load effectively across many parties and to perform the training computations as well as the model updates in a distributed manner on the securely encoded data. We provide the pri…
Wuhan University（武汉大学）；The State Key Laboratory of Cryptography（密码学国家重点实验室）；Huawei Technologies Co., Ltd（华为技术有限公司）
With powerful parallel computing GPUs and massive user data, neural-network-based deep learning can well exert its strong power in problem modeling and solving, and has archived great success in many applications such as image classification, speech recognition and machine translation etc. While deep learning has been increasingly popular, the problem of privacy leakage becomes more and more urgent. Given the fact that the training data may contain highly sensitive information, e.g., personal medical records, directly sharing them among the users (i.e., participants) or centrally storing them …
Brave Software（Brave软件公司）；Imperial College London（帝国理工学院）
Recommender systems are commonly trained on centrally collected user interaction data like views or clicks. This practice however raises serious privacy concerns regarding the recommender's collection and handling of potentially sensitive data. Several privacy-aware recommender systems have been proposed in recent literature, but comparatively little attention has been given to systems at the intersection of implicit feedback and privacy. To address this shortcoming, we propose a practical federated recommender system for implicit data under user-level local differential privacy (LDP). The pri…
Decentralised, Collaborative, and Privacy-preserving Machine Learning for Multi-Hospital Data
University of Toronto（多伦多大学）；University Health Network（大学健康网络）；Vector Institute（矢量研究所）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Simon Fraser University（西蒙菲莎大学）；St. Michael’s Hospital（圣迈克尔医院）；Unity Health Toronto（多伦多团结医疗）；Institute of Health Policy, Management and Evaluation, University of Toronto（多伦多大学卫生政策、管理与评价研究所）；Temerty Faculty of Medicine, University of Toronto（多伦多大学特梅蒂医学院）
