Recently, deep learning, which uses Deep Neural Networks (DNN), plays an important role in many fields. A secure neural network model with a secure training/inference scheme is indispensable to many applications. To accomplish such a task usually needs one of the entities (the customer or the service provider) to provide private information (customer's data or the model) to the other. Without a secure scheme and the mutual trust between the service providers and their customers, it will be an impossible mission. In this paper, we propose a novel privacy-preserving deep learning model and a secure training/inference scheme to protect the input, the output, and the model in the application of the neural network. We utilize the innate properties of a deep neural network to design a secure mechanism without using any complicated cryptography component. The security analysis shows our proposed scheme is secure and the experimental results also demonstrate that our method is very efficient and suitable for real applications.
Differential privacy (DP) has arisen as the state-of-the-art metric for quantifying individual privacy when sensitive data are analyzed, and it is starting to see practical deployment in organizations such as the US Census Bureau, Apple, Google, etc. There are two popular models for deploying differential privacy - standard differential privacy (SDP), where a trusted server aggregates all the data and runs the DP mechanisms, and local differential privacy (LDP), where each user perturbs their own data and perturbed data is analyzed. Due to security concerns arising from aggregating raw data at…
Privacy Preserving Data Mining is a method which ensures privacy of individual information during mining. Most important task involves retrieving information from multiple data bases which is distributed. The data once in the data warehouse can be used by mining algorithms to retrieve confidential information. The proposed framework has two major tasks, secure transmission and privacy of confidential information during mining. Secure transmission is handled by using elliptic curve cryptography and data distortion for privacy preservation ensuring highly secure environment.
Due to successful applications of data analysis technologies in many fields, various institutions have accumulated a large amount of data to improve their services. As the speed of data collection has increased dramatically over the last few years, an increasing number of users are growing concerned about their personal information. Therefore, privacy preservation has become an urgent problem to be solved. Differential privacy as a strong privacy preservation tool has attracted significant attention. In this survey, we focus on improving utility of between differentially private mechanisms thr…
A boardroom election is an election that takes place in a single room -- the boardroom -- in which all voters can see and hear each other. We present an initial exploration of boardroom elections with ballot privacy and voter verifiability that use only "low-tech cryptography" without using computers to mark or collect ballots. Specifically, we define the problem, introduce several building blocks, and propose a new protocol that combines these blocks in novel ways. Our new building blocks include "foldable ballots" that can be rotated to hide the alignment of ballot choices with voting marks,…
Recent work has shown that cell phone mobility data has the unique potential to create accurate models for human mobility and consequently the spread of infected diseases. While prior studies have exclusively relied on a mobile network operator's subscribers' aggregated data in modelling disease dynamics, it may be preferable to contemplate aggregated mobility data of infected individuals only. Clearly, naively linking mobile phone data with health records would violate privacy by either allowing to track mobility patterns of infected individuals, leak information on who is infected, or both. …
Bloomberg LP（彭博有限合伙企业）；University of Edinburgh（爱丁堡大学）
Privacy is a major issue in learning from distributed data. Recently the cryptographic literature has provided several tools for this task. However, these tools either reduce the quality/accuracy of the learning algorithm---e.g., by adding noise---or they incur a high performance penalty and/or involve trusting external authorities. We propose a methodology for {\sl private distributed machine learning from light-weight cryptography} (in short, PD-ML-Lite). We apply our methodology to two major ML algorithms, namely non-negative matrix factorization (NMF) and singular value decomposition (SVD)…
The emerging technologies for large scale data analysis raise new challenges to the security and privacy of sensitive user data. In this work we investigate the problem of private statistical analysis of time-series data in the distributed and semi-honest setting. In particular, we study some properties of Private Stream Aggregation (PSA), first introduced by Shi et al. 2017. This is a computationally secure protocol for the collection and aggregation of data in a distributed network and has a very small communication cost. In the non-adaptive query model, a secure PSA scheme can be built upon…
Smartphones are quickly moving toward complementing or even replacing traditional car keys. We advocate a role-based access control policy mixed with attributes that facilitates access to various functionalities of vehicular on-board units from smartphones. We use a rights-based access control policy for in-vehicle functionalities similar to the case of a file allocation table of a contemporary OS, in which read, write or execute operations can be performed over various vehicle functions. Further, to assure the appropriate security, we develop a protocol suite using identity-based cryptography…
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Mathematics and Computer Science, Fuzhou University（福州大学数学与计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Mobile crowdsensing (MCS) is an emerging sensing data collection pattern with scalability, low deployment cost, and distributed characteristics. Traditional MCS systems suffer from privacy concerns and fair reward distribution. Moreover, existing privacy-preserving MCS solutions usually focus on the privacy protection of data collection rather than that of data processing. To tackle faced problems of MCS, in this paper, we integrate federated learning (FL) into MCS and propose a privacy-preserving MCS system, called \textsc{CrowdFL}. Specifically, in order to protect privacy, participants loca…
Iowa State University（爱荷华州立大学）；University of South Florida（南佛罗里达大学）
For privacy concerns to be addressed adequately in current machine learning systems, the knowledge gap between the machine learning and privacy communities must be bridged. This article aims to provide an introduction to the intersection of both fields with special emphasis on the techniques used to protect the data.
Cryptography and data science research grew exponential with the internet boom. Legacy encryption techniques force users to make a trade-off between usability, convenience, and security. Encryption makes valuable data inaccessible, as it needs to be decrypted each time to perform any operation. Billions of dollars could be saved, and millions of people could benefit from cryptography methods that don't compromise between usability, convenience, and security. Homomorphic encryption is one such paradigm that allows running arbitrary operations on encrypted data. It enables us to run any sophisti…
Laboratory for Data Security -- EPFL（数据安全实验室 -- 洛桑联邦理工学院）
Tree-based models are among the most efficient machine learning techniques for data mining nowadays due to their accuracy, interpretability, and simplicity. The recent orthogonal needs for more data and privacy protection call for collaborative privacy-preserving solutions. In this work, we survey the literature on distributed and privacy-preserving training of tree-based models and we systematize its knowledge based on four axes: the learning algorithm, the collaborative model, the protection mechanism, and the threat model. We use this to identify the strengths and limitations of these works…
Technical University of Munich（慕尼黑工业大学）；Bytedance Inc.（字节跳动公司）；Massachusetts Institute of Technology（麻省理工学院）
Federated learning is the distributed machine learning framework that enables collaborative training across multiple parties while ensuring data privacy. Practical adaptation of XGBoost, the state-of-the-art tree boosting framework, to federated learning remains limited due to high cost incurred by conventional privacy-preserving methods. To address the problem, we propose two variants of federated XGBoost with privacy guarantee: FedXGBoost-SMM and FedXGBoost-LDP. Our first protocol FedXGBoost-SMM deploys enhanced secure matrix multiplication method to preserve privacy with lossless accuracy a…
In the current pandemic, testing continues to be the most important tool for monitoring and curbing the disease spread and early identification of the disease to perform health-related interventions like quarantine, contact tracing and etc. Therefore, the ability to verify the testing status is pertinent as public places prepare to safely open. Recent advances in cryptographic tools have made it possible to build a secure and resilient digital-id system. In this work, we propose to build an end to end COVID-19 results verification protocol that takes privacy, computation, and other practical c…
Vanderbilt University（范德堡大学）；University of Virginia（弗吉尼亚大学）
Safeguarding privacy in machine learning is highly desirable, especially in collaborative studies across many organizations. Privacy-preserving distributed machine learning (based on cryptography) is popular to solve the problem. However, existing cryptographic protocols still incur excess computational overhead. Here, we make a novel observation that this is partially due to naive adoption of mainstream numerical optimization (e.g., Newton method) and failing to tailor for secure computing. This work presents a contrasting perspective: customizing numerical optimization specifically for secur…
Cryptography has been widely accepted for security and partly for privacy control as discovered from past works. However, many of these works did not provide a way to manage cryptographic keys effectively especially in EHR applications, as this is the Achilles heel of cryptographic techniques currently proposed. The issue of accountability for legitimate users also has not been so popular and only a few considered it in EHR. Unless a different approach is used, the reliant on cryptography and password or escrow based system for key management will impede trust of the system and hence its accep…
Generating secure random numbers is a central problem in cryptography that needs a reliable source of enough computing entropy. Without enough entropy available - meaning no good source of secure random numbers - a device is susceptible to cryptographic protocol failures such as weak, factorable, or predictable keys, which lead to various security and privacy vulnerabilities. In this paper, the author presents a significant improvement: a reliable way for any CPU-powered device - from the small, simple CPUs in embedded devices, to larger, more complex CPUs in modern servers - to collect virtua…
Imperial College London（帝国理工学院）；DeepMind（深度思维）；University of Oxford（牛津大学）；Case Western Reserve University（凯斯西储大学）
We detail a new framework for privacy preserving deep learning and discuss its assets. The framework puts a premium on ownership and secure processing of data and introduces a valuable representation based on chains of commands and tensors. This abstraction allows one to implement complex privacy preserving constructs such as Federated Learning, Secure Multiparty Computation, and Differential Privacy while still exposing a familiar deep learning API to the end-user. We report early results on the Boston Housing and Pima Indian Diabetes datasets. While the privacy features apart from Differenti…
Federated learning (FL) has recently become a hot research topic, in which Byzantine robustness, communication efficiency and privacy preservation are three important aspects. However, the tension among these three aspects makes it hard to simultaneously take all of them into account. In view of this challenge, we theoretically analyze the conditions that a communication compression method should satisfy to be compatible with existing Byzantine-robust methods and privacy-preserving methods. Motivated by the analysis results, we propose a novel communication compression method called consensus …
While advances in quantum computing promise new opportunities for scientific advancement (e.g., material science and machine learning), many people are not aware that they also threaten the widely deployed cryptographic algorithms that are the foundation of today's digital security and privacy. From mobile communications to online banking to personal data privacy, literally billions of Internet users rely on cryptography every day to ensure that private communications and data stay private. Indeed, the emergence and growth of the public Internet and electronic commerce was arguably enabled by …
In the course of a survey of privacy-preserving record linkage, we reviewed the approach taken by the UK Office for National Statistics (ONS) as described in their series of reports "Beyond 2011". Our review identifies a number of matters of concern. Some of the issues discovered are sufficiently severe to present a risk to privacy.
In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but to also protect the privacy of test (query) images for the first time. In addition, it can also maintain the same accuracy as normally trained models. In an experiment, the proposed method was demonstr…
In this paper, we review the state of the art of privacy-preserving schemes for ad hoc social networks, including, mobile social networks (MSNs) and vehicular social networks (VSNs). Specifically, we select and in-detail examine thirty-three privacy preserving schemes developed for or applied in the context of ad hoc social networks. These schemes are published between 2008 and 2016. Based on this existing privacy preservation schemes, we survey privacy preservation models, including location privacy, identity privacy, anonymity, traceability, interest privacy, backward privacy, and content or…
University College London（伦敦大学学院）；The Alan Turing Institute（艾伦·图灵研究所）
Our everyday interactions with pervasive systems generate traces that capture various aspects of human behavior and enable machine learning algorithms to extract latent information about users. In this paper, we propose a machine learning interpretability framework that enables users to understand how these generated traces violate their privacy.
In Advanced Metering Infrastructure (AMI) systems, smart meters (SM) send fine-grained power consumption information to the utility company, yet this power consumption information can uncover sensitive information about the consumers' lifestyle. To allow the utility company to gather the power consumption information while safeguarding the consumers' privacy, different methods that broadly utilize symmetric key and asymmetric key cryptography operation have been generally utilized. In this paper, we propose an effective method that uses symmetric key cryptography and hashing operation to gathe…
We introduce a formal model for membership and order queries on privacy-preserving authenticated lists. In this model, the queries are performed on the list stored in the cloud where data integrity and privacy have to be maintained. We then present an efficient construction of privacy-preserving authenticated lists based on bilinear accumulators and bilinear maps, analyze the performance, and prove the integrity and privacy of this construction under widely accepted assumptions.
With increasing usage of deep learning algorithms in many application, new research questions related to privacy and adversarial attacks are emerging. However, the deep learning algorithm improvement needs more and more data to be shared within research community. Methodologies like federated learning, differential privacy, additive secret sharing provides a way to train machine learning models on edge without moving the data from the edge. However, it is very computationally intensive and prone to adversarial attacks. Therefore, this work introduces a privacy preserving FedCollabNN framework …
Luxembourg Institute of Science and Technology (LIST)（卢森堡科学技术研究院（LIST））
Collaborative filtering recommenders provide effective personalization services at the cost of sacrificing the privacy of their end users. Due to the increasing concerns from the society and stricter privacy regulations, it is an urgent research challenge to design privacy-preserving and yet robust recommenders which offer recommendation services to privacy-aware users. Our analysis shows that existing solutions fall short in several aspects, including lacking attention to the precise output to end users and ignoring the correlated robustness issues. In this paper, we provide a general system …
Establishing how a set of learners can provide privacy-preserving federated learning in a fully decentralized (peer-to-peer, no coordinator) manner is an open problem. We propose the first privacy-preserving consensus-based algorithm for the distributed learners to achieve decentralized global model aggregation in an environment of high mobility, where the communication graph between the learners may vary between successive rounds of model aggregation. In particular, in each round of global model aggregation, the Metropolis-Hastings method is applied to update the weighted adjacency matrix bas…
Vertical federated learning (VFL) allows an active party with labeled feature to leverage auxiliary features from the passive parties to improve model performance. Concerns about the private feature and label leakage in both the training and inference phases of VFL have drawn wide research attention. In this paper, we propose a general privacy-preserving vertical federated deep learning framework called FedPass, which leverages adaptive obfuscation to protect the feature and label simultaneously. Strong privacy-preserving capabilities about private features and labels are theoretically proved …
Credit risk modeling has permeated our everyday life. Most banks and financial companies use this technique to model their clients' trustworthiness. While machine learning is increasingly used in this field, the resulting large-scale collection of user private information has reinvigorated the privacy debate, considering dozens of data breach incidents every year caused by unauthorized hackers, and (potentially even more) information misuse/abuse by authorized parties. To address those critical concerns, this paper proposes a framework of Privacy-preserving Credit risk modeling based on Advers…
In this work, we study the problem of privacy preserving computation on PageRank algorithm. The idea is to enforce the secure multi party computation of the algorithm iteratively using homomorphic encryption based on Paillier scheme. In the proposed PageRank computation, a user encrypt its own graph data using asymmetric encryption method, sends the data set into different parties in a privacy-preserving manner. Each party computes its own encrypted entity, but learns nothing about the data at other parties.
The question of how government agencies can acquire actionable, useful information about legitimate but unknown targets without intruding upon the electronic activity of innocent parties is extremely important. We address this question by providing experimental evidence that actionable, useful information can indeed be obtained in a manner that preserves the privacy of innocent parties and that holds government agencies accountable. In particular, we present practical, privacy-preserving protocols for two operations that law-enforcement and intelligence agencies have used effectively: set inte…
Can we sense our location in an unfamiliar environment by taking a sublinear-size sample of our surroundings? Can we efficiently encrypt a message that only someone physically close to us can decrypt? To solve this kind of problems, we introduce and study a new type of hash functions for finding shifts in sublinear time. A function $h:\{0,1\}^n\to \mathbb{Z}_n$ is a $(d,δ)$ {\em locality-preserving hash function for shifts} (LPHS) if: (1) $h$ can be computed by (adaptively) querying $d$ bits of its input, and (2) $\Pr [ h(x) \neq h(x \ll 1) + 1 ] \leq δ$, where $x$ is random and $\ll 1$ denote…
Data markets have the potential to foster new data-driven applications and help growing data-driven businesses. When building and deploying such markets in practice, regulations such as the European Union's General Data Protection Regulation (GDPR) impose constraints and restrictions on these markets especially when dealing with personal or privacy-sensitive data. In this paper, we present a candidate architecture for a privacy-preserving personal data market, relying on cryptographic primitives such as multi-party computation (MPC) capable of performing privacy-preserving computations on the …
In this early draft, we present an end-to-end decentralized protocol for the secure and privacy preserving workflow of vaccination, vaccination status verification, and adverse reactions or symptoms reporting. The proposed system improves the efficiency, privacy, equity, and effectiveness of the existing manual system while remaining interoperable with its capabilities. We also discuss various security concerns and alternate methodologies based on the proposed protocols.
Secure cloud storage is an issue of paramount importance that both businesses and end-users should take into consideration before moving their data to, potentially, untrusted clouds. Migrating data to the cloud raises multiple privacy issues, as they are completely controlled by a cloud provider. Hence, an untrusted cloud provider can potentially breach users; privacy and gain access to sensitive information. The problem becomes even more pronounced when the could provider is required to store a statistical database and periodically publish analytics. In this work, we first present a detailed …
In privacy-preserving machine learning, individual parties are reluctant to share their sensitive training data due to privacy concerns. Even the trained model parameters or prediction can pose serious privacy leakage. To address these problems, we demonstrate a generally applicable Distributed Privacy-Preserving Prediction (DPPP) framework, in which instead of sharing more sensitive data or model parameters, an untrusted aggregator combines only multiple models' predictions under provable privacy guarantee. Our framework integrates two main techniques to guarantee individual privacy. First, w…
In recent years, several studies proposed privacy-preserving algorithms for solving Distributed Constraint Optimization Problems (DCOPs). All of those studies assumed that agents do not collude. In this study we propose the first privacy-preserving DCOP algorithm that is immune to coalitions, under the assumption of honest majority. Our algorithm -- PC-SyncBB -- is based on the classical Branch and Bound DCOP algorithm. It offers constraint, topology and decision privacy. We evaluate its performance on different benchmarks, problem sizes, and constraint densities. We show that achieving securi…
A privacy-preserving Context-Aware Publish-Subscribe System (CA-PSS) enables an intermediary (broker) to match the content from a publisher and the subscription by a subscriber based on the current context while preserving confidentiality of the subscriptions and notifications. While a privacy-preserving Ride-Hailing Service (RHS) enables an intermediary (service provider) to match a ride request with a taxi driver in a privacy-friendly manner. In this work, we attack a privacy-preserving CA-PSS proposed by Nabeel et al. (2013), where we show that any entity in the system including the broker …
Auction is widely regarded as an effective way in dynamic spectrum redistribution. Recently, considerable research efforts have been devoted to designing privacy-preserving spectrum auctions in a variety of auction settings. However, none of existing work has addressed the privacy issue in the most generic scenario, double spectrum auctions where each seller sells multiple channels and each buyer buys multiple channels. To fill this gap, in this paper we propose PP-MCSA, a Privacy Preserving mechanism for Multi-Channel double Spectrum Auctions. Technically, by leveraging garbled circuits, we m…
Recently, Xie et al. (IEEE Transactions on Information Forensics and Security, vol. 16, pp. 3068-3081, 2021) proposed a privacy-preserving Online Ride-Hailing (ORH) protocol that does not make use of a trusted third-party server. The primary goal of such privacy-preserving ORH protocols is to ensure the privacy of riders' and drivers' location data w.r.t. the ORH Service Provider (SP). In this note, we demonstrate a passive attack by the SP in the protocol of Xie et al. that enables it to completely recover the location of the rider as well as that of the responding drivers in each and every r…
Collaboration among multiple organizations is imperative for contemporary intrusion detection. As modern threats become well sophisticated it is difficult for organizations to defend with threat context local to their networks alone. Availability of global \emph{threat intelligence} is must for organizations to defend against modern advanced persistent threats (APTs). In order to benefit from such global context of attacks, privacy concerns continue to be of major hindrance. In this position paper we identify real world privacy problems as precise use cases, relevant cryptographic technologies…
