Asymmetric Private Set Intersection with Applications to Contact Tracing and Private Vertical Federated Machine Learning
Morfix；OpenMined；École Supérieure en Informatique, Sidi Bel Abbès（西迪贝勒阿巴斯高等计算机学院）；Bitdefender（比特梵德）；Edinburgh Napier University（爱丁堡龙比亚大学）；apheris AI；University of California Los Angeles（加利福尼亚大学洛杉矶分校）；Humboldt-Universität zu Berlin（柏林洪堡大学）；Tessella
Nick Angelou, Ayoub Benaissa, Bogdan Cebere, William Clark, Adam James Hall, Michael A. Hoeh, Daniel Liu, Pavlos Papadopoulos, Robin Roehm, Robert Sandmann, Phi… 展开作者
Nick Angelou, Ayoub Benaissa, Bogdan Cebere, William Clark, Adam James Hall, Michael A. Hoeh, Daniel Liu, Pavlos Papadopoulos, Robin Roehm, Robert Sandmann, Phillipp Schoppmann, Tom Titcombe
备注：
NeurIPS 2020 Workshop on Privacy Preserving Machine Learning (PPML 2020)
We present a multi-language, cross-platform, open-source library for asymmetric private set intersection (PSI) and PSI-Cardinality (PSI-C). Our protocol combines traditional DDH-based PSI and PSI-C protocols with compression based on Bloom filters that helps reduce communication in the asymmetric setting. Currently, our library supports C++, C, Go, WebAssembly, JavaScript, Python, and Rust, and runs on both traditional hardware (x86) and browser targets. We further apply our library to two use cases: (i) a privacy-preserving contact tracing protocol that is compatible with existing approaches, but improves their privacy guarantees, and (ii) privacy-preserving machine learning on vertically partitioned data.
We propose PrYVeCT, a private-yet-verifiable contact tracing system. PrYVeCT works also as an authorization framework allowing for the definition of fine-grained policies, which a certain facility can define and apply to better model its own access rules. Users are authorized to access the facility only when they exhibit a contact trace that complies with the policy. The policy evaluation process is carried out without disclosing the personal data of the user. At the same time, each user can prove to a third party (e.g., a public authority) that she received a certain authorization. PrYVeCT ta…
Federated learning has gained great attention recently as a privacy-enhancing tool to jointly train a machine learning model by multiple parties. As a sub-category, vertical federated learning (vFL) focuses on the scenario where features and labels are split into different parties. The prior work on vFL has mostly studied how to protect label privacy during model training. However, model evaluation in vFL might also lead to potential leakage of private label information. One mitigation strategy is to apply label differential privacy (DP) but it gives bad estimations of the true (non-private) m…
In this work we describe a token-based solution to Contact Tracing via Distributed Point Functions (DPF) and, more generally, Function Secret Sharing (FSS). The key idea behind the solution is that FSS natively supports secure keyword search on raw sets of keywords without a need for processing the keyword sets via a data structure for set membership. Furthermore, the FSS functionality enables adding up numerical payloads associated with multiple matches without additional interaction. These features make FSS an attractive tool for lightweight privacy-preserving searching on a database of toke…
Existing Bluetooth-based Private Contact Tracing (PCT) systems can privately detect whether people have come into direct contact with COVID-19 patients. However, we find that the existing systems lack functionality and flexibility, which may hurt the success of the contact tracing. Specifically, they cannot detect indirect contact (e.g., people may be exposed to coronavirus because of used the same elevator even without direct contact); they also cannot flexibly change the rules of "risky contact", such as how many hours of exposure or how close to a COVID-19 patient that is considered as risk…
Contact tracing is an important measure to counter the COVID-19 pandemic. In the early phase, many countries employed manual contact tracing to contain the rate of disease spread, however it has many issues. The manual approach is cumbersome, time consuming and also requires active participation of a large number of people to realize it. In order to overcome these drawbacks, digital contact tracing has been proposed that typically involves deploying a contact tracing application on people's mobile devices which can track their movements and close social interactions. While studies suggest that…
Throughout the course of the COVID-19 pandemic, several countries have developed and released contact tracing and exposure notification smartphone applications (apps) to help slow the spread of the disease. To support such apps, Apple and Google have released Exposure Notification Application Programming Interfaces (APIs) to infer device (user) proximity using Bluetooth Low Energy (BLE) beacons. The Private Automated Contact Tracing (PACT) team has shown that accurately estimating the distance between devices using only BLE radio signals is challenging. This paper describes the design and impl…
Debate about the adoption of digital contact tracing (DCT) apps to control the spread of COVID-19 has focussed on risks to individual privacy (Sharma & Bashir 2020, Tang 2020). This emphasis reveals significant challenges to ethical deployment of DCT, but generates constraints which undermine justification to implement DCT. It would be a mistake to view this result solely as the successful operation of ethical foresight analysis (Floridi & Strait 2020), preventing deployment of potentially harmful technology. Privacy-centric analysis treats data as private property, frames the relationship bet…
A successful machine learning (ML) algorithm often relies on a large amount of high-quality data to train well-performed models. Supervised learning approaches, such as deep learning techniques, generate high-quality ML functions for real-life applications, however with large costs and human efforts to label training data. Recent advancements in federated learning (FL) allow multiple data owners or organisations to collaboratively train a machine learning model without sharing raw data. In this light, vertical FL allows organisations to build a global model when the participating organisations…
Vertical federated learning (vFL) has gained much attention and been deployed to solve machine learning problems with data privacy concerns in recent years. However, some recent work demonstrated that vFL is vulnerable to privacy leakage even though only the forward intermediate embedding (rather than raw features) and backpropagated gradients (rather than raw labels) are communicated between the involved participants. As the raw labels often contain highly sensitive information, some recent work has been proposed to prevent the label leakage from the backpropagated gradients effectively in vF…
Especially in the Big Data era, the usage of different classification methods is increasing day by day. The success of these classification methods depends on the effectiveness of learning methods. Extreme learning machine (ELM) classification algorithm is a relatively new learning method built on feed-forward neural-network. ELM classification algorithm is a simple and fast method that can create a model from high-dimensional data sets. Traditional ELM learning algorithm implicitly assumes complete access to whole data set. This is a major privacy concern in most of cases. Sharing of private …
Vertical federated learning (VFL) allows an active party with labeled feature to leverage auxiliary features from the passive parties to improve model performance. Concerns about the private feature and label leakage in both the training and inference phases of VFL have drawn wide research attention. In this paper, we propose a general privacy-preserving vertical federated deep learning framework called FedPass, which leverages adaptive obfuscation to protect the feature and label simultaneously. Strong privacy-preserving capabilities about private features and labels are theoretically proved …
Covid-19 is primarily spread through contact with the virus which may survive on surfaces with lifespan of more than hours. To curb its spread, it is hence of vital importance to detect and quarantine those who have been in contact with the virus for sustained period of time, the so-called close contacts. In this work, we study, for the first time, automatic contact detection when the virus has a lifespan. Leveraging upon the ubiquity of WiFi signals, we propose a novel, private, and fully distributed WiFi-based approach called vContact. Users installing an app continuously scan WiFi and store…
Communication-Efficient and Differentially Private Vertical Federated Learning with Zeroth-Order Optimization
Elmore Family School of Electrical and Computer Engineering, Purdue University（电气与计算机工程学院，普渡大学）；Department of Computer Science and Engineering, Yonsei University（计算机科学与工程系，延世大学）
Contact tracing is an essential tool in containing infectious diseases such as COVID-19. Many countries and research groups have launched or announced mobile apps to facilitate contact tracing by recording contacts between users with some privacy considerations. Most of the focus has been on using random tokens, which are exchanged during encounters and stored locally on users' phones. Prior systems allow users to search over released tokens in order to learn if they have recently been in the proximity of a user that has since been diagnosed with the disease. However, prior approaches do not p…
Despite much study, the computational complexity of differential privacy remains poorly understood. In this paper we consider the computational complexity of accurately answering a family $Q$ of statistical queries over a data universe $X$ under differential privacy. A statistical query on a dataset $D \in X^n$ asks "what fraction of the elements of $D$ satisfy a given predicate $p$ on $X$?" Dwork et al. (STOC'09) and Boneh and Zhandry (CRYPTO'14) showed that if both $Q$ and $X$ are of polynomial size, then there is an efficient differentially private algorithm that accurately answers all the …
Governments, Healthcare, and Private Organizations in the global scale have been using digital tracking to keep COVID-19 outbreaks under control. Although this method could limit pandemic contagion, it raises significant concerns about user privacy. Known as ~"Contact Tracing Apps", these mobile applications are facilitated by Cellphone Service Providers (CSPs), who enable the spatial and temporal real-time user tracking. Accordingly, it might be speculated that CSPs collect information violating the privacy policies such as GDPR, CCPA, and others. To further clarify, we conducted an in-depth …
Several contact tracing solutions have been proposed and implemented all around the globe to combat the spread of COVID-19 pandemic. But, most of these solutions endanger the privacy rights of the individuals and hinder their widespread adoption. We propose a privacy-preserving contact tracing protocol for the efficient tracing of the spread of the global pandemic. It is based on the private set intersection (PSI) protocol and utilizes the homomorphic properties to preserve the privacy at the individual level. A hierarchical model for the representation of landscapes and rate-limiting factor o…
While sharing files in a peer-to-peer (P2P) system significantly increases both the speed of retrieving the contents and the robustness of the system, tracing the access of files is not straightforward, even in the case of private P2P networks. In fact, a participant that has uploaded a file to a P2P network is not necessarily involved in its download. Additionally, due to the nature of the P2P network it is possible for a participant to already have all the fragments of a file, even before requesting it. This work tries to address the problem of tracing file access in a private P2P file shari…
Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所（HIIT））；Aalto University（阿尔托大学）；University of Manchester（曼彻斯特大学）；University of Helsinki（赫尔辛基大学）
In this work, we introduce a differentially private method for generating synthetic data from vertically partitioned data, \emph{i.e.}, where data of the same individuals is distributed across multiple data holders or parties. We present a differentially privacy stochastic gradient descent (DP-SGD) algorithm to train a mixture model over such partitioned data using variational inference. We modify a secure multiparty computation (MPC) framework to combine MPC with differential privacy (DP), in order to use differentially private MPC effectively to learn a probabilistic generative model under D…
Federated learning is a distributed machine learning method that aims to preserve the privacy of sample features and labels. In a federated learning system, ID-based sample alignment approaches are usually applied with few efforts made on the protection of ID privacy. In real-life applications, however, the confidentiality of sample IDs, which are the strongest row identifiers, is also drawing much attention from many participants. To relax their privacy concerns about ID privacy, this paper formally proposes the notion of asymmetrical vertical federated learning and illustrates the way to pro…
The COVID-19 pandemic has prompted technological measures to control the spread of the disease. Private contact tracing (PCT) is one of the promising techniques for the purpose. However, the recently proposed Bluetooth-based PCT has several limitations in terms of functionality and flexibility. The existing systems are only able to detect direct contact (i.e., human-human contact), but cannot detect indirect contact (i.e., human-object, such as the disease transmission through surface). Moreover, the rule of risky contact cannot be flexibly changed with the environmental situation and the natu…
OpenMined；Edinburgh Napier University（爱丁堡龙比亚大学）；Apheris；Tessella；Birla Institute of Technology, Mesra（比拉理工学院（梅斯拉校区））
We introduce PyVertical, a framework supporting vertical federated learning using split neural networks. The proposed framework allows a data scientist to train neural networks on data features vertically partitioned across multiple owners while keeping raw data on an owner's device. To link entities shared across different datasets' partitions, we use Private Set Intersection on IDs associated with data points. To demonstrate the validity of the proposed framework, we present the training of a simple dual-headed split neural network for a MNIST classification task, with data samples verticall…
Eliminating Label Leakage in Tree-Based Vertical Federated Learning
The University of Tokyo（东京大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Split learning is a distributed training framework that allows multiple parties to jointly train a machine learning model over vertically partitioned data (partitioned by attributes). The idea is that only intermediate computation results, rather than private features and labels, are shared between parties so that raw training data remains private. Nevertheless, recent works showed that the plaintext implementation of split learning suffers from severe privacy risks that a semi-honest adversary can easily reconstruct labels. In this work, we propose \textsf{TPSL} (Transcript Private Split Lear…
During a pandemic, contact tracing is an essential tool to drive down the infection rate within a population. To accelerate the laborious manual contact tracing process, digital contact tracing (DCT) tools can track contact events transparently and privately by using the sensing and signaling capabilities of the ubiquitous cell phone. However, an effective DCT must not only preserve user privacy but also augment the existing manual contact tracing process. Indeed, not every member of a population may own a cell phone or have a DCT app installed and enabled. We present KHOVID to fulfill the com…
National University of Singapore（新加坡国立大学）；Zhejiang University（浙江大学）
Federated learning (FL) is an emerging paradigm that enables multiple organizations to jointly train a model without revealing their private data to each other. This paper studies {\it vertical} federated learning, which tackles the scenarios where (i) collaborating organizations own data of the same set of users but with disjoint features, and (ii) only one organization holds the labels. We propose Pivot, a novel solution for privacy preserving vertical decision tree training and prediction, ensuring that no intermediate information is disclosed other than those the clients have agreed to rel…
Federated learning (FL) is a privacy-preserving learning paradigm that allows multiple parities to jointly train a powerful machine learning model without sharing their private data. According to the form of collaboration, FL can be further divided into horizontal federated learning (HFL) and vertical federated learning (VFL). In HFL, participants share the same feature space and collaborate on data samples, while in VFL, participants share the same sample IDs and collaborate on features. VFL has a broader scope of applications and is arguably more suitable for joint model training between lar…
Due to the rising concerns on privacy protection, how to build machine learning (ML) models over different data sources with security guarantees is gaining more popularity. Vertical federated learning (VFL) describes such a case where ML models are built upon the private data of different participated parties that own disjoint features for the same set of instances, which fits many real-world collaborative tasks. Nevertheless, we find that existing solutions for VFL either support limited kinds of input features or suffer from potential data leakage during the federated execution. To this end,…
Vertical Federated Learning for Effectiveness, Security, Applicability: A Survey
The National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；School of Computer Science, Wuhan University（武汉大学计算机学院）；United Institute of Informatics Problems of Belarus National Academy of Sciences（白俄罗斯国家科学院信息问题联合研究所）；National Academy of Sciences of Belarus（白俄罗斯国家科学院）；Department of Computer Science, Hong Kong Baptist University（香港浸会大学计算机系）
University of Surrey（萨里大学）；Delft University of Technology（代尔夫特理工大学）
Federated learning (FL) is an emerging privacy preserving machine learning protocol that allows multiple devices to collaboratively train a shared global model without revealing their private local data. Non-parametric models like gradient boosting decision trees (GBDT) have been commonly used in FL for vertically partitioned data. However, all these studies assume that all the data labels are stored on only one client, which may be unrealistic for real-world applications. Therefore, in this work, we propose a secure vertical FL framework, named PIVODL, to train GBDT with data labels distribut…
Recently, Graph Neural Network (GNN) has achieved remarkable progresses in various real-world tasks on graph data, consisting of node features and the adjacent information between different nodes. High-performance GNN models always depend on both rich features and complete edge information in graph. However, such information could possibly be isolated by different data holders in practice, which is the so-called data isolation problem. To solve this problem, in this paper, we propose VFGNN, a federated GNN learning paradigm for privacy-preserving node classification task under data vertically …
The 2020 COVID-19 pandemic has led to a global lockdown with severe health and economical consequences. As a result, authorities around the globe have expressed their needs for better tools to monitor the spread of the virus and to support human labor. Researchers and technology companies such as Google and Apple have offered to develop such tools in the form of contact tracing applications. The goal of these applications is to continuously track people's proximity and to make the smartphone users aware if they have ever been in contact with positively diagnosed people, so that they could self…
A central problem in differentially private data analysis is how to design efficient algorithms capable of answering large numbers of counting queries on a sensitive database. Counting queries of the form "What fraction of individual records in the database satisfy the property q?" We prove that if one-way functions exist, then there is no algorithm that takes as input a database D in ({0,1}^d)^n, and k = n^{2+o(1)} arbitrary efficiently computable counting queries, runs in time poly(d, n), and returns an approximate answer to each query, while satisfying differential privacy. We also consider…
Revisiting Label Inference Attacks in Vertical Federated Learning: Why They Are Vulnerable and How to Defend
Key Laboratory of High Confidence Software Technologies (Peking University), Ministry of Education（高可信软件技术重点实验室（北京大学））；School of Computer Science, Peking University（北京大学计算机学院）；Zhongguancun Laboratory, Beijing, China（中关村实验室，北京，中国）
University of Pennsylvania（宾夕法尼亚大学）；Rutgers University（罗格斯大学）
We propose differentially private algorithms for parameter estimation in both low-dimensional and high-dimensional sparse generalized linear models (GLMs) by constructing private versions of projected gradient descent. We show that the proposed algorithms are nearly rate-optimal by characterizing their statistical performance and establishing privacy-constrained minimax lower bounds for GLMs. The lower bounds are obtained via a novel technique, which is based on Stein's Lemma and generalizes the tracing attack technique for privacy-constrained lower bounds. This lower bound argument can be of …
Activity-tracking applications and location-based services using short-range communication (SRC) techniques have been abruptly demanded in the COVID-19 pandemic, especially for automated contact tracing. The attention from both public and policy keeps raising on related practical problems, including \textit{1) how to protect data security and location privacy? 2) how to efficiently and dynamically deploy SRC Internet of Thing (IoT) witnesses to monitor large areas?} To answer these questions, in this paper, we propose a decentralized and permissionless blockchain protocol, named \textit{Bychai…
IBM Research（IBM研究院）；University of Pittsburgh（匹兹堡大学）
Federated learning (FL) has been proposed to allow collaborative training of machine learning (ML) models among multiple parties where each party can keep its data private. In this paradigm, only model updates, such as model weights or gradients, are shared. Many existing approaches have focused on horizontal FL, where each party has the entire feature set and labels in the training data set. However, many real scenarios follow a vertically-partitioned FL setup, where a complete feature set is formed only when all the datasets from the parties are combined, and the labels are only available to…
Distributed health data networks that use information from multiple sources have drawn substantial interest in recent years. However, missing data are prevalent in such networks and present significant analytical challenges. The current state-of-the-art methods for handling missing data require pooling data into a central repository before analysis, which may not be possible in a distributed health data network. In this paper, we propose a privacy-preserving distributed analysis framework for handling missing data when data are vertically partitioned. In this framework, each institution with a…
