DP-DCAN: Differentially Private Deep Contrastive Autoencoder Network for Single-cell Clustering
East China Normal University（华东师范大学）；Shanghai Key Laboratory of Trustworthy Computing（上海市可信计算重点实验室）；Xinhua Hospital（新华医院）；Shanghai Jiao Tong University（上海交通大学）
Single-cell RNA sequencing (scRNA-seq) is important to transcriptomic analysis of gene expression. Recently, deep learning has facilitated the analysis of high-dimensional single-cell data. Unfortunately, deep learning models may leak sensitive information about users. As a result, Differential Privacy (DP) is increasingly used to protect privacy. However, existing DP methods usually perturb whole neural networks to achieve differential privacy, and hence result in great performance overheads. To address this challenge, in this paper, we take advantage of the uniqueness of the autoencoder that it outputs only the dimension-reduced vector in the middle of the network, and design a Differentially Private Deep Contrastive Autoencoder Network (DP-DCAN) by partial network perturbation for single-cell clustering. Since only partial network is added with noise, the performance improvement is obvious and twofold: one part of network is trained with less noise due to a bigger privacy budget, and the other part is trained without any noise. Experimental results of six datasets have verified that DP-DCAN is superior to the traditional DP scheme with whole network perturbation. Moreover, DP-DCAN demonstrates strong robustness to adversarial attacks.
Federated learning (FL) is getting increased attention for processing sensitive, distributed datasets common to domains such as healthcare. Instead of directly training classification models on these datasets, recent works have considered training data generators capable of synthesising a new dataset which is not protected by any privacy restrictions. Thus, the synthetic data can be made available to anyone, which enables further evaluation of machine learning architectures and research questions off-site. As an additional layer of privacy-preservation, differential privacy can be introduced i…
Machine learning has been applied to almost all fields of computer science over the past decades. The introduction of GANs allowed for new possibilities in fields of medical research and text prediction. However, these new fields work with ever more privacy-sensitive data. In order to maintain user privacy, a combination of federated learning, differential privacy and GANs can be used to work with private data without giving away a users' privacy. Recently, two implementations of such combinations have been published: DP-Fed-Avg GAN and GS-WGAN. This paper compares their performance and introd…
Software Competence Center Hagenberg（哈根贝格软件能力中心）；University of Rostock（罗斯托克大学）
This paper considers the problem of differentially private semi-supervised transfer and multi-task learning. The notion of \emph{membership-mapping} has been developed using measure theory basis to learn data representation via a fuzzy membership function. An alternative conception of deep autoencoder, referred to as \emph{Conditionally Deep Membership-Mapping Autoencoder (CDMMA)}, is considered for transferrable deep learning. Under practice-oriented settings, an analytical solution for the learning of CDMMA can be derived by means of variational optimization. The paper proposes a transfer an…
Deep learning models have demonstrated superior performance in several application problems, such as image classification and speech processing. However, creating a deep learning model using health record data requires addressing certain privacy challenges that bring unique concerns to researchers working in this domain. One effective way to handle such private data issues is to generate realistic synthetic data that can provide practically acceptable data quality and correspondingly the model performance. To tackle this challenge, we develop a differentially private framework for synthetic da…
Helsinki Institute for Information Technology HIIT（赫尔辛基信息技术研究所（HIIT））；Aalto University（阿尔托大学）；University of Helsinki（赫尔辛基大学）
Motivation: Human genomic datasets often contain sensitive information that limits use and sharing of the data. In particular, simple anonymisation strategies fail to provide sufficient level of protection for genomic data, because the data are inherently identifiable. Differentially private machine learning can help by guaranteeing that the published results do not leak too much information about any individual data point. Recent research has reached promising results on differentially private drug sensitivity prediction using gene expression data. Differentially private learning with genomic…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Shanghai Jiao Tong University（上海交通大学）；Macquarie University（麦考瑞大学）；Data61-CSIRO（澳大利亚联邦科学与工业研究组织Data61研究所）
Deep neural networks (DNNs) have recently been widely adopted in various applications, and such success is largely due to a combination of algorithmic breakthroughs, computation resource improvements, and access to a large amount of data. However, the large-scale data collections required for deep learning often contain sensitive information, therefore raising many privacy concerns. Prior research has shown several successful attacks in inferring sensitive training data information, such as model inversion, membership inference, and generative adversarial networks (GAN) based leakage attacks a…
University of the Philippines Diliman（菲律宾大学迪利曼分校）；Asian Institute of Management（亚洲管理学院）
Single-pixel imaging (SPI) is a novel, unconventional method that goes beyond the notion of traditional cameras but can be computationally expensive and slow for real-time applications. Deep learning has been proposed as an alternative approach for solving the SPI reconstruction problem, but a detailed analysis of its performance and generated basis patterns when used for SPI is limited. We present a modified deep convolutional autoencoder network (DCAN) for SPI on 64x64 pixel images with up to 6.25% compression ratio and apply binary and orthogonality regularizers during training. Training a …
Rice University（莱斯大学）；Texas A&M University（得克萨斯农工大学）；J.P. Morgan AI Research（摩根大通人工智能研究院）
Counterfactual, serving as one emerging type of model explanation, has attracted tons of attentions recently from both industry and academia. Different from the conventional feature-based explanations (e.g., attributions), counterfactuals are a series of hypothetical samples which can flip model decisions with minimal perturbations on queries. Given valid counterfactuals, humans are capable of reasoning under ``what-if'' circumstances, so as to better understand the model decision boundaries. However, releasing counterfactuals could be detrimental, since it may unintentionally leak sensitive i…
Georgia Institute of Technology（佐治亚理工学院）；Stanford University（斯坦福大学）；Southern Methodist University（南卫理公会大学）；Amazon（亚马逊）
We introduce the DP-auto-GAN framework for synthetic data generation, which combines the low dimensional representation of autoencoders with the flexibility of Generative Adversarial Networks (GANs). This framework can be used to take in raw sensitive data and privately train a model for generating synthetic data that will satisfy similar statistical properties as the original data. This learned model can generate an arbitrary amount of synthetic data, which can then be freely shared due to the post-processing guarantee of differential privacy. Our framework is applicable to unlabeled mixed-ty…
CrySyS Lab, BME-HIT（CrySyS实验室，布达佩斯技术与经济大学-网络系统与服务系）；University College London（伦敦大学学院）；INRIA（法国国家信息与自动化研究所）
Generative models are used in a wide range of applications building on large amounts of contextually rich information. Due to possible privacy violations of the individuals whose data is used to train these models, however, publishing or sharing generative models is not always viable. In this paper, we present a novel technique for privately releasing generative models and entire high-dimensional datasets produced by these models. We model the generator distribution of the training data with a mixture of $k$ generative neural networks. These are trained together and collectively learn the gene…
KTH Royal Institute of Technology（瑞典皇家理工学院）；Elekta AB（医科达公司）
When it comes to preserving privacy in medical machine learning, two important considerations are (1) keeping data local to the institution and (2) avoiding inference of sensitive information from the trained model. These are often addressed using federated learning and differential privacy, respectively. However, the commonly used Federated Averaging algorithm requires a high degree of synchronization between participating institutions. For this reason, we turn our attention to Private Aggregation of Teacher Ensembles (PATE), where all local models can be trained independently without inter-i…
Differentially-Private Data Synthetisation for Efficient Re-Identification Risk Control
Faculty of Sciences, University of Porto（波尔图大学理学院）；Lucy Family Institute for Data & Society, University of Notre Dame（圣母大学露西数据与社会家族研究所）；TekPrivacy（TekPrivacy公司）
The Alan Turing Institute（阿兰·图灵研究所）；Vector Institute（向量研究所）；Université de Lille（里尔大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Centrale Lille（里尔中央理工学院）；Université de Lorraine（洛林大学）；University of Toronto（多伦多大学）
Sharing real-world speech utterances is key to the training and deployment of voice-based services. However, it also raises privacy risks as speech contains a wealth of personal data. Speaker anonymization aims to remove speaker information from a speech utterance while leaving its linguistic and prosodic attributes intact. State-of-the-art techniques operate by disentangling the speaker information (represented via a speaker embedding) from these attributes and re-synthesizing speech based on the speaker embedding of another speaker. Prior research in the privacy community has shown that anon…
We present an approach to quantify and compare the privacy-accuracy trade-off for differentially private Variational Autoencoders. Our work complements previous work in two aspects. First, we evaluate the the strong reconstruction MI attack against Variational Autoencoders under differential privacy. Second, we address the data scientist's challenge of setting privacy parameter epsilon, which steers the differential privacy strength and thus also the privacy-accuracy trade-off. In our experimental study we consider image and time series data, and three local and central differential privacy me…
Modern machine learning systems achieve great success when trained on large datasets. However, these datasets usually contain sensitive information (e.g. medical records, face images), leading to serious privacy concerns. Differentially private generative models (DPGMs) emerge as a solution to circumvent such privacy concerns by generating privatized sensitive data. Similar to other differentially private (DP) learners, the major challenge for DPGM is also how to achieve a subtle balance between utility and privacy. We propose DP$^2$-VAE, a novel training mechanism for variational autoencoders…
LINE Corporation（LINE公司）；Kyoto University（京都大学）；SOKENDAI（综合研究大学院大学）
This paper studies how to learn variational autoencoders with a variety of divergences under differential privacy constraints. We often build a VAE with an appropriate prior distribution to describe the desired properties of the learned representations and introduce a divergence as a regularization term to close the representations to the prior. Using differentially private SGD (DP-SGD), which randomizes a stochastic gradient by injecting a dedicated noise designed according to the gradient's sensitivity, we can easily build a differentially private model. However, we reveal that attaching sev…
DP-TLDM: Differentially Private Tabular Latent Diffusion Model
TU Delft, Delft, Netherlands , University of Neuchâtel, Neuchâtel, Switzerland Universidad de los Andes, Bogotá, Colombia Centrum Wiskunde \& Informatica, Amsterdam, Netherlands
University of British Columbia（不列颠哥伦比亚大学）；University of Tübingen（蒂宾根大学）；International Max Planck Research School for Intelligent Systems (IMPRS-IS)（国际马克斯·普朗克智能系统研究院）；Technical University of Denmark（丹麦技术大学）
Privacy-Preserving Federated Autoencoder for ECG Anomaly Detection on Edge Devices
Google Research, New York, NY（谷歌研究，纽约，纽约州）；University of California, Berkeley（加州大学伯克利分校）；University of Cambridge（剑桥大学）；University of Toronto（多伦多大学）；University of Melbourne（墨尔本大学）；University of Sydney（悉尼大学）
Learning Privacy-Preserving Student Networks via Discriminative-Generative Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）
