Privacy and data protection have become more and more important in recent years since an increasing number of enterprises and startups are harvesting personal data as a part of their business model. One central requirement of the GDPR is the implementation of a data protection impact assessment for privacy critical systems. However, the law does not dictate or recommend the use of any particular framework. In this paper we compare different data protection impact assessment frameworks. We have developed a comparison and evaluation methodology and applied this to three popular impact assessment frameworks. The result of this comparison shows the weaknesses and strengths, but also clearly indicates that none of the tested frameworks fulfill all desired properties. Thus, the development of a new or improved data protection impact assessment framework is an important open issue for future work, especially for sector specific applications.
Since its introduction in 2006, differential privacy has emerged as a predominant statistical tool for quantifying data privacy in academic works. Yet despite the plethora of research and open-source utilities that have accompanied its rise, with limited exceptions, differential privacy has failed to achieve widespread adoption in the enterprise domain. Our study aims to shed light on the fundamental causes underlying this academic-industrial utilization gap through detailed interviews of 24 privacy practitioners across 9 major companies. We analyze the results of our survey to provide key fin…
The Austrian eID system constitutes a main pillar within the Austrian e-Government strategy. The eID system ensures unique identification and secure authentication for citizens protecting access to applications where sensitive and personal data is involved. In particular, the Austrian eID system supports three main use cases: Identification and authentication of Austrian citizens, electronic representation, and foreign citizen authentication at Austrian public sector applications. For supporting all these use cases, several components -- either locally deployed in the applications' domain or c…
Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；Macquarie University（麦考瑞大学）；University of Technology Sydney（悉尼科技大学）；Nanyang Technological University（南洋理工大学）；Tencent Technology (Shenzhen) Co. Ltd（腾讯科技（深圳）有限公司）
In spite that Federated Learning (FL) is well known for its privacy protection when training machine learning models among distributed clients collaboratively, recent studies have pointed out that the naive FL is susceptible to gradient leakage attacks. In the meanwhile, Differential Privacy (DP) emerges as a promising countermeasure to defend against gradient leakage attacks. However, the adoption of DP by clients in FL may significantly jeopardize the model accuracy. It is still an open problem to understand the practicality of DP from a theoretic perspective. In this paper, we make the firs…
Existing work on privacy by design mostly focus on technologies rather than methodologies and on components rather than architectures. In this paper, we advocate the idea that privacy by design should also be addressed at the architectural level and be associated with suitable methodologies. Among other benefits, architectural descriptions enable a more systematic exploration of the design space. In addition, because privacy is intrinsically a complex notion that can be in tension with other requirements, we believe that formal methods should play a key role in this area. After presenting our …
Software applications continue to challenge user privacy when users interact with them. Privacy practices (e.g. Data Minimisation (DM), Privacy by Design (PbD) or General Data Protection Regulation (GDPR)) and related "privacy engineering" methodologies exist and provide clear instructions for developers to implement privacy into software systems they develop that preserve user privacy. However, those practices and methodologies are not yet a common practice in the software development community. There has been no previous research focused on developing "educational" interventions such as seri…
Brain-Computer Interfaces (BCIs) comprise a rapidly evolving field of technology with the potential of far-reaching impact in domains ranging from medical over industrial to artistic, gaming, and military. Today, these emerging BCI applications are typically still at early technology readiness levels, but because BCIs create novel, technical communication channels for the human brain, they have raised privacy and security concerns. To mitigate such risks, a large body of countermeasures has been proposed in the literature, but a general framework is lacking which would describe how privacy and…
Case studies of application software data models indicate that timestamps are excessively used in connection with user activity. This contradicts the principle of data minimisation which demands a limitation to data necessary for a given purpose. Prior work has also identified common purposes of timestamps that can be realised by more privacy-preserving alternatives like counters and dates with purpose-oriented precision. In this paper, we follow up by demonstrating the real-world applicability of those alternatives. We design and implement three timestamp alternatives for the popular web deve…
Privacy is a well-understood concept in the physical world, with us all desiring some escape from the public gaze. However, while individuals might recognise locking doors as protecting privacy, they have difficulty practising equivalent actions online. Privacy salience considers the tangibility of this important principle; one which is often obscured in digital environments. Through extensively surveying a range of studies, we construct the first taxonomies of privacy salience. After coding articles and identifying commonalities, we categorise works by their methodologies, platforms and under…
As an effective resource allocation approach, double auctions (DAs) have been extensively studied in electronic commerce. Most previous studies have focused on how to design strategy-proof DA mechanisms, while not much research effort has been done concerning privacy and security issues. However, security, especially privacy issues have become such a public concern that the European governments lay down the law to enforce the privacy guarantees recently. In this paper, to address the privacy issue in electronic auctions, we concentrate on how to design a privacy-preserving mechanism for double…
An increasing number of individuals are turning to Direct-To-Consumer (DTC) genetic testing to learn about their predisposition to diseases, traits, and/or ancestry. DTC companies like 23andme and Ancestry.com have started to offer popular and affordable ancestry and genealogy tests, with services allowing users to find unknown relatives and long-distant cousins. Naturally, access and possible dissemination of genetic data prompts serious privacy concerns, thus motivating the need to design efficient primitives supporting private genetic tests. In this paper, we present an effective protocol f…
UC San Diego（加州大学圣地亚哥分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
ldp deployments are vulnerable to inference attacks as an adversary can link the noisy responses to their identity and subsequently, auxiliary information using the order of the data. An alternative model, shuffle DP, prevents this by shuffling the noisy responses uniformly at random. However, this limits the data learnability -- only symmetric functions (input order agnostic) can be learned. In this paper, we strike a balance and show that systematic shuffling of the noisy responses can thwart specific inference attacks while retaining some meaningful data learnability. To this end, we propos…
The increasingly rapid use of mobile devices for data transaction around the world has consequently led to a new problem, and that is, how to engage in mobile data transactions while maintaining an acceptable level of data privacy and security. While most mobile devices engage in data transactions through a data cloud or a set of data servers, it is still possible to apply data confidentiality across data servers, and, as such, preserving privacy in any mobile data transaction. Yet still, it is essential that a review of data privacy, data utility, the techniques, and methodologies employed in…
UC San Diego（加州大学圣地亚哥分校）；University of Amsterdam（阿姆斯特丹大学）
Bayesian inference has great promise for the privacy-preserving analysis of sensitive data, as posterior sampling automatically preserves differential privacy, an algorithmic notion of data privacy, under certain conditions (Dimitrakakis et al., 2014; Wang et al., 2015). While this one posterior sample (OPS) approach elegantly provides privacy "for free," it is data inefficient in the sense of asymptotic relative efficiency (ARE). We show that a simple alternative based on the Laplace mechanism, the workhorse of differential privacy, is as asymptotically efficient as non-private posterior infe…
We study the problem of providing privacy-preserving access to an outsourced honest-but-curious data repository for a group of trusted users. We show that such privacy-preserving data access is possible using a combination of probabilistic encryption, which directly hides data values, and stateless oblivious RAM simulation, which hides the pattern of data accesses. We give simulations that have only an $O(\log n)$ amortized time overhead for simulating a RAM algorithm, $\cal A$, that has a memory of size $n$, using a scheme that is data-oblivious with very high probability assuming the simulat…
With increasing usage of deep learning algorithms in many application, new research questions related to privacy and adversarial attacks are emerging. However, the deep learning algorithm improvement needs more and more data to be shared within research community. Methodologies like federated learning, differential privacy, additive secret sharing provides a way to train machine learning models on edge without moving the data from the edge. However, it is very computationally intensive and prone to adversarial attacks. Therefore, this work introduces a privacy preserving FedCollabNN framework …
Blockchain technologies have been boosting the development of data-driven decentralized services in a wide range of fields. However, with the spirit of full transparency, many public blockchains expose all types of data to the public such as Ethereum. Besides, the on-chain persistence of large data is significantly expensive technically and economically. These issues lead to the difficulty of sharing fairly large private data while preserving attractive properties of public blockchains. Although direct encryption for on-chain data persistence can introduce confidentiality, new challenges such …
Past work has shown that large language models are susceptible to privacy attacks, where adversaries generate sequences from a trained model and detect which sequences are memorized from the training set. In this work, we show that the success of these attacks is largely due to duplication in commonly used web-scraped training sets. We first show that the rate at which language models regenerate training sequences is superlinearly related to a sequence's count in the training set. For instance, a sequence that is present 10 times in the training data is on average generated ~1000 times more of…
Utah State University（犹他州立大学）；University of Wisconsin-Whitewater（威斯康星大学白水分校）；Idaho National Laboratory（爱达荷国家实验室）
The exponential growth of internet connected systems has generated numerous challenges, such as spectrum shortage issues, which require efficient spectrum sharing (SS) solutions. Complicated and dynamic SS systems can be exposed to different potential security and privacy issues, requiring protection mechanisms to be adaptive, reliable, and scalable. Machine learning (ML) based methods have frequently been proposed to address those issues. In this article, we provide a comprehensive survey of the recent development of ML based SS methods, the most critical security issues, and corresponding de…
In the big data era, more and more cloud-based data-driven applications are developed that leverage individual data to provide certain valuable services (the utilities). On the other hand, since the same set of individual data could be utilized to infer the individual's certain sensitive information, it creates new channels to snoop the individual's privacy. Hence it is of great importance to develop techniques that enable the data owners to release privatized data, that can still be utilized for certain premised intended purpose. Existing data releasing approaches, however, are either privacy…
Differential Privacy (DP) provides strong guarantees on the risk of compromising a user's data in statistical learning applications, though these strong protections make learning challenging and may be too stringent for some use cases. To address this, we propose element level differential privacy, which extends differential privacy to provide protection against leaking information about any particular "element" a user has, allowing better utility and more robust results than classical DP. By carefully choosing these "elements," it is possible to provide privacy protections at a desired granul…
Have it your way: Individualized Privacy Assignment for DP-SGD
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Free University Berlin（柏林自由大学）；Columbia University（哥伦比亚大学）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
An increasing number of businesses are replacing their data storage and computation infrastructure with cloud services. Likewise, there is an increased emphasis on performing analytics based on multiple datasets obtained from different data sources. While ensuring security of data and computation outsourced to a third party cloud is in itself challenging, supporting analytics using data distributed across multiple, independent clouds is even further from trivial. In this paper we present CloudMine, a cloud-based service which allows multiple data owners to perform privacy-preserved computation…
As organizations struggle with processing vast amounts of information, outsourcing sensitive data to third parties becomes a necessity. To protect the data, various cryptographic techniques are used in outsourced database systems to ensure data privacy, while allowing efficient querying. A rich collection of attacks on such systems has emerged. Even with strong cryptography, just communication volume or access pattern is enough for an adversary to succeed. In this work we present a model for differentially private outsourced database system and a concrete construction, $\mathcal{E}\text{psolut…
Thanks to rapid technological developments, new innovative solutions and practical applications of the Industrial Internet of Things (IIoT) are being created, upgrading the structures of many industrial enterprises. IIoT brings the physical and digital environment together with minimal human intervention and profoundly transforms the economy and modern business. Data flowing through IIoT feed artificial intelligence tools, which perform intelligent functions such as performance tuning of interconnected machines, error correction, and preventive maintenance. However, IIoT deployments are vulner…
Federated learning (FL) is a technique that trains machine learning models from decentralized data sources. We study FL under local notions of privacy constraints, which provides strong protection against sensitive data disclosures via obfuscating the data before leaving the client. We identify two major concerns in designing practical privacy-preserving FL algorithms: communication efficiency and high-dimensional compatibility. We then develop a gradient-based learning algorithm called \emph{sqSGD} (selective quantized stochastic gradient descent) that addresses both concerns. The proposed al…
In this paper, we consider methods for performing hypothesis tests on data protected by a statistical disclosure control technology known as differential privacy. Previous approaches to differentially private hypothesis testing either perturbed the test statistic with random noise having large variance (and resulted in a significant loss of power) or added smaller amounts of noise directly to the data but failed to adjust the test in response to the added noise (resulting in biased, unreliable $p$-values). In this paper, we develop a variety of practical hypothesis tests that address these pro…
Recent studies show that 20.4% of the internet traffic originates from automated agents. To identify and block such ill-intentioned traffic, mechanisms that verify the humanness of the user are widely deployed, with CAPTCHAs being the most popular. Traditional CAPTCHAs require extra user effort (e.g., solving mathematical puzzles), which can severely downgrade the end-user's experience, especially on mobile, and provide sporadic humanness verification of questionable accuracy. More recent solutions like Google's reCAPTCHA v3, leverage user data, thus raising significant privacy concerns. To ad…
This paper tackles the problem of ensuring training data privacy in a federated learning context. Relying on Homomorphic Encryption (HE) and Differential Privacy (DP), we propose a framework addressing threats on the privacy of the training data. Notably, the proposed framework ensures the privacy of the training data from all actors of the learning process, namely the data owners and the aggregating server. More precisely, while HE blinds a semi-honest server during the learning protocol, DP protects the data from semi-honest clients participating in the training process as well as end-users …
The advent of miniature biosensors has generated numerous opportunities for deploying wireless sensor networks in healthcare. However, an important barrier is that acceptance by healthcare stakeholders is influenced by the effectiveness of privacy safeguards for personal and intimate information which is collected and transmitted over the air, within and beyond these networks. In particular, these networks are progressing beyond traditional sensors, towards also using multimedia sensors, which raise further privacy concerns. Paradoxically, less research has addressed privacy protection, compar…
In the modern era of computing, machine learning tools have demonstrated their potential in vital sectors, such as healthcare and finance, to derive proper inferences. The sensitive and confidential nature of the data in such sectors raises genuine concerns for data privacy. This motivated the area of Privacy-preserving Machine Learning (PPML), where privacy of data is guaranteed. In this thesis, we design an efficient platform, MPCLeague, for PPML in the Secure Outsourced Computation (SOC) setting using Secure Multi-party Computation (MPC) techniques. MPC, the holy-grail problem of secure dis…
University of Technology Sydney（悉尼科技大学）；City University of Macau（澳门城市大学）
Reinforcement learning (RL) is one of the most important branches of AI. Due to its capacity for self-adaption and decision-making in dynamic environments, reinforcement learning has been widely applied in multiple areas, such as healthcare, data markets, autonomous driving, and robotics. However, some of these applications and systems have been shown to be vulnerable to security or privacy attacks, resulting in unreliable or unstable services. A large number of studies have focused on these security and privacy problems in reinforcement learning. However, few surveys have provided a systemati…
Differential Privacy-Driven Framework for Enhancing Heart Disease Prediction
School of Computer Science and Technology（计算机科学与技术学院）；Algoma University（阿尔戈马大学）；School of Electrical Engineering and Computer Science（电气工程与计算机科学学院）；University of Ottawa（渥太华大学）
