In the digital era, users share their personal data with service providers to obtain some utility, e.g., access to high-quality services. Yet, the induced information flows raise privacy and integrity concerns. Consequently, cautious users may want to protect their privacy by minimizing the amount of information they disclose to curious service providers. Service providers are interested in verifying the integrity of the users' data to improve their services and obtain useful knowledge for their business. In this work, we present a generic solution to the trade-off between privacy, integrity, and utility, by achieving authenticity verification of data that has been encrypted for offloading to service providers. Based on lattice-based homomorphic encryption and commitments, as well as zero-knowledge proofs, our construction enables a service provider to process and reuse third-party signed data in a privacy-friendly manner with integrity guarantees. We evaluate our solution on different use cases such as smart-metering, disease susceptibility, and location-based activity tracking, thus showing its versatility. Our solution achieves broad generality, quantum-resistance, and relaxes some assumptions of state-of-the-art solutions without affecting performance.
Privacy by design will become a legal obligation in the European Community if the Data Protection Regulation eventually gets adopted. However, taking into account privacy requirements in the design of a system is a challenging task. We propose an approach based on the specification of privacy architectures and focus on a key aspect of privacy, data minimisation, and its tension with integrity requirements. We illustrate our formal framework through a smart metering case study.
With the wide application of cloud storage, cloud security has become a crucial concern. Related works have addressed security issues such as data confidentiality and integrity, which ensure that the remotely stored data are well maintained by the cloud. However, how to define zero-knowledge proof algorithms for stored data integrity check has not been formally defined and investigated. We believe that it is important that the cloud server is unable to reveal any useful information about the stored data. In this paper, we introduce a novel definition of data privacy for integrity checks, which…
In Advanced Metering Infrastructure (AMI) networks, smart meters should send fine-grained power consumption readings to electric utilities to perform real-time monitoring and energy management. However, these readings can leak sensitive information about consumers' activities. Various privacy-preserving schemes for collecting fine-grained readings have been proposed for AMI networks. These schemes aggregate individual readings and send an aggregated reading to the utility, but they extensively use asymmetric-key cryptography which involves large computation/communication overhead. Furthermore,…
Federated learning (FL) enables clients to collaborate with a server to train a machine learning model. To ensure privacy, the server performs secure aggregation of updates from the clients. Unfortunately, this prevents verification of the well-formedness (integrity) of the updates as the updates are masked. Consequently, malformed updates designed to poison the model can be injected without detection. In this paper, we formalize the problem of ensuring \textit{both} update privacy and integrity in FL and present a new system, \textsf{EIFFeL}, that enables secure aggregation of \textit{verifie…
With the rapid demand of data and computational resources in deep learning systems, a growing number of algorithms to utilize collaborative machine learning techniques, for example, federated learning, to train a shared deep model across multiple participants. It could effectively take advantage of the resources of each participant and obtain a more powerful learning system. However, integrity and privacy threats in such systems have greatly obstructed the applications of collaborative learning. And a large amount of works have been proposed to maintain the model integrity and mitigate the pri…
EIFL: Efficiently Protecting Global Model Privacy and Integrity Against an Untrusted Server in Federated Learning
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Department of Computer Science, Illinois Institute of Technology（伊利诺伊理工学院计算机科学系）
We introduce a formal model for membership and order queries on privacy-preserving authenticated lists. In this model, the queries are performed on the list stored in the cloud where data integrity and privacy have to be maintained. We then present an efficient construction of privacy-preserving authenticated lists based on bilinear accumulators and bilinear maps, analyze the performance, and prove the integrity and privacy of this construction under widely accepted assumptions.
Online advertising fuels the (seemingly) free internet. However, although users can access most of the web services free of charge, they pay a heavy coston their privacy. They are forced to trust third parties and intermediaries, who not only collect behavioral data but also absorb great amounts of ad revenues. Consequently, more and more users opt out from advertising by resorting to ad blockers, thus costing publishers millions of dollars in lost ad revenues. Albeit there are various privacy-preserving advertising proposals (e.g.,Adnostic, Privad, Brave Ads) from both academia and industry, …
Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. However, Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. This work presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relies on cooperative execution between trusted execution environments (TEE) and accelerators, where the TEE provides …
Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train or infer with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. Tackling such a challenge requires unifying theoretical privacy algorithms with hardware security capabilities. This paper presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relie…
An increasing amount of sensitive information today is stored electronically and a substantial part of this information (e.g., health records, tax data, legal documents) must be retained over long time periods (e.g., several decades or even centuries). When sensitive data is stored, then integrity and confidentiality must be protected to ensure reliability and privacy. Commonly used cryptographic schemes, however, are not designed for protecting data over such long time periods. Recently, the first storage architecture combining long-term integrity with long-term confidentiality protection was…
Conventional Data Leakage Prevention (DLP) systems suffer from the following major drawback: Privacy policies that define what constitutes data leakage cannot be seamlessly defined and enforced across heterogeneous forms of communication. Administrators have the dual burden of: (1) manually self-interpreting policies from handbooks to specify rules (which is error-prone); (2) extracting relevant information flows from heterogeneous communication protocols and enforcing policies to determine which flows should be admissible. To address these issues, we present the Verifiable and ACtionable Cont…
Online advertising fuels the (seemingly) free internet. However, although users can access most websites free of charge, they need to pay a heavy cost on their privacy and blindly trust third parties and intermediaries that absorb great amounts of adrevenues and user data. This is one of the reasons users opt out from advertising by resorting ad blockers thatin turn cost publishers millions of dollars in lost adrevenues. Existing privacy-preserving advertising approaches(e.g., Adnostic, Privad, Brave Ads) from both industry and academia cannot guarantee the integrity of the performance analyti…
Membership Inference Attacks fueled by Few-Short Learning to detect privacy leakage tackling data integrity
University of Granada（格拉纳达大学）；ADIA Lab（ADIA实验室）；Andalusian Research Institute in Data Science and Computational Intelligence (DaSCI)（安达卢西亚数据科学与计算智能研究所（DaSCI））
University of Bayreuth（拜罗伊特大学）；FIM Research Center（FIM研究中心）；Fraunhofer FIT（弗劳恩霍夫应用信息技术研究所）
Federated machine learning (FL) allows to collectively train models on sensitive data as only the clients' models and not their training data need to be shared. However, despite the attention that research on FL has drawn, the concept still lacks broad adoption in practice. One of the key reasons is the great challenge to implement FL systems that simultaneously achieve fairness, integrity, and privacy preservation for all participating clients. To contribute to solving this issue, our paper suggests a FL system that incorporates blockchain technology, local differential privacy, and zero-know…
Smartphone-based contact-tracing applications are at the epicenter of the global fight against the Covid-19 pandemic. While governments and healthcare agencies are eager to mandate the deployment of such applications en-masse, they face increasing scrutiny from the popular press, security companies, and human rights watch agencies that fear the exploitation of these technologies as surveillance tools. Finding the optimal balance between community safety and privacy has been a challenge, and strategies to address these concerns have varied among countries. This paper describes two important att…
Differential privacy (DP) has been widely used to protect the privacy of confidential cyber physical energy systems (CPES) data. However, applying DP without analyzing the utility, privacy, and security requirements can affect the data utility as well as help the attacker to conduct integrity attacks (e.g., False Data Injection(FDI)) leveraging the differentially private data. Existing anomaly-detection-based defense strategies against data integrity attacks in DP-based smart grids fail to minimize the attack impact while maximizing data privacy and utility. To address this challenge, it is no…
Recent privacy awareness initiatives such as the EU General Data Protection Regulation subdued Machine Learning (ML) to privacy and security assessments. Federated Learning (FL) grants a privacy-driven, decentralized training scheme that improves ML models' security. The industry's fast-growing adaptation and security evaluations of FL technology exposed various vulnerabilities that threaten FL's confidentiality, integrity, or availability (CIA). This work assesses the CIA of FL by reviewing the state-of-the-art (SoTA) and creating a threat model that embraces the attack's surface, adversarial…
Widely used payment splitting apps allow members of a group to keep track of debts between members by sending charges for expenses paid by one member on behalf of others. While offering a great deal of convenience, these apps gain access to sensitive data on users' financial transactions. In this paper, we present a payment splitting app that hides all transaction data within a group from the service provider, provides privacy protections between users in a group, and provides integrity against malicious users or even a malicious server. The core protocol proceeds in a series of rounds in whic…
Recently, Government of India has taken several initiatives to make India digitally strong such as to provide each resident a unique digital identity, referred to as Aadhaar, and to provide several online e-Governance services based on Aadhaar such as DigiLocker. DigiLocker is an online service which provides a shareable private storage space on public cloud to its subscribers. Although DigiLocker ensures traditional security such as data integrity and secure data access, privacy of e-documents are yet to addressed. Ciphertext-Policy Attribute-Based Encryption (CP-ABE) can improve data privacy…
Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；Carnegie Mellon University（卡内基梅隆大学）；National University of Singapore（新加坡国立大学）
Data tampering is often considered a severe problem in industrial applications as it can lead to inaccurate financial reports or even a corporate security crisis. A correct representation of data is essential for companies' core business processes and is demanded by investors and customers. Traditional data audits are performed through third-party auditing services; however, these services are expensive and can be untrustworthy in some cases. Blockchain and smart contracts provide a decentralized mechanism to achieve secure and trustworthy data integrity verification; however, existing solutio…
Users today expect more security from services that handle their data. In addition to traditional data privacy and integrity requirements, they expect transparency, i.e., that the service's processing of the data is verifiable by users and trusted auditors. Our goal is to build a multi-user system that provides data privacy, integrity, and transparency for a large number of operations, while achieving practical performance. To this end, we first identify the limitations of existing approaches that use authenticated data structures. We find that they fall into two categories: 1) those that hide…
We present an empirical study exploring how privacy influences the acceptance of vaccination certificate (VC) deployments across different realistic usage scenarios. The study employed the privacy framework of Contextual Integrity, which has been shown to be particularly effective in capturing people's privacy expectations across different contexts. We use a vignette methodology, where we selectively manipulate salient contextual parameters to learn whether and how they affect people's attitudes towards VCs. We surveyed 890 participants from a demographically-stratified sample of the US popula…
PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems
ServiceNow AI Research（ServiceNow 人工智能研究院）；Mila – Quebec AI Institute（Mila – 魁北克人工智能研究所）；Université Laval（拉瓦尔大学）；McGill University（麦吉尔大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
With the rapidly evolving next-generation systems-of-systems, we face new security, resilience, and operational assurance challenges. In the face of the increasing attack landscape, it is necessary to cater to efficient mechanisms to verify software and device integrity to detect run-time modifications. Towards this direction, remote attestation is a promising defense mechanism that allows a third party, the verifier, to ensure a remote device's (the prover's) integrity. However, many of the existing families of attestation solutions have strong assumptions on the verifying entity's trustworth…
Data security and privacy is an important but challenging problem in cloud computing. One of the security concerns from cloud users is how to efficiently verify the integrity of their data stored on the cloud server. Third Party Auditing (TPA) is a new technique proposed in recent years to achieve this goal. In a recent paper (IEEE Transactions on Computers 62(2): 362-375 (2013)), Wang et al. proposed a highly efficient and scalable TPA protocol and also a Zero Knowledge Public Auditing protocol which can prevent offline guessing attacks. However, in this paper, we point out several security w…
This thesis proposes techniques aiming to make blockchain technologies and smart contract platforms practical by improving their scalability, latency, and privacy. This thesis starts by presenting the design and implementation of Chainspace, a distributed ledger that supports user defined smart contracts and execute user-supplied transactions on their objects. The correct execution of smart contract transactions is publicly verifiable. Chainspace is scalable by sharding state; it is secure against subsets of nodes trying to compromise its integrity or availability properties through Byzantine …
Protecting the privacy of input data is of growing importance as machine learning methods reach new application domains. In this paper, we provide a unified training and inference framework for large DNNs while protecting input privacy and computation integrity. Our approach called DarKnight uses a novel data blinding strategy using matrix masking to create input obfuscation within a trusted execution environment (TEE). Our rigorous mathematical proof demonstrates that our blinding process provides information-theoretic privacy guarantee by bounding information leakage. The obfuscated data can…
This paper introduces SPOT, a Secure and Privacy-preserving prOximity based protocol for e-healthcare systems. It relies on a distributed proxy-based approach to preserve users' privacy and a semi-trusted computing server to ensure data consistency and integrity. The proposed protocol ensures a balance between security, privacy and scalability. As far as we know, in terms of security, SPOT is the first one to prevent malicious users from colluding and generating false positives. In terms of privacy, SPOT supports both anonymity of users being in proximity of infected people and unlinkability o…
Cloud storage services have become accessible and used by everyone. Nevertheless, stored data are dependable on the behavior of the cloud servers, and losses and damages often occur. One solution is to regularly audit the cloud servers in order to check the integrity of the stored data. The Dynamic Provable Data Possession scheme with Public Verifiability and Data Privacy presented in ACISP'15 is a straightforward design of such solution. However, this scheme is threatened by several attacks. In this paper, we carefully recall the definition of this scheme as well as explain how its security i…
