When training a machine learning model, it is standard procedure for the researcher to have full knowledge of both the data and model. However, this engenders a lack of trust between data owners and data scientists. Data owners are justifiably reluctant to relinquish control of private information to third parties. Privacy-preserving techniques distribute computation in order to ensure that data remains in the control of the owner while learning takes place. However, architectures distributed amongst multiple agents introduce an entirely new set of security and trust complications. These include data poisoning and model theft. This paper outlines a distributed infrastructure which is used to facilitate peer-to-peer trust between distributed agents; collaboratively performing a privacy-preserving workflow. Our outlined prototype sets industry gatekeepers and governance bodies as credential issuers. Before participating in the distributed learning workflow, malicious actors must first negotiate valid credentials. We detail a proof of concept using Hyperledger Aries, Decentralised Identifiers (DIDs) and Verifiable Credentials (VCs) to establish a distributed trust architecture during a privacy-preserving machine learning experiment. Specifically, we utilise secure and authenticated DID communication channels in order to facilitate a federated learning workflow related to mental health care data.
A common privacy issue in traditional machine learning is that data needs to be disclosed for the training procedures. In situations with highly sensitive data such as healthcare records, accessing this information is challenging and often prohibited. Luckily, privacy-preserving technologies have been developed to overcome this hurdle by distributing the computation of the training and ensuring the data privacy to their owners. The distribution of the computation to multiple participating entities introduces new privacy complications and risks. In this paper, we present a privacy-preserving de…
Distributed multi-agent learning enables agents to cooperatively train a model without requiring to share their datasets. While this setting ensures some level of privacy, it has been shown that, even when data is not directly shared, the training process is vulnerable to privacy attacks including data reconstruction and model inversion attacks. Additionally, malicious agents that train on inverted labels or random data, may arbitrarily weaken the accuracy of the global model. This paper addresses these challenges and presents Privacy-preserving and trustable Distributed Learning (PT-DL), a fu…
The Chinese University of Hong Kong（香港中文大学）；University of Electronic Science and Technology of China（电子科技大学）；Peng Cheng Lab（鹏城实验室）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
Trustworthy artificial intelligence (AI) technology has revolutionized daily life and greatly benefited human society. Among various AI technologies, Federated Learning (FL) stands out as a promising solution for diverse real-world scenarios, ranging from risk evaluation systems in finance to cutting-edge technologies like drug discovery in life sciences. However, challenges around data isolation and privacy threaten the trustworthiness of FL systems. Adversarial attacks against data privacy, learning algorithm stability, and system confidentiality are particularly concerning in the context of…
Imperial College London（帝国理工学院）；Telefónica Research（西班牙电信研究院）
We propose and implement a Privacy-preserving Federated Learning ($PPFL$) framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluatio…
ZTFed-MAS2S: A Zero-Trust Federated Learning Framework with Verifiable Privacy and Trust-Aware Aggregation for Wind Power Data Imputation
School of Electrical Engineering, Northeast Electric Power University（电气工程学院，东北电力大学）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学）；State Grid Xiamen Electric Power Supply Company（国家电网厦门供电有限公司）；Department of Electrical Engineering, City University of Hong Kong（电气工程系，香港城市大学）
TrustFed: Enabling Trustworthy Medical AI under Data Privacy Constraints
Department of Applied Mechanics（应用力学系）；Indian Institute of Technology Delhi（印度理工学院德里）；Grainger College of Engineering（工程学院）；Nuclear, Plasma & Radiological Engineering Department（核物理与辐射工程系）；National Center for Supercomputing Applications（国家超级计算应用中心）；Yardi School of Artificial Intelligence (ScAI)（Yardi人工智能学院（ScAI））；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Federated learning (FL) has recently become a hot research topic, in which Byzantine robustness, communication efficiency and privacy preservation are three important aspects. However, the tension among these three aspects makes it hard to simultaneously take all of them into account. In view of this challenge, we theoretically analyze the conditions that a communication compression method should satisfy to be compatible with existing Byzantine-robust methods and privacy-preserving methods. Motivated by the analysis results, we propose a novel communication compression method called consensus …
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Mathematics and Computer Science, Fuzhou University（福州大学数学与计算机科学学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）
Mobile crowdsensing (MCS) is an emerging sensing data collection pattern with scalability, low deployment cost, and distributed characteristics. Traditional MCS systems suffer from privacy concerns and fair reward distribution. Moreover, existing privacy-preserving MCS solutions usually focus on the privacy protection of data collection rather than that of data processing. To tackle faced problems of MCS, in this paper, we integrate federated learning (FL) into MCS and propose a privacy-preserving MCS system, called \textsc{CrowdFL}. Specifically, in order to protect privacy, participants loca…
Imperial College London（帝国理工学院）；DeepMind（深度思维）；University of Oxford（牛津大学）；Case Western Reserve University（凯斯西储大学）
We detail a new framework for privacy preserving deep learning and discuss its assets. The framework puts a premium on ownership and secure processing of data and introduces a valuable representation based on chains of commands and tensors. This abstraction allows one to implement complex privacy preserving constructs such as Federated Learning, Secure Multiparty Computation, and Differential Privacy while still exposing a familiar deep learning API to the end-user. We report early results on the Boston Housing and Pima Indian Diabetes datasets. While the privacy features apart from Differenti…
In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but to also protect the privacy of test (query) images for the first time. In addition, it can also maintain the same accuracy as normally trained models. In an experiment, the proposed method was demonstr…
FINATRAX, University of Luxembourg（卢森堡大学FINATRAX）；University of Luxembourg（卢森堡大学）
With high levels of intermittent power generation and dynamic demand patterns, accurate forecasts for residential loads have become essential. Smart meters can play an important role when making these forecasts as they provide detailed load data. However, using smart meter data for load forecasting is challenging due to data privacy requirements. This paper investigates how these requirements can be addressed through a combination of federated learning and privacy preserving techniques such as differential privacy and secure aggregation. For our analysis, we employ a large set of residential l…
The privacy risks of machine learning models is a major concern when training them on sensitive and personal data. We discuss the tradeoffs between data privacy and the remaining goals of trustworthy machine learning (notably, fairness, robustness, and explainability).
Many successful services rely on trustworthy contributions from users. To establish that trust, such services often require access to privacy-sensitive information from users, thus creating a conflict between privacy and trust. Although it is likely impractical to expect both absolute privacy and trustworthiness at the same time, we argue that the current state of things, where individual privacy is usually sacrificed at the altar of trustworthy services, can be improved with a pragmatic $Glimmer$ $of$ $Trust$, which allows services to validate user contributions in a trustworthy way without f…
Health App Reviews for Privacy & Trust (HARPT): A Corpus for Analyzing Patient Privacy Concerns, Trust in Providers and Trust in Applications
Institute for Data Science & Informatics, University of Missouri（密苏里大学数据科学与信息学研究所）；Dept. Electrical Engineering & Computer Science, University of Missouri（密苏里大学电气工程与计算机科学系）；Dept. of Computer Science, Yale University（耶鲁大学计算机科学系）
A smart grid is an advanced method for supplying electricity to the consumers alleviating the limitations of the existing system. It causes frequent meter reading transmission from the end-user to the supplier. This frequent data transmission poses privacy risks. Several works have been proposed to solve this problem but cannot ensure privacy at the optimal level. This work is based on a distributed trust-based data aggregation system leveraging a secret sharing mechanism. In this work, we show that {\em three aggregators} are enough for ensuring consumer's privacy in a distributed trust-based…
Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. However, Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. This work presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relies on cooperative execution between trusted execution environments (TEE) and accelerators, where the TEE provides …
Boston University（波士顿大学）；Wayne State University（韦恩州立大学）
We consider the standard $K$-armed bandit problem under a distributed trust model of differential privacy (DP), which enables to guarantee privacy without a trustworthy server. Under this trust model, previous work largely focus on achieving privacy using a shuffle protocol, where a batch of users data are randomly permuted before sending to a central server. This protocol achieves ($ε,δ$) or approximate-DP guarantee by sacrificing an additional additive $O\!\left(\!\frac{K\log T\sqrt{\log(1/δ)}}ε\!\right)\!$ cost in $T$-step cumulative regret. In contrast, the optimal privacy cost for achievi…
A growing framework of legal and ethical requirements limit scientific and commercial evalua-tion of personal data. Typically, pseudonymization, encryption, or methods of distributed com-puting try to protect individual privacy. However, computational infrastructures still depend on human system administrators. This introduces severe security risks and has strong impact on privacy: system administrators have unlimited access to the computers that they manage in-cluding encryption keys and pseudonymization-tables. Distributed computing and data obfuscation technologies reduce but do not elimina…
The use of trusted hardware has become a promising solution to enable privacy-preserving machine learning. In particular, users can upload their private data and models to a hardware-enforced trusted execution environment (e.g. an enclave in Intel SGX-enabled CPUs) and run machine learning tasks in it with confidentiality and integrity guaranteed. To improve performance, AI accelerators have been widely employed for modern machine learning tasks. However, how to protect privacy on an AI accelerator remains an open question. To address this question, we propose a solution for efficient privacy-…
Private data generated by edge devices -- from smart phones to automotive electronics -- are highly informative when aggregated but can be damaging when mishandled. A variety of solutions are being explored but have not yet won the public's trust and full backing of mobile platforms. In this work, we propose numerical aggregation protocols that empirically improve upon prior art, while providing comparable local differential privacy guarantees. Sharing a single private bit per value supports privacy metering that enable privacy controls and guarantees that are not covered by differential priva…
In this paper, we introduce an efficient and trustworthy conditional privacy-preserving communication protocol for VANETs based on proxy re-signature. The proposed protocol is characterized by the Trusted Authority (TA) designating the Roadside Units (RSUs) to translate signatures computed by the On-Board Units (OBUs) into one that are valid with respect to TA's public key. In addition, the proposed protocol offers both a priori and a posteriori countermeasures: it can not only provide fast anonymous authentication and privacy tracking, but guarantees message trustworthiness for vehicle-to-veh…
Recently, Xie et al. (IEEE Transactions on Information Forensics and Security, vol. 16, pp. 3068-3081, 2021) proposed a privacy-preserving Online Ride-Hailing (ORH) protocol that does not make use of a trusted third-party server. The primary goal of such privacy-preserving ORH protocols is to ensure the privacy of riders' and drivers' location data w.r.t. the ORH Service Provider (SP). In this note, we demonstrate a passive attack by the SP in the protocol of Xie et al. that enables it to completely recover the location of the rider as well as that of the responding drivers in each and every r…
In this paper, we present a protocol for facilitating trust-less cross-chain cryptocurrency transfers that preserve privacy of bridge withdrawals. We leverage zero-knowledge primitives that are commonly used to design cryptocurrency mixing protocols to provide similar functionality but across two or more blockchains. To that end, we receive cryptocurrency mixing for free through the bridge operations and de-scribe how to extend these protocols to incentivise bridge transfers using past ideas. We describe how resulting protocols lead to similar vampire style attacks coined in the Uniswap vs. Su…
Cryptographic primitives are essential for constructing privacy-preserving communication mechanisms. There are situations in which two parties that do not know each other need to exchange sensitive information on the Internet. Trust management mechanisms make use of digital credentials and certificates in order to establish trust among these strangers. We address the problem of choosing which credentials are exchanged. During this process, each party should learn no information about the preferences of the other party other than strictly required for trust establishment. We present a method to…
Modern cars are evolving in many ways. Technologies such as infotainment systems and companion mobile applications collect a variety of personal data from drivers to enhance the user experience. This paper investigates the extent to which car drivers understand the implications for their privacy, including that car manufacturers must treat that data in compliance with the relevant regulations. It does so by distilling out drivers' concerns on privacy and relating them to their perceptions of trust on car cyber-security. A questionnaire is designed for such purposes to collect answers from a se…
In this paper, we propose a trust-centric privacy-preserving blockchain for dynamic spectrum access in IoT networks. To be specific, we propose a trust evaluation mechanism to evaluate the trustworthiness of sensing nodes and design a Proof-of-Trust (PoT) consensus mechanism to build a scalable blockchain with high transaction-per-second (TPS). Moreover, a privacy protection scheme is proposed to protect sensors' real-time geolocatioin information when they upload sensing data to the blockchain. Two smart contracts are designed to make the whole procedure (spectrum sensing, spectrum auction, a…
Differential privacy has emerged as the main definition for private data analysis and machine learning. The {\em global} model of differential privacy, which assumes that users trust the data collector, provides strong privacy guarantees and introduces small errors in the output. In contrast, applications of differential privacy in commercial systems by Apple, Google, and Microsoft, use the {\em local model}. Here, users do not trust the data collector, and hence randomize their data before sending it to the data collector. Unfortunately, local model is too strong for several important applica…
The Internet of Things (IoT) is considered as the key enabling technology for smart services. Security and privacy are particularly open challenges for IoT applications due to the widespread use of commodity devices. This work introduces two hardware-based lightweight security mechanisms to ensure sensed data trustworthiness (i.e., sensed data protection and sensor node protection) and usage privacy of the sensors (i.e., privacy-aware reporting of the sensed data) for centralized and decentralized IoT applications. Physically unclonable functions (PUF) form the basis of both proposed mechanism…
With an enormous range of applications, Internet of Things (IoT) has magnetized industries and academicians from everywhere. IoT facilitates operations through ubiquitous connectivity by providing Internet access to all the devices with computing capabilities. With the evolution of wireless infrastructure, the focus from simple IoT has been shifted to smart, connected and mobile IoT (M-IoT) devices and platforms, which can enable low-complexity, low-cost and efficient computing through sensors, machines, and even crowdsourcing. All these devices can be grouped under a common term of M-IoT. Eve…
Privacy and security-related concerns are growing as machine learning reaches diverse application domains. The data holders want to train or infer with private data while exploiting accelerators, such as GPUs, that are hosted in the cloud. Cloud systems are vulnerable to attackers that compromise the privacy of data and integrity of computations. Tackling such a challenge requires unifying theoretical privacy algorithms with hardware security capabilities. This paper presents DarKnight, a framework for large DNN training while protecting input privacy and computation integrity. DarKnight relie…
Decentralised cloud computing platforms enable individuals to offer and rent resources in a peer-to-peer fashion. They must assign resources from multiple sellers to multiple buyers and derive prices that match the interests and capacities of both parties. The assignment process must be decentralised, fair and transparent, but also protect the privacy of buyers. We present PASTRAMI, a decentralised platform enabling trustworthy assignments of items and prices between a large number of sellers and bidders, through the support of multi-item auctions. PASTRAMI uses threshold blind signatures and …
Collaborative systems, such as Online Social Networks and the Internet of Things, enable users to share privacy sensitive content. Content in these systems is often co-owned by multiple users with different privacy expectations, leading to possible multiuser privacy conflicts. In order to resolve these conflicts, various agreement mechanisms have been designed and agents that could participate in such mechanisms have been proposed. However, research shows that users hesitate to use software tools for managing their privacy. To remedy this, we argue that users should be supported by trustworthy…
Threat information sharing is considered as one of the proactive defensive approaches for enhancing the overall security of trusted partners. Trusted partner organizations can provide access to past and current cybersecurity threats for reducing the risk of a potential cyberattack - the requirements for threat information sharing range from simplistic sharing of documents to threat intelligence sharing. Therefore, the storage and sharing of highly sensitive threat information raises considerable concerns regarding constructing a secure, trusted threat information exchange infrastructure. Estab…
In response to the COVID-19 pandemic, Bluetooth-based contact tracing has been deployed in many countries with the help of the developers of smartphone operating systems that provide APIs for privacy-preserving exposure notification. However, it has been assumed by the design that the OS developers, smartphone vendors, or governments will not violate people's privacy. We propose a privacy-preserving exposure notification under situations where none of the middle entities can be trusted. We believe that it can be achieved with small changes to the existing mechanism: random numbers are generate…
Lack of trust between organisations and privacy concerns about their data are impediments to an otherwise potentially symbiotic joint data analysis. We propose DataRing, a data sharing system that allows mutually mistrusting participants to query each others' datasets in a privacy-preserving manner while ensuring the correctness of input datasets and query answers even in the presence of (cheating) participants deviating from their true datasets. By relying on the assumption that if only a small subset of rows of the true dataset are known, participants cannot submit answers to queries deviati…
Arpanet, Internet, Internet of Services, Internet of Things, Internet of Skills. What next? We conjecture that in 15-20 years from now we will have the Internet of Neurons, a new Internet paradigm in which humans will be able to connect bi-directionally to the net using only their brain. The Internet of Neurons will provide new, tremendous opportunities thanks to constant access to unlimited information. It will empower all those outside of the technical industry, actually it will empower all human beings, to access and use technological products and services as everybody will be able to conne…
The roles of trust, security and privacy are somewhat interconnected, but different facets of next generation networks. The challenges in creating a trustworthy 6G are multidisciplinary spanning technology, regulation, techno-economics, politics and ethics. This white paper addresses their fundamental research challenges in three key areas. Trust: Under the current "open internet" regulation, the telco cloud can be used for trust services only equally for all users. 6G network must support embedded trust for increased level of information security in 6G. Trust modeling, trust policies and trus…
Network data needs to be shared for distributed security analysis. Anonymization of network data for sharing sets up a fundamental tradeoff between privacy protection versus security analysis capability. This privacy/analysis tradeoff has been acknowledged by many researchers but this is the first paper to provide empirical measurements to characterize the privacy/analysis tradeoff for an enterprise dataset. Specifically we perform anonymization options on single-fields within network packet traces and then make measurements using intrusion detection system alarms as a proxy for security analy…
Leveraging parallel hardware (e.g. GPUs) for deep neural network (DNN) training brings high computing performance. However, it raises data privacy concerns as GPUs lack a trusted environment to protect the data. Trusted execution environments (TEEs) have emerged as a promising solution to achieve privacy-preserving learning. Unfortunately, TEEs' limited computing power renders them not comparable to GPUs in performance. To improve the trade-off among privacy, computing performance, and model accuracy, we propose an \emph{asymmetric} model decomposition framework, \AsymML{}, to (1) accelerate t…
In this paper a homomorphic privacy preserving association rule mining algorithm is proposed which can be deployed in resource constrained devices (RCD). Privacy preserved exchange of counts of itemsets among distributed mining sites is a vital part in association rule mining process. Existing cryptography based privacy preserving solutions consume lot of computation due to complex mathematical equations involved. Therefore less computation involved privacy solutions are extremely necessary to deploy mining applications in RCD. In this algorithm, a semi-trusted mixer is used to unify the count…
Privacy-Preserving Cloud Computing is an emerging technology with many applications in various fields. Cloud computing is important because it allows for scalability, adaptability, and improved security. Likewise, privacy in cloud computing is important because it ensures that the integrity of data stored on the cloud maintains intact. This survey paper on privacy-preserving cloud computing can help pave the way for future research in related areas. This paper helps to identify existing trends by establishing a layered architecture along with a life cycle and an ecosystem for privacy-preservin…
The recent proliferation of smart devices has given rise to ubiquitous computing, an emerging computing paradigm which allows anytime & anywhere computing possible. In such a ubiquitous computing environment, customers release different computing or sensing tasks, and people, also known as data processors, participate in these tasks and get paid for providing their idle computing and communication resources. Thus, how to select an appropriate and reliable customer while not disclosing processors' privacy has become an interesting problem. In this article, we present a trust-based and privacy-p…
Android devices are equipped with many pre-installed applications which have the capability of tracking and monitoring users. Although applications coming pre-installed pose a great danger to user security and privacy, they have received little attention so far among researchers in the field. In this study, we collect a dataset comprising such applications and make it publicly available. Using this dataset, we analyze tracker SDKs, manifest files and the use of cloud services and report our results. We also conduct a user survey to understand concerns and perceptions of users. Last but not lea…
The public key infrastructure (PKI) based authentication protocol provides the basic security services for vehicular ad-hoc networks (VANETs). However, trust and privacy are still open issues due to the unique characteristics of vehicles. It is crucial for VANETs to prevent internal vehicles from broadcasting forged messages while simultaneously protecting the privacy of each vehicle against tracking attacks. In this paper, we propose a blockchain-based anonymous reputation system (BARS) to break the linkability between real identities and public keys to preserve privacy. The certificate and r…
