Mitigating Leakage in Federated Learning with Trusted Hardware
Hong Kong University of Science & Technology（香港科技大学）
Javad Ghareh Chamani, Dimitrios Papadopoulos
备注：
Presented at the Privacy Preserving Machine Learning Workshop (PriML/PPML Joint Edition) at the 34th Conference on Neural Information Processing Systems (NeurIPS 2020)
In federated learning, multiple parties collaborate in order to train a global model over their respective datasets. Even though cryptographic primitives (e.g., homomorphic encryption) can help achieve data privacy in this setting, some partial information may still be leaked across parties if this is done non-judiciously. In this work, we study the federated learning framework of SecureBoost [Cheng et al., FL@IJCAI'19] as a specific such example, demonstrate a leakage-abuse attack based on its leakage profile, and experimentally evaluate the effectiveness of our attack. We then propose two secure versions relying on trusted execution environments. We implement and benchmark our protocols to demonstrate that they are 1.2-5.4X faster in computation and need 5-49X less communication than SecureBoost.
Game-Theoretic Machine Unlearning: Mitigating Extra Privacy Leakage
School of Information Engineering, Zhongnan University of Economics and Law（中南财经政法大学信息工程学院）；Faculty of Data Science, City University of Macau（澳门城市大学数据科学学院）
Imperial College London（帝国理工学院）；Telefónica Research（西班牙电信研究院）
We propose and implement a Privacy-preserving Federated Learning ($PPFL$) framework for mobile systems to limit privacy leakages in federated learning. Leveraging the widespread presence of Trusted Execution Environments (TEEs) in high-end and mobile devices, we utilize TEEs on clients for local training, and on servers for secure aggregation, so that model/gradient updates are hidden from adversaries. Challenged by the limited memory size of current TEEs, we leverage greedy layer-wise training to train each model's layer inside the trusted area until its convergence. The performance evaluatio…
Petrus team, Inria, France（法国国家信息与自动化研究所 Petrus 团队）；Magnet team, Inria, France（法国国家信息与自动化研究所 Magnet 团队）
Imagine a group of citizens willing to collectively contribute their personal data for the common good to produce socially useful information, resulting from data analytics or machine learning computations. Sharing raw personal data with a centralized server performing the computation could raise concerns about privacy and a perceived risk of mass surveillance. Instead, citizens may trust each other and their own devices to engage into a decentralized computation to collaboratively produce an aggregate data release to be shared. In the context of secure computing nodes exchanging messages over…
Concept bottleneck models perform classification by first predicting which of a list of human provided concepts are true about a datapoint. Then a downstream model uses these predicted concept labels to predict the target label. The predicted concepts act as a rationale for the target prediction. Model trust issues emerge in this paradigm when soft concept labels are used: it has previously been observed that extra information about the data distribution leaks into the concept predictions. In this work we show how Monte-Carlo Dropout can be used to attain soft concept predictions that do not c…
A large body of work shows that machine learning (ML) models can leak sensitive or confidential information about their training data. Recently, leakage due to distribution inference (or property inference) attacks is gaining attention. In this attack, the goal of an adversary is to infer distributional information about the training data. So far, research on distribution inference has focused on demonstrating successful attacks, with little attention given to identifying the potential causes of the leakage and to proposing mitigations. To bridge this gap, as our main contribution, we theoreti…
Modern databases and data-warehousing systems separate query processing and durable storage. Storage systems have idiosyncratic bugs and security vulnerabilities, thus attacks that compromise only storage are a realistic threat. In this paper, we show that encryption alone is not sufficient to protect databases from compromised storage. Using MongoDB WiredTiger as a concrete example, we demonstrate that sizes of encrypted writes to a durable write-ahead log can reveal sensitive information about the inputs and activities of MongoDB applications. We then design, implement, and evaluate BigFoot,…
Vanderbilt University（范德堡大学）；University of Virginia（弗吉尼亚大学）；Case Western Reserve University（凯斯西储大学）
Graph Neural Networks (GNNs) have shown great power in learning node representations on graphs. However, they may inherit historical prejudices from training data, leading to discriminatory bias in predictions. Although some work has developed fair GNNs, most of them directly borrow fair representation learning techniques from non-graph domains without considering the potential problem of sensitive attribute leakage caused by feature propagation in GNNs. However, we empirically observe that feature propagation could vary the correlation of previously innocuous non-sensitive features to the sen…
Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs
Faculty of Engineering, Shenzhen MSU-BIT University（深圳北理莫斯科大学工程学院）；Faculty of CMC, Shenzhen MSU-BIT University（深圳北理莫斯科大学计算机、数学与力学学院）；Department of CDS, Indian Institute of Science（印度科学学院计算机与数据科学系）
Auditing and Mitigating Privacy Leakage in Cloud-Edge Collaborative Decoding
The Hong Kong Polytechnic University（香港理工大学）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）；School of Software, Tsinghua University（清华大学软件学院）
Mitigating Sensitive Information Leakage in LLMs4Code through Machine Unlearning
College of Computer Science and Technology, National University of Defense Technology, Changsha, Hunan, 410073, China（计算机科学与技术学院，国防科技大学，长沙，湖南，410073，中国）；Xichang Satellite Launch Center, Xichang, Sichuan, 615000, China（西昌卫星发射中心，西昌，四川，615000，中国）；Academy of Military Sciences, Beijing, 100091, China（军事科学院，北京，100091，中国）
Beyond Vector Hiding: Breaking and Mitigating Shared-Direction Weight Obfuscation in TEE-Offloaded Large Language Models
School of Cyber Science and Technology, Shandong University（山东大学网络空间安全学院）；Tsinghua University（清华大学）；Suzhou Research Institute of Shandong University（山东大学苏州研究院）
Compression is an emerging source of exploitable side-channel leakage that threatens data security, particularly in web applications where compression is indispensable for performance reasons. Current approaches to mitigating compression side channels have drawbacks in that they either degrade compression ratio drastically or require too much effort from developers to be widely adopted. To bridge the gap, we develop Debreach, a static analysis and program transformation based approach to mitigating compression side channels. Debreach consists of two steps. First, it uses taint analysis to soun…
RAG-CT: Mitigating Privacy Risks on Retrieval-Augmented Generation Systems via Scanning Prompt Distribution
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；City University of Hong Kong（香港城市大学）；Virginia Tech（弗吉尼亚理工大学）；University of South Florida（南佛罗里达大学）；Rochester Institute of Technology（罗切斯特理工学院）
Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
University of California, Irvine（加州大学欧文分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）；University of California, Davis（加州大学戴维斯分校）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
School of Computer Engineering and Science（计算机工程与科学学院）；Shanghai University（上海大学）
It has been proved that deep neural networks are facing a new threat called backdoor attacks, where the adversary can inject backdoors into the neural network model through poisoning the training dataset. When the input containing some special pattern called the backdoor trigger, the model with backdoor will carry out malicious task such as misclassification specified by adversaries. In text classification systems, backdoors inserted in the models can cause spam or malicious speech to escape detection. Previous work mainly focused on the defense of backdoor attacks in computer vision, little a…
The proliferation of smart, connected, always listening devices have introduced significant privacy risks to users in a smart home environment. Beyond the notable risk of eavesdropping, intruders can adopt machine learning techniques to infer sensitive information from audio recordings on these devices, resulting in a new dimension of privacy concerns and attack variables to smart home users. Techniques such as sound masking and microphone jamming have been effectively used to prevent eavesdroppers from listening in to private conversations. In this study, we explore the problem of adversaries…
Robust and leakage-resilient device-independent oblivious transfer in MiniQCrypt
Centre for Quantum Technologies, Singapore（新加坡量子技术中心）；Department of Computer Science, National University of Singapore（新加坡国立大学计算机系）；MajuLab, UMI 3654, Singapore（新加坡UMI 3654 MajuLab）
Data deduplication is able to effectively identify and eliminate redundant data and only maintain a single copy of files and chunks. Hence, it is widely used in cloud storage systems to save storage space and network bandwidth. However, the occurrence of deduplication can be easily identified by monitoring and analyzing network traffic, which leads to the risk of user privacy leakage. The attacker can carry out a very dangerous side channel attack, i.e., learn-the-remaining-information (LRI) attack, to reveal users' privacy information by exploiting the side channel of network traffic in dedup…
