Privacy Threats Against Federated Matrix Factorization
Hong Kong University of Science and Technology（香港科技大学）；Southern University of Science and Technology（南方科技大学）；WeBank Co. Ltd（微众银行）
Dashan Gao, Ben Tan, Ce Ju, Vincent W. Zheng, Qiang Yang
备注：
6 pages, 2 figures, 1 table, Accepted for Workshop on Federated Learning for Data Privacy and Confidentiality in Conjunction with IJCAI 2020 (FL-IJCAI'20)
Matrix Factorization has been very successful in practical recommendation applications and e-commerce. Due to data shortage and stringent regulations, it can be hard to collect sufficient data to build performant recommender systems for a single company. Federated learning provides the possibility to bridge the data silos and build machine learning models without compromising privacy and security. Participants sharing common users or items collaboratively build a model over data from all the participants. There have been some works exploring the application of federated learning to recommender systems and the privacy issues in collaborative filtering systems. However, the privacy threats in federated matrix factorization are not studied. In this paper, we categorize federated matrix factorization into three types based on the partition of feature space and analyze privacy threats against each type of federated matrix factorization model. We also discuss privacy-preserving approaches. As far as we are aware, this is the first study of privacy threats of the matrix factorization method in the federated learning framework.
Hong Kong University of Science and Technology（香港科技大学）；Clustar；Peking University（北京大学）；WeBank（微众银行）
To protect user privacy and meet law regulations, federated (machine) learning is obtaining vast interests in recent years. The key principle of federated learning is training a machine learning model without needing to know each user's personal raw private data. In this paper, we propose a secure matrix factorization framework under the federated learning setting, called FedMF. First, we design a user-level distributed matrix factorization framework where the model can be learned when each user only uploads the gradient information (instead of the raw preference data) to the server. While gra…
Helsinki Research Center（赫尔辛基研究中心）；Europe Cloud Service Competence Center（欧洲云服务能力中心）；Huawei Technologies Oy (Finland) Co. Ltd（华为技术有限公司（芬兰））
We introduce the federated multi-view matrix factorization method that extends the federated learning framework to matrix factorization with multiple data sources. Our method is able to learn the multi-view model without transferring the user's personal data to a central server. As far as we are aware this is the first federated model to provide recommendations using multi-view matrix factorization. The model is rigorously evaluated on three datasets on production settings. Empirical validation confirms that federated multi-view matrix factorization outperforms simpler methods that do not take…
Pohang University of Science and Technology（浦项科技大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California, San Diego（加利福尼亚大学圣迭戈分校）
Tensor factorization models offer an effective approach to convert massive electronic health records into meaningful clinical concepts (phenotypes) for data analysis. These models need a large amount of diverse samples to avoid population bias. An open challenge is how to derive phenotypes jointly across multiple hospitals, in which direct patient-level data sharing is not possible (e.g., due to institutional policies). In this paper, we developed a novel solution to enable federated tensor factorization for computational phenotyping without sharing patient-level data. We developed secure data…
Federated Binary Matrix Factorization using Proximal Optimization
KTH Royal Institute of Technology（瑞典皇家理工学院）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Institute for AI in Medicine, UK Essen（埃森大学医院人工智能医学研究所）；Ruhr University Bochum（波鸿鲁尔大学）；Monash University（莫纳什大学）
University of Maryland, Baltimore County（马里兰大学巴尔的摩郡分校）；LANL（洛斯阿拉莫斯国家实验室）
Non-negative matrix factorization (NMF) with missing-value completion is a well-known effective Collaborative Filtering (CF) method used to provide personalized user recommendations. However, traditional CF relies on the privacy-invasive collection of users' explicit and implicit feedback to build a central recommender model. One-shot federated learning has recently emerged as a method to mitigate the privacy problem while addressing the traditional communication bottleneck of federated learning. In this paper, we present the first unsupervised one-shot federated CF implementation, named FedSP…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Recent demands on data privacy have called for federated learning (FL) as a new distributed learning paradigm in massive and heterogeneous networks. Although many FL algorithms have been proposed, few of them have considered the matrix factorization (MF) model, which is known to have a vast number of signal processing and machine learning applications. Different from the existing FL algorithms that are designed for smooth problems with single block of variables, in federated MF (FedMF), one has to deal with challenging non-convex and non-smooth problems (due to constraints or regularization) w…
Shandong University（山东大学）；University of Amsterdam（阿姆斯特丹大学）；Ahold Delhaize（皇家阿霍德德尔海兹集团）
Federated recommender systems have distinct advantages in terms of privacy protection over traditional recommender systems that are centralized at a data center. However, previous work on federated recommender systems does not fully consider the limitations of storage, RAM, energy and communication bandwidth in a mobile environment. The scales of the models proposed are too large to be easily run on mobile devices. And existing federated recommender systems need to fine-tune recommendation models on each device, making it hard to effectively exploit collaborative filtering information among us…
Ping An Technology (Shenzhen) Co., Ltd.（平安科技（深圳）有限公司）；Duke University（杜克大学）；Sun Yat-sen University（中山大学）
Non-negative matrix factorization (NMF) based topic modeling is widely used in natural language processing (NLP) to uncover hidden topics of short text documents. Usually, training a high-quality topic model requires large amount of textual data. In many real-world scenarios, customer textual data should be private and sensitive, precluding uploading to data centers. This paper proposes a Federated NMF (FedNMF) framework, which allows multiple clients to collaboratively train a high-quality NMF based topic model with locally stored data. However, standard federated learning will significantly …
Tensor factorization has been proved as an efficient unsupervised learning approach for health data analysis, especially for computational phenotyping, where the high-dimensional Electronic Health Records (EHRs) with patients' history of medical procedures, medications, diagnosis, lab tests, etc., are converted to meaningful and interpretable medical concepts. Federated tensor factorization distributes the tensor computation to multiple workers under the coordination of a central server, which enables jointly learning the phenotypes across multiple hospitals while preserving the privacy of the…
Democritus University of Thrace（德谟克利特色雷斯大学）；Athena Research Center（雅典娜研究中心）
In this work, we present a federated version of the state-of-the-art Neural Collaborative Filtering (NCF) approach for item recommendations. The system, named FedNCF, enables learning without requiring users to disclose or transmit their raw data. Data localization preserves data privacy and complies with regulations such as the GDPR. Although federated learning enables model training without local data dissemination, the transmission of raw clients' updates raises additional privacy issues. To address this challenge, we incorporate a privacy-preserving aggregation method that satisfies the se…
In this study, we propose a novel SeqMF model to solve the problem of predicting the next app launch during mobile device usage. Although this problem can be represented as a classical collaborative filtering problem, it requires proper modification since the data are sequential, the user feedback is distributed among devices and the transmission of users' data to aggregate common patterns must be protected against leakage. According to such requirements, we modify the structure of the classical matrix factorization model and update the training procedure to sequential learning. Since the data…
Northeastern University（东北大学）；University of Virginia（弗吉尼亚大学）
The cyber-threat landscape has evolved tremendously in recent years, with new threat variants emerging daily, and large-scale coordinated campaigns becoming more prevalent. In this study, we propose CELEST (CollaborativE LEarning for Scalable Threat detection, a federated machine learning framework for global threat detection over HTTP, which is one of the most commonly used protocols for malware dissemination and communication. CELEST leverages federated learning in order to collaboratively train a global model across multiple clients who keep their data locally, thus providing increased priv…
Federated learning is emerging as a machine learning technique that trains a model across multiple decentralized parties. It is renowned for preserving privacy as the data never leaves the computational devices, and recent approaches further enhance its privacy by hiding messages transferred in encryption. However, we found that despite the efforts, federated learning remains privacy-threatening, due to its interactive nature across different parties. In this paper, we analyze the privacy threats in industrial-level federated learning frameworks with secure computation, and reveal such threats…
WeBank（微众银行）；Hong Kong University of Science and Technology（香港科技大学）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
With the emergence of data silos and popular privacy awareness, the traditional centralized approach of training artificial intelligence (AI) models is facing strong challenges. Federated learning (FL) has recently emerged as a promising solution under this new reality. Existing FL protocol design has been shown to exhibit vulnerabilities which can be exploited by adversaries both within and without the system to compromise data privacy. It is thus of paramount importance to make FL system designers to be aware of the implications of future FL algorithm design on privacy-preservation. Currentl…
Andalusian Research Institute in Data Science and Computational Intelligence（安达卢西亚数据科学与计算智能研究院）；University of Granada（格拉纳达大学）
Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real…
Federated learning (FL) is a machine learning technique that aims at training an algorithm across decentralized entities holding their local data private. Wireless mobile networks allow users to communicate with other fixed or mobile users. The road traffic network represents an infrastructure-based configuration of a wireless mobile network where the Connected and Automated Vehicles (CAV) represent the communicating entities. Applying FL in a wireless mobile network setting gives rise to a new threat in the mobile environment that is very different from the traditional fixed networks. The thr…
A Survey of Privacy Threats and Defense in Vertical Federated Learning: From Model Life Cycle Perspective
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Zhejiang University（浙江大学）；AntGroup（蚂蚁集团）；Ulsan National Institute of Science and Technology（蔚山国立科学技术院）；The University of Hong Kong（香港大学）
International Hellenic University（国际希腊大学）；Democritus University of Thrace（德谟克利特大学）
Given the increasing complexity of threats in smart cities, the changing environment, and the weakness of traditional security systems, which in most cases fail to detect serious threats such as zero-day attacks, the need for alternative more active and more effective security methods keeps increasing. Such approaches are the adoption of intelligent solutions to prevent, detect and deal with threats or anomalies under the conditions and the operating parameters of the infrastructure in question. This research paper introduces the development of an intelligent Threat Defense system, employing B…
Decentralized Federated Learning: A Survey on Security and Privacy
University of Windsor（温莎大学）；University of New Brunswick（新不伦瑞克大学）；University of Western Ontario（西安大略大学）；Hong Kong University of Science and Technology（香港科技大学）；WeBank（微众银行）
Nowadays, blockchain-based technologies are being developed in various industries to improve data security. In the context of the Industrial Internet of Things (IIoT), a chain-based network is one of the most notable applications of blockchain technology. IIoT devices have become increasingly prevalent in our digital world, especially in support of developing smart factories. Although blockchain is a powerful tool, it is vulnerable to cyber attacks. Detecting anomalies in blockchain-based IIoT networks in smart factories is crucial in protecting networks and systems from unexpected attacks. In…
Federated Attack Campaign Detection via Contrastive Encoding of Threat Indicators in Gradient Updates
Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学）；Bielefeld University（比勒费尔德大学）；Center for Cybersecurity TTZ-WUE（TTZ-WUE网络安全中心）
Recent years have witnessed the rapid growth of federated learning (FL), an emerging privacy-aware machine learning paradigm that allows collaborative learning over isolated datasets distributed across multiple participants. The salient feature of FL is that the participants can keep their private datasets local and only share model updates. Very recently, some research efforts have been initiated to explore the applicability of FL for matrix factorization (MF), a prevalent method used in modern recommendation systems and services. It has been shown that sharing the gradient updates in federat…
Cognitive Threat Intelligence and Explainable Federated Security Analytics for distributed Infrastructure Systems
Dept. of Information Studies, Trine University（信息研究系，特林大学）；Dept. of Business Information Systems, Central Michigan University（商业信息系统系，中央密歇根大学）；Dept. of CS, Maharishi International University（计算机科学系， Maharishi 国际大学）
Federated recommender system (FRS), which enables many local devices to train a shared model jointly without transmitting local raw data, has become a prevalent recommendation paradigm with privacy-preserving advantages. However, previous work on FRS performs similarity search via inner product in continuous embedding space, which causes an efficiency bottleneck when the scale of items is extremely large. We argue that such a scheme in federated settings ignores the limited capacities in resource-constrained user devices (i.e., storage space, computational overhead, and communication bandwidth…
University of the Basque Country UPV/EHU（巴斯克大学 UPV/EHU）
Network intrusion detection systems are evolving into intelligent systems that perform data analysis while searching for anomalies in their environment. Indeed, the development of deep learning techniques paved the way to build more complex and effective threat detection models. However, training those models may be computationally infeasible in most Edge or IoT devices. Current approaches rely on powerful centralized servers that receive data from all their parties - violating basic privacy constraints and substantially affecting response times and operational costs due to the huge communicat…
Florida International University（佛罗里达国际大学）；Southern Illinois University（南伊利诺伊大学）
Federated Learning (FL) has gained widespread popularity in recent years due to the fast booming of advanced machine learning and artificial intelligence along with emerging security and privacy threats. FL enables efficient model generation from local data storage of the edge devices without revealing the sensitive data to any entities. While this paradigm partly mitigates the privacy issues of users' sensitive data, the performance of the FL process can be threatened and reached a bottleneck due to the growing cyber threats and privacy violation techniques. To expedite the proliferation of F…
Ericsson Research（爱立信研究院）；Ericsson Security Research（爱立信安全研究院）；KTH Royal Institute of Technology（瑞典皇家理工学院）
Machine Learning (ML) is an important enabler for optimizing, securing and managing mobile networks. This leads to increased collection and processing of data from network functions, which in turn may increase threats to sensitive end-user information. Consequently, mechanisms to reduce threats to end-user privacy are needed to take full advantage of ML. We seamlessly integrate Federated Learning (FL) into the 3GPP 5G Network Data Analytics (NWDA) architecture, and add a Multi-Party Computation (MPC) protocol for protecting the confidentiality of local updates. We evaluate the protocol and fin…
Beyond Traditional Threats: A Persistent Backdoor Attack on Federated Learning
College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）；Southampton Ocean Engineering Joint Institute, Harbin Engineering University（哈尔滨工程大学南安普顿海洋工程联合学院）
Imperial College London（帝国理工学院）；IBM Research（IBM研究院）
Federated learning (FL) is one of the most important paradigms addressing privacy and data governance issues in machine learning (ML). Adversarial training has emerged, so far, as the most promising approach against evasion threats on ML models. In this paper, we take the first known steps towards federated adversarial training (FAT) combining both methods to reduce the threat of evasion during inference while preserving the data privacy during training. We investigate the effectiveness of the FAT protocol for idealised federated settings using MNIST, Fashion-MNIST, and CIFAR10, and provide fi…
Recent privacy awareness initiatives such as the EU General Data Protection Regulation subdued Machine Learning (ML) to privacy and security assessments. Federated Learning (FL) grants a privacy-driven, decentralized training scheme that improves ML models' security. The industry's fast-growing adaptation and security evaluations of FL technology exposed various vulnerabilities that threaten FL's confidentiality, integrity, or availability (CIA). This work assesses the CIA of FL by reviewing the state-of-the-art (SoTA) and creating a threat model that embraces the attack's surface, adversarial…
Leveraging Soft Prompts for Privacy Attacks in Federated Prompt Tuning
CISE, University of Florida, FL, USA（佛罗里达大学计算机与信息科学系）；North Carolina State University, NC, USA（北卡罗来纳州立大学）；Washington State University, WA, USA（华盛顿州立大学）；Seoul National University of Science and Technology, South Korea（韩国科学技术大学）
Smart meter measurements, though critical for accurate demand forecasting, face several drawbacks including consumers' privacy, data breach issues, to name a few. Recent literature has explored Federated Learning (FL) as a promising privacy-preserving machine learning alternative which enables collaborative learning of a model without exposing private raw data for short term load forecasting. Despite its virtue, standard FL is still vulnerable to an intractable cyber threat known as Byzantine attack carried out by faulty and/or malicious clients. Therefore, to improve the robustness of federat…
