Recommendation services are extensively adopted in several user-centered applications as a tool to alleviate the information overload problem and help users in orienteering in a vast space of possible choices. In such scenarios, data ownership is a crucial concern since users may not be willing to share their sensitive preferences (e.g., visited locations) with a central server. Unfortunately, data harvesting and collection is at the basis of modern, state-of-the-art approaches to recommendation. To address this issue, we present FPL, an architecture in which users collaborate in training a central factorization model while controlling the amount of sensitive data leaving their devices. The proposed approach implements pair-wise learning-to-rank optimization by following the Federated Learning principles, originally conceived to mitigate the privacy risks of traditional machine learning. The public implementation is available at https://split.to/sisinflab-fpl.
Deep retrieval models are widely used for learning entity representations and recommendations. Federated learning provides a privacy-preserving way to train these models without requiring centralization of user data. However, federated deep retrieval models usually perform much worse than their centralized counterparts due to non-IID (independent and identically distributed) training data on clients, an intrinsic property of federated learning that limits negatives available for training. We demonstrate that this issue is distinct from the commonly studied client drift problem. This work propo…
A Systematic Survey on Federated Sequential Recommendation
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；The Hong Kong Polytechnic University（香港理工大学）
A Survey of Personalized Federated Foundation Models for Privacy-Preserving Recommendation
Australian Artificial Intelligence Institute, University of Technology Sydney（澳大利亚人工智能研究所，悉尼技术大学）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University（香港理工大学数据科学与人工智能系）
City University of Hong Kong（香港城市大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
In this paper, we are interested in what we term the federated private bandits framework, that combines differential privacy with multi-agent bandit learning. We explore how differential privacy based Upper Confidence Bound (UCB) methods can be applied to multi-agent environments, and in particular to federated learning environments both in `master-worker' and `fully decentralized' settings. We provide a theoretical analysis on the privacy and regret performance of the proposed methods and explore the tradeoffs between these two.
From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcare
Imperial College London（伦敦帝国学院）；Massachusetts General Hospital（麻省总医院）；Zhejiang University（浙江大学）；Weill Cornell Medicine of Cornell University（康奈尔大学威尔医学院）；Royal Brompton Hospital（皇家布朗普顿医院）；King’s College London（伦敦国王学院）
Federated Latent Factor Model for Bias-Aware Recommendation with Privacy-Preserving
College of Computer and Information Science, Southwest University, Chongqing 400715, China（西南大学计算机与信息科学学院）；School of Engineering, University of Western Australia, Australia（西澳大学工程学院）；School of Cyber Science and Technology, Beihang University, Beijing 100191, China（北航信息科学与技术学院）
With the advent of big data era and the development of artificial intelligence and other technologies, data security and privacy protection have become more important. Recommendation systems have many applications in our society, but the model construction of recommendation systems is often inseparable from users' data. Especially for deep learning-based recommendation systems, due to the complexity of the model and the characteristics of deep learning itself, its training process not only requires long training time and abundant computational resources but also needs to use a large amount of …
Federated learning (FL) is an effective mechanism for data privacy in recommender systems by running machine learning model training on-device. While prior FL optimizations tackled the data and system heterogeneity challenges faced by FL, they assume the two are independent of each other. This fundamental assumption is not reflective of real-world, large-scale recommender systems -- data and system heterogeneity are tightly intertwined. This paper takes a data-driven approach to show the inter-dependence of data and system heterogeneity in real-world data and quantifies its impact on the overa…
Institute for Language and Speech Processing, Athena Research Center（雅典娜研究中心语言与语音处理研究所）；Department of Electrical and Computer Engineering, Democritus University of Thrace（色雷斯德谟克利特大学电气与计算机工程系）
With the growing number of Location-Based Social Networks, privacy preserving location prediction has become a primary task for helping users discover new points-of-interest (POIs). Traditional systems consider a centralized approach that requires the transmission and collection of users' private data. In this work, we present FedPOIRec, a privacy preserving federated learning approach enhanced with features from users' social circles for top-$N$ POI recommendations. First, the FedPOIRec framework is built on the principle that local data never leave the owner's device, while the local updates…
Hong Kong University of Science and Technology（香港科技大学）；Clustar；Peking University（北京大学）；Fuzhou University（福州大学）
Federated recommendation addresses the data silo and privacy problems altogether for recommender systems. Current federated recommender systems mainly utilize cryptographic or obfuscation methods to protect the original ratings from leakage. However, the former comes with extra communication and computation costs, and the latter damages model accuracy. Neither of them could simultaneously satisfy the real-time feedback and accurate personalization requirements of recommender systems. In this paper, we proposed federated masked matrix factorization (FedMMF) to protect the data privacy in federa…
FedFlex: Federated Learning for Diverse Netflix Recommendations
Vrije Universiteit Amsterdam Netherlands；Georgia Institute of Technology Mexico；Centrum Wiskunde \& Informatica Netherlands；Georgia Institute of Technology
College of Information Science and Engineering（信息科学与工程学院）；Ningbo University（宁波大学）
Recommender System (RS) is currently an effective way to solve information overload. To meet users' next click behavior, RS needs to collect users' personal information and behavior to achieve a comprehensive and profound user preference perception. However, these centrally collected data are privacy-sensitive, and any leakage may cause severe problems to both users and service providers. This paper proposed a novel privacy-preserved recommender system framework (PPRSF), through the application of federated learning paradigm, to enable the recommendation algorithm to be trained and carry out i…
Free-rider attacks against federated learning consist in dissimulating participation to the federated learning process with the goal of obtaining the final aggregated model without actually contributing with any data. This kind of attacks is critical in sensitive applications of federated learning, where data is scarce and the model has high commercial value. We introduce here the first theoretical and experimental analysis of free-rider attacks on federated learning schemes based on iterative parameters aggregation, such as FedAvg or FedProx, and provide formal guarantees for these attacks to…
University of Texas at Austin（得克萨斯大学奥斯汀分校）；CognitiveScale（认知规模公司）
Machine learning methods allow us to make recommendations to users in applications across fields including entertainment, dating, and commerce, by exploiting similarities in users' interaction patterns. However, in domains that demand protection of personally sensitive data, such as medicine or banking, how can we learn such a model without accessing the sensitive data, and without inadvertently leaking private information? We propose a new federated approach to learning global and local private models for recommendation without collecting raw data, user statistics, or information about person…
University of California, Davis（加州大学戴维斯分校）；The Ohio State University（俄亥俄州立大学）；Michigan State University（密歇根州立大学）；DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室）
Federated learning (FL) hyper-parameters significantly affect the training overheads in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters puts a high burden on FL practitioners since various applications prefer different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements of FL training. FedTune is lightweight and flexible, achieving 8.48%-26.75% improvement for different datasets…
Federated learning enables many local devices to train a deep learning model jointly without sharing the local data. Currently, most of federated training schemes learns a global model by averaging the parameters of local models. However, most of these training schemes suffer from high communication cost resulted from transmitting full local model parameters. Moreover, directly averaging model parameters leads to a significant performance degradation, due to the class-imbalanced non-iid data on different devices. Especially for the real life federated learning tasks involving extreme classific…
Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）
Federated recommendation systems can provide good performance without collecting users' private data, making them attractive. However, they are susceptible to low-cost poisoning attacks that can degrade their performance. In this paper, we develop a novel federated recommendation technique that is robust against the poisoning attack where Byzantine clients prevail. We argue that the key to Byzantine detection is monitoring of gradients of the model parameters of clients. We then propose a robust learning strategy where instead of using model parameters, the central server computes and utilizes…
JD Intelligent Cities Research, JD Digits（京东智能城市研究院，京东数科）；JD Intelligent Cities Business Unit, JD Digits（京东智能城市业务部，京东数科）；Institute of Artificial Intelligence, Southwest Jiaotong University（西南交通大学人工智能研究院）
It is commonly observed that the data are scattered everywhere and difficult to be centralized. The data privacy and security also become a sensitive topic. The laws and regulations such as the European Union's General Data Protection Regulation (GDPR) are designed to protect the public's data privacy. However, machine learning requires a large amount of data for better performance, and the current circumstances put deploying real-life AI applications in an extremely difficult situation. To tackle these challenges, in this paper we propose a novel privacy-preserving federated machine learning …
Current content filtering and blocking methods are susceptible to various circumvention techniques and are relatively slow in dealing with new threats. This is due to these methods using shallow pattern recognition that is based on regular expression rules found in crowdsourced block lists. We propose a novel system that aims to remedy the aforementioned issues by examining deep textual patterns of network-oriented content relating to the domain being interacted with. Moreover, we propose to use federated learning that allows users to take advantage of each other's localized knowledge/experien…
BGTplanner: Maximizing Training Accuracy for Differentially Private Federated Recommenders via Strategic Privacy Budget Allocation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）；Department of Computing, Faculty of Science and Engineering, Macquarie University（麦考瑞大学计算机系）；Machine Learning Department, Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学机器学习系）
Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；University of New South Wales（新南威尔士大学）
Federated learning is an emerging machine learning paradigm that enables multiple devices to train models locally and formulate a global model, without sharing the clients' local data. A federated learning system can be viewed as a large-scale distributed system, involving different components and stakeholders with diverse requirements and constraints. Hence, developing a federated learning system requires both software system design thinking and machine learning knowledge. Although much effort has been put into federated learning from the machine learning perspectives, our previous systematic…
Beijing University of Civil Engineering and Architecture（北京建筑大学）；Southwest Jiaotong University（西南交通大学）
Nowadays, user privacy is becoming an issue that cannot be bypassed for system developers, especially for that of web applications where data can be easily transferred through internet. Thankfully, federated learning proposes an innovative method to train models with distributed devices while data are kept in local storage. However, unlike general neural networks, although graph-based networks have achieved great success in classification tasks and advanced recommendation system, its high performance relies on the rich context provided by a graph structure, which is vulnerable when data attrib…
Shanghai Jiao Tong University（上海交通大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Alibaba Group（阿里巴巴集团）
Federated learning was proposed with an intriguing vision of achieving collaborative machine learning among numerous clients without uploading their private data to a cloud server. However, the conventional framework requires each client to leverage the full model for learning, which can be prohibitively inefficient for resource-constrained clients and large-scale deep learning tasks. We thus propose a new framework, called federated submodel learning, where clients download only the needed parts of the full model, namely submodels, and then upload the submodel updates. Nevertheless, the "posi…
Helsinki Research Center（赫尔辛基研究中心）；Europe Cloud Service Competence Center（欧洲云服务能力中心）；Huawei Technologies Oy (Finland) Co. Ltd（华为技术有限公司（芬兰））
We introduce the federated multi-view matrix factorization method that extends the federated learning framework to matrix factorization with multiple data sources. Our method is able to learn the multi-view model without transferring the user's personal data to a central server. As far as we are aware this is the first federated model to provide recommendations using multi-view matrix factorization. The model is rigorously evaluated on three datasets on production settings. Empirical validation confirms that federated multi-view matrix factorization outperforms simpler methods that do not take…
CSIRO（联邦科学与工业研究组织）；University of New South Wales（新南威尔士大学）
Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. A federated learning system can be viewed as a large-scale distributed system with different components and stakeholders as numerous client devices participate in federated learning. Designing a federated learning system requires software system design thinking apart from machine learning knowledge. Although much effort has been put into federated learning from the machine learning technique aspects, the software architecture design conc…
University of Science and Technology of China（中国科学技术大学）；Nanjing University of Science and Technology（南京理工大学）
Federated learning (FL) supports training models on geographically distributed devices. However, traditional FL systems adopt a centralized synchronous strategy, putting high communication pressure and model generalization challenge. Existing optimizations on FL either fail to speedup training on heterogeneous devices or suffer from poor communication efficiency. In this paper, we propose HADFL, a framework that supports decentralized asynchronous training on heterogeneous devices. The devices train model locally with heterogeneity-aware local steps using local data. In each aggregation cycle,…
Federated learning (FL) is becoming a popular paradigm for collaborative learning over distributed, private datasets owned by non-trusting entities. FL has seen successful deployment in production environments, and it has been adopted in services such as virtual keyboards, auto-completion, item recommendation, and several IoT applications. However, FL comes with the challenge of performing training over largely heterogeneous datasets, devices, and networks that are out of the control of the centralized FL server. Motivated by this inherent setting, we make a first step towards characterizing t…
National University of Singapore（新加坡国立大学）；Institute of Data Science, National University of Singapore（新加坡国立大学数据科学研究所）
The emerging availability of trained machine learning models has put forward the novel concept of Machine Learning Model Market in which one can harness the collective intelligence of multiple well-trained models to improve the performance of the resultant model through one-shot federated learning and ensemble learning in a data-free manner. However, picking the models available in the market for ensemble learning is time-consuming, as using all the models is not always the best approach. It is thus crucial to have an effective ensemble selection strategy that can find a good subset of the bas…
