Federated User Behavior Modeling for Privacy-Preserving LLM Recommendation
School of Computing and Artificial Intelligence, Shandong Normal University(山东师范大学计算与人工智能学院);School of Computer Science and Technology, Shandong University(山东大学计算机科学与技术学院);School of Electrical Engineering and Computer Science, The University of Queensland(昆士兰大学电气与计算机工程学院);School of Computing and Artificial Intelligence, Shandong University of Finance and Economic(山东财经大学计算与人工智能学院)
大型语言模型在推荐系统中表现出色,但用户数据有限稀疏限制了其建模能力。为解决此问题,现有研究探索了跨域推荐任务。然而,先前方法通常假设领域重叠且可访问。None of the LLM methods address the privacy-preserving issues in the CDR settings, that is, Privacy-Preserving Cross-Domain Recommendation. Conducting non-overlapping PPCDR with LLM is challenging since: 1)The inability to share user identity or behavioral data across domains impedes effective cross-domain alignment. 2)The heterogeneity of data modalities across domains complicates knowledge integration. 3)Fusing collaborative filtering signals from traditional recommendation models with LLMs is difficult, as they operate within distinct feature spaces. To address the above issues, we propose SF-UBM, a Semantic-enhanced Federated User Behavior Modeling method. Specifically, to deal with Challenge 1, we leverage natural language as a universal bridge to connect disjoint domains via a semantic-enhanced federated architecture. Here, text-based item representations are encrypted and shared, while user-specific data remains local. To handle Challenge 2, we design a Fact-counter Knowledge Distillation module to integrate domain-agnostic knowledge with domain-specific knowledge, across different data modalities. To tackle Challenge 3, we project pre-learned user preferences and cross-domain item representations into the soft prompt space, aligning behavioral and semantic spaces for effective LLM learning. We conduct extensive experiments on three pairs of real-world domains, and the experimental results demonstrate the effectiveness of SF-UBM compared to the recent SOTA methods.
英文摘要:
Large Language Models have shown great success in recommender systems. However, the limited and sparse nature of user data often restricts the LLM's ability to effectively model behavior patterns. To address this, existing studies have explored cross-domain solutions by conducting Cross-Domain Recommendation tasks. But previous methods typically assume domains are overlapped and can be accessed readily. None of the LLM methods address the privacy-preserving issues in the CDR settings, that is, Privacy-Preserving Cross-Domain Recommendation. Conducting non-overlapping PPCDR with LLM is challenging since: 1)The inability to share user identity or behavioral data across domains impedes effective cross-domain alignment. 2)The heterogeneity of data modalities across domains complicates knowledge integration. 3)Fusing collaborative filtering signals from traditional recommendation models with LLMs is difficult, as they operate within distinct feature spaces. To address the above issues, we propose SF-UBM, a Semantic-enhanced Federated User Behavior Modeling method. Specifically, to deal with Challenge 1, we leverage natural language as a universal bridge to connect disjoint domains via a semantic-enhanced federated architecture. Here, text-based item representations are encrypted and shared, while user-specific data remains local. To handle Challenge 2, we design a Fact-counter Knowledge Distillation module to integrate domain-agnostic knowledge with domain-specific knowledge, across different data modalities. To tackle Challenge 3, we project pre-learned user preferences and cross-domain item representations into the soft prompt space, aligning behavioral and semantic spaces for effective LLM learning. We conduct extensive experiments on three pairs of real-world domains, and the experimental results demonstrate the effectiveness of SF-UBM compared to the recent SOTA methods.
Multimodal-enhanced Federated Recommendation: A Group-wise Fusion Approach
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University(吉林大学教育部符号计算与知识工程重点实验室);PolyU Academy for Artificial Intelligence, Hong Kong Polytechnic University(香港理工大学PolyU人工智能学院);Australian Artificial Intelligence Institute, FEIT, University of Technology Sydney(悉尼科技大学FEIT澳大利亚人工智能研究所);College of Computer Science, Inner Mongolia University(内蒙古大学计算机学院)
TransFR: Transferable Federated Recommendation with Adapter Tuning on Pre-trained Language Models
Key Laboratory of Big Data & Artificial Intelligence in Transportation, Ministry of Education, China(交通运输部大数据与人工智能重点实验室);Beijing Jiaotong University, China(北京交通大学);University of Technology Sydney, Australia(悉尼科技大学);Peking University, China(北京大学);Nanyang Technological University, Singapore(南洋理工大学)
A Systematic Survey on Federated Sequential Recommendation
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);The Hong Kong Polytechnic University(香港理工大学)
Personalized Recommendation Models in Federated Settings: A Survey
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Hong Kong University of Science and Technology(香港科技大学);WeBank(微众银行)
Joint SDU-NTU Centre for Artificial Intelligence Research (C-FAIR)(山东大学-南洋理工大学人工智能国际联合研究院(C-FAIR));Software School, Shandong University(山东大学软件学院);Shandong University(山东大学);Sino-Singapore International Joint Research Institute(中新国际联合研究院);Alibaba-NTU Singapore Joint Research Institute(阿里巴巴-新加坡南洋理工大学联合研究学院);Nanyang Technological University(南洋理工大学);Microsoft Research Asia(微软亚洲研究院)
Navigating the Future of Federated Recommendation Systems with Foundation Models
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机科学与技术学院);Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University(香港理工大学数据科学与人工智能系)
FedHCDR: Federated Cross-Domain Recommendation with Hypergraph Signal Decoupling
Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Peng Cheng Laboratory(鹏城实验室);Sun Yat-sen University(中山大学);Chongqing University(重庆大学);Shanghai Pudong Development Bank(上海浦东发展银行)