TransFR: 基于预训练语言模型的可迁移联邦推荐
TransFR: Transferable Federated Recommendation with Adapter Tuning on Pre-trained Language Models
AI总结：
TransFR通过结合预训练语言模型的通用能力和本地数据微调的个性化能力，解决联邦推荐中的可迁移性、冷启动和隐私问题。
摘要：
联邦推荐（FRs）作为一种在设备端流行的服务，使得多个本地客户端能够共同学习一个全局模型，而无需披露用户隐私数据。在传统联邦推荐中，主要采用离散身份来表示客户端和物品，然后映射到领域特定的嵌入以参与模型训练。尽管性能表现良好，我们揭示了在联邦设置中存在三个固有局限性，即跨领域不可迁移性、冷启动设置下的无效性以及联邦训练期间可能的隐私侵犯问题。为此，我们提出了一种可迁移的联邦推荐模型TransFR，巧妙地结合了预训练模型赋予的一般能力以及通过微调本地私有数据获得的个性化能力。具体而言，它首先通过利用预训练模型和公开文本语料库学习领域无关的物品表示。为了适应FR任务，我们进一步引入了高效的联邦适配器微调和测试时适应机制，这些机制通过拟合每个客户端的私有数据分布来为每个客户端定制个性化本地适配器。我们从理论上证明了在联邦推荐中结合适配器微调的优势，既在有效性方面也体现在隐私保护方面。通过广泛的实验，我们表明我们的TransFR模型在可迁移性上超过了几种最先进的联邦推荐方法。
英文摘要：
Federated recommendations (FRs), facilitating multiple local clients to collectively learn a global model without disclosing user private data, have emerged as a prevalent on-device service. In conventional FRs, a dominant paradigm is to utilize discrete identities to represent clients and items, which are then mapped to domain-specific embeddings to participate in model training. Despite considerable performance, we reveal three inherent limitations that can not be ignored in federated settings, i.e., non-transferability across domains, ineffectiveness in cold-start settings, and potential privacy violations during federated training. To this end, we propose a transferable federated recommendation model, TransFR, which delicately incorporates the general capabilities empowered by pre-trained models and the personalized abilities by fine-tuning local private data. Specifically, it first learns domain-agnostic representations of items by exploiting pre-trained models with public textual corpora. To tailor for FR tasks, we further introduce efficient federated adapter-tuning and test-time adaptation mechanisms, which facilitate personalized local adapters for each client by fitting their private data distributions. We theoretically prove the advantages of incorporating adapter tuning in FRs regarding both effectiveness and privacy. Through extensive experiments, we show that our TransFR model surpasses several state-of-the-art FRs on transferability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
