arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2408.03533cs.IRcs.AI
收藏

面向推荐的大语言模型终身个性化低秩适配

Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation

Shanghai Jiao Tong University（上海交通大学）

Jiachen Zhu, Jianghao Lin, Xinyi Dai, Bo Chen, Rong Shan, Jieming Zhu, Ruiming Tang, Yong Yu, Weinan Zhang

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对大语言模型推荐中LoRA个性化不足、长行为序列效率低、大数据集训练受限的问题，提出含个性化LoRA、长短模态检索器和Few2Many学习策略的RecLoRA，在公开数据集上性能优于现有基线。

摘要：

我们主要聚焦于大语言模型（LLMs）推荐领域，该领域近年来受到广泛研究，如何有效增强推荐系统的逻辑推理能力和开放世界知识是一项重大挑战。当前主流研究主要通过定制输入模板，或在预测层对齐语义空间与推荐空间的表征，将推荐模型的个性化信息注入LLMs中。但这类方法存在三大显著局限：（1）现有研究大多以LoRA（低秩适配）为核心组件，但LoRA参数的个性化程度不足，所有用户共享的LoRA矩阵无法适配不同用户的特征，导致性能欠佳。（2）终身个性化行为序列虽是实现个性化的理想数据，但由于LLM的训练和推理时间会随文本长度增加而上升，使用这类序列会带来效果与效率层面的问题。（3）受训练效率限制，现有方法无法适配大规模数据集，LLM仅能接触到数据集的一小部分（如不足10%）而非全部数据，限制了其对完整训练空间的覆盖。为解决上述问题，我们提出RecLoRA模型。该模型包含个性化LoRA模块（为不同用户维护独立的LoRA）和长短模态检索器（为不同模态检索不同长度的历史信息），在仅增加极小时耗的前提下显著提升了性能。此外，我们设计了Few2Many学习策略，以传统推荐模型为媒介，将小规模训练空间扩展至完整训练空间。在公开数据集上的大量实验表明，与现有基线模型相比，RecLoRA具备更优的效果。

英文摘要：

We primarily focus on the field of large language models (LLMs) for recommendation, which has been actively explored recently and poses a significant challenge in effectively enhancing recommender systems with logical reasoning abilities and open-world knowledge. Current mainstream efforts mainly center around injecting personalized information from recommendation models into LLMs by customizing input templates or aligning representations between semantic and recommendation spaces at the prediction layer. However, they face three significant limitations: (1) LoRA is mostly used as a core component in existing works, but personalization is not well established in LoRA parameters as the LoRA matrix shared by every user may not cater to different users' characteristics, leading to suboptimal performance. (2) Although lifelong personalized behavior sequences are ideal for personalization, their use raises effectiveness and efficiency issues since LLMs require escalating training and inference time to extend text lengths. (3) Existing approaches aren't scalable for large datasets due to training efficiency constraints. Thus, LLMs only see a small fraction of the datasets (e.g., less than 10%) instead of the whole datasets, limiting their exposure to the full training space. To address these problems, we propose RecLoRA. This model incorporates a Personalized LoRA module that maintains independent LoRAs for different users and a Long-Short Modality Retriever that retrieves different history lengths for different modalities, significantly improving performance while adding minimal time cost. Furthermore, we design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces. Extensive experiments on public datasets demonstrate the efficacy of our RecLoRA compared to existing baseline models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑