arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.13344cs.IRcs.AI
收藏

面向推荐中终身序列行为理解的全栈优化大语言模型

Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation

Shanghai Jiao Tong University(上海交通大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室)

Rong Shan, Jiachen Zhu, Jianghao Lin, Chenxu Zhu, Bo Chen, Ruiming Tang, Yong Yu, Weinan Zhang

AI总结:

针对LLM在推荐中难以理解长用户行为序列的问题,提出ReLLaX框架,通过数据、提示和参数三层优化(SUBR、SPA、CFLoRA)增强序列行为理解,实验证明其优于现有基线。

摘要:

在本文中,我们解决了大语言模型(LLM)在推荐中的终身序列行为不理解问题,即LLM即使在上下文限制内也难以从长用户行为序列中提取有用信息。为解决这一问题,我们提出了ReLLaX(检索增强大语言模型增强版),一个在数据、提示和参数层面提供优化的框架。在数据层面,我们引入了语义用户行为检索(SUBR)以减少序列异质性,使LLM更容易提取关键信息。在提示层面增强方面,我们采用软提示增强(SPA)注入协作知识,使物品表示与推荐任务对齐,并改善LLM对物品关系的探索。最后,在参数层面,我们提出了组件全交互LoRA(CFLoRA),通过启用其组件之间的交互来增强LoRA的表达能力,从而更好地捕捉序列信息。此外,我们提出了新的视角来比较当前基于LoRA的LLM4Rec方法,即从组合和分解两个视角。我们从理论上证明,它们将LoRA用于推荐的方式是我们CFLoRA的退化版本,对原子组件交互有不同的约束。在三个公共数据集上的大量实验表明,ReLLaX优于现有基线,并能有效缓解终身序列行为不理解问题。

英文摘要:

In this paper, we address the lifelong sequential behavior incomprehension problem in large language models (LLMs) for recommendation, where LLMs struggle to extract useful information from long user behavior sequences, even within their context limits. To tackle this, we propose ReLLaX (Retrieval-enhanced Large Language models Plus), a framework offering optimization across data, prompt, and parameter levels. At the data level, we introduce Semantic User Behavior Retrieval (SUBR) to reduce sequence heterogeneity, making it easier for LLMs to extract key information. For prompt-level enhancement, we employ Soft Prompt Augmentation (SPA) to inject collaborative knowledge, aligning item representations with recommendation tasks and improving LLMs's exploration of item relationships. Finally, at the parameter level, we propose Component Fully-interactive LoRA (CFLoRA), which enhances LoRA's expressiveness by enabling interactions between its components, allowing better capture of sequential information. Moreover, we present new perspectives to compare current LoRA-based LLM4Rec methods, i.e. from both a composite and a decomposed view. We theoretically demonstrate that the ways they employ LoRA for recommendation are degraded versions of our CFLoRA, with different constraints on atom component interactions. Extensive experiments on three public datasets demonstrate ReLLaX's superiority over existing baselines and its ability to mitigate lifelong sequential behavior incomprehension effectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑