arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.09543cs.IR
收藏

重新思考用于序列推荐的大型语言模型架构

Rethinking Large Language Model Architectures for Sequential Recommendations

Hanbing Wang, Xiaorui Liu, Wenqi Fan, Xiangyu Zhao, Venkataramana Kini, Devendra Yadav, Fei Wang, Zhen Wen, Jiliang Tang, Hui Liu

AI总结:

本文提出 Lite-LLM4Rec 模型,通过直接的物品投影头替代 beam search 解码并引入分层 LLM 结构,大幅提升了序列推荐的推理效率与性能。

摘要:

最近,序列推荐已被适配到 LLM 范式中,以享受 LLM 的强大能力。基于 LLM 的方法通常将推荐信息表述为自然语言,并以自回归方式训练模型来预测下一个物品。尽管取得了显著成功,但推理的巨大计算开销对其实际应用构成了重大障碍。在这项工作中,我们致力于简化现有的基于 LLM 的推荐模型,并提出了一种简单但高效的模型 Lite-LLM4Rec。Lite-LLM4Rec 的主要目标是实现序列推荐任务的高效推理。Lite-LLM4Rec 通过使用直接的物品投影头来生成排名分数,从而规避了 beam search 解码。这一设计源于我们的经验观察:beam search 解码对于序列推荐最终是不必要的。此外,Lite-LLM4Rec 引入了专为高效处理与物品相关的大量上下文信息而定制的分层 LLM 结构,从而在享受 LLM 能力的同时降低了计算开销。在三个公开可用的数据集上的实验证实了 Lite-LLM4Rec 在性能和推理效率上的有效性(在 ML-1m 上显著提升了 46.8% 的性能和 97.28% 的效率),优于现有的基于 LLM 的方法。我们的实现将开源。

英文摘要:

Recently, sequential recommendation has been adapted to the LLM paradigm to enjoy the power of LLMs. LLM-based methods usually formulate recommendation information into natural language and the model is trained to predict the next item in an auto-regressive manner. Despite their notable success, the substantial computational overhead of inference poses a significant obstacle to their real-world applicability. In this work, we endeavor to streamline existing LLM-based recommendation models and propose a simple yet highly effective model Lite-LLM4Rec. The primary goal of Lite-LLM4Rec is to achieve efficient inference for the sequential recommendation task. Lite-LLM4Rec circumvents the beam search decoding by using a straight item projection head for ranking scores generation. This design stems from our empirical observation that beam search decoding is ultimately unnecessary for sequential recommendations. Additionally, Lite-LLM4Rec introduces a hierarchical LLM structure tailored to efficiently handle the extensive contextual information associated with items, thereby reducing computational overhead while enjoying the capabilities of LLMs. Experiments on three publicly available datasets corroborate the effectiveness of Lite-LLM4Rec in both performance and inference efficiency (notably 46.8% performance improvement and 97.28% efficiency improvement on ML-1m) over existing LLM-based methods. Our implementations will be open sourced.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑