房间里的大象：重新思考预训练语言模型在序列推荐中的使用
The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation
AI总结：
本研究针对序列推荐中预训练语言模型（PLM）利用率低、参数冗余的问题，提出用行为调优PLM初始化传统ID类序列推荐模型物品嵌入的轻量化方案，在五数据集上无额外推理成本即超越经典及SOTA模型。
摘要：
序列推荐（SR）在预训练语言模型（PLMs）的助力下已取得显著进展。部分基于PLM的SR模型直接利用PLM对用户历史行为的文本序列进行编码以学习用户表示，但鲜有研究深入探究PLM在行为序列建模中的能力与适配性。本研究首先对PLM与基于PLM的SR模型开展了大量模型分析，发现PLM在行为序列建模中存在严重的能力未充分利用及参数冗余问题。受此启发，我们探索了PLM在SR中的多种轻量化使用方式，旨在最大化激发PLM的SR能力，同时满足实际系统的效率与可用性需求。研究发现，采用经行为调优的PLM为传统基于ID的SR模型进行物品初始化，是基于PLM的SR中最经济的框架，该框架不会带来任何额外推理成本，却能较原始版本实现性能的大幅提升。在五个数据集上开展的大量实验表明，我们提出的这一简单通用的框架，在无需额外推理成本的情况下，相较经典SR模型与当前最优（SOTA）的基于PLM的SR模型均取得了显著提升。我们的代码可在https://github.com/777pomingzi/Rethinking-PLM-in-RS获取。
英文摘要：
Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs). Some PLM-based SR models directly use PLM to encode user historical behavior's text sequences to learn user representations, while there is seldom an in-depth exploration of the capability and suitability of PLM in behavior sequence modeling. In this work, we first conduct extensive model analyses between PLMs and PLM-based SR models, discovering great underutilization and parameter redundancy of PLMs in behavior sequence modeling. Inspired by this, we explore different lightweight usages of PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while satisfying the efficiency and usability demands of practical systems. We discover that adopting behavior-tuned PLMs for item initializations of conventional ID-based SR models is the most economical framework of PLM-based SR, which would not bring in any additional inference cost but could achieve a dramatic performance boost compared with the original version. Extensive experiments on five datasets show that our simple and universal framework leads to significant improvement compared to classical SR and SOTA PLM-based SR models without additional inference costs. Our code can be found in https://github.com/777pomingzi/Rethinking-PLM-in-RS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
