arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.17410cs.IRcs.AI
收藏

通过中间层自硬负样本改进基于LLM的推荐

Improving LLM-based Recommendation with Self-Hard Negatives from Intermediate Layers

GSAI, Renmin University of China(GSAI,中国人民大学)

Bingqian Li, Bowen Zheng, Xiaolei Wang, Long Zhang, Jinpeng Wang, Sheng Chen, Wayne Xin Zhao, Ji-rong Wen

AI总结:

ILRec通过利用中间层自硬负样本改进LLM推荐系统,结合跨层优化和蒸馏提升负样本质量与判别性。

摘要:

大型语言模型(LLMs)在推荐系统中展现出巨大潜力,其中监督微调(SFT)常用于适应。后续研究进一步引入偏好学习以将负样本纳入训练过程。然而,现有方法依赖于序列级、离线生成的负样本,当适应LLM到具有大规模负样本空间的推荐任务时,其判别性和信息性较差。为了解决这些挑战,我们提出ILRec,一种新的基于LLM的推荐偏好微调框架,利用从中间层提取的自硬负样本信号来改进偏好学习。具体来说,我们从中间层识别出自硬负样本标记作为细粒度的负样本监督,动态反映模型的偏好学习过程。为了有效将这些信号整合到训练中,我们设计了一个包含跨层偏好优化和跨层偏好蒸馏的两阶段框架,使模型能够共同区分信息丰富的负样本并提高中间层负样本信号的质量。此外,我们引入了一个轻量级的协同过滤模型,为负样本信号分配token级奖励,以缓解过度惩罚假负样本的风险。在三个数据集上的广泛实验表明,ILRec在增强基于LLM的推荐系统性能方面是有效的。

英文摘要:

Large language models (LLMs) have shown great promise in recommender systems, where supervised fine-tuning (SFT) is commonly used for adaptation. Subsequent studies further introduce preference learning to incorporate negative samples into the training process. However, existing methods rely on sequence-level, offline-generated negatives, making them less discriminative and informative when adapting LLMs to recommendation tasks with large negative item spaces. To address these challenges, we propose ILRec, a novel preference fine-tuning framework for LLM-based recommendation, leveraging self-hard negative signals extracted from intermediate layers to improve preference learning. Specifically, we identify self-hard negative tokens from intermediate layers as fine-grained negative supervision that dynamically reflects the model's preference learning process. To effectively integrate these signals into training, we design a two-stage framework comprising cross-layer preference optimization and cross-layer preference distillation, enabling the model to jointly discriminate informative negatives and enhance the quality of negative signals from intermediate layers. In addition, we introduce a lightweight collaborative filtering model to assign token-level rewards for negative signals, mitigating the risk of over-penalizing false negatives. Extensive experiments on three datasets demonstrate ILRec's effectiveness in enhancing the performance of LLM-based recommender systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑