融合分层出口的紧凑型大语言模型推荐系统

中文摘要 AI 辅助

基于大语言模型的推荐系统（LLM-RS）展现出卓越能力，但对许多实际应用而言计算成本过高。紧凑型大语言模型（Compact LLMs）是实用替代方案，但其容量缩减常需推理或知识蒸馏方法，这会增加延迟或依赖更大模型，结合自回归生成时还面临严重的可扩展性瓶颈。相比之下，判别式LLM-RS通过嵌入相似性实现高效的全语料库排序，但其紧凑型主干在表达能力和结构适应性上仍有限制。本文提出用于序列推荐的分层出口融合框架FLEXRec，这是一种增强紧凑型LLM同时保留可扩展全语料库排序的判别式框架。FLEXRec在多个Transformer层插入预测头（即出口）并自适应融合其分数分布；自适应连续路由器（AC-Router）为每个用户序列动态选择出口的数量和具体出口，同时新型目标k铰链损失调节路由稀疏性。在三个真实数据集上，使用Qwen 3 1.7B和Llama 3.2 3B开展的实验表明，FLEXRec在紧凑型主干方法中达到了最先进的准确率，同时保持极高的效率。代码：this https URL

英文摘要

Large language model-based recommender systems (LLM-RSs) have demonstrated remarkable capabilities, but are computationally unsustainable for many real-world applications. Compact LLMs offer a practical alternative, yet their reduced capacity often requires reasoning or knowledge distillation methods that increase latency or depend on larger models. Combined with autoregressive generation, these approaches face severe scalability bottlenecks. In contrast, discriminative LLM-RSs enable efficient full-corpus ranking through embedding similarity, but compact backbones remain limited in expressiveness and structural adaptivity. We propose the Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking. FLEXRec inserts prediction heads (i.e., exits) at multiple transformer layers and adaptively fuses their score distributions. An adaptive continuous router (AC-Router) dynamically selects both the number and identity of exits for each user sequence, while a novel target-k hinge loss regulates routing sparsity. Experiments on three real-world datasets with Qwen 3 1.7B and Llama 3.2 3B show that FLEXRec achieves state-of-the-art accuracy among competing methods while remaining highly efficient. Code: https://github.com/xurong-liang/FLEXRec

补充信息

备注

Accepted by ICDM'26