arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.21986cs.IR
收藏

SpecTran: 基于频域的变压器适配器用于LLM增强的序列推荐

SpecTran: Spectral-Aware Transformer-based Adapter for LLM-Enhanced Sequential Recommendation

Zhejiang University(浙江大学);OPPO Research Institute(OPPO研究院)

Yu Cui, Feng Liu, Zhaoxiang Wang, Changwang Zhang, Jun Wang, Can Wang, Jiawei Chen

AI总结:

本文提出SpecTran,一种基于频域的变压器适配器,用于改进LLM增强的序列推荐。通过学习频域位置编码,引导注意力聚焦于重要频谱成分,从而提升推荐性能。

摘要:

传统序列推荐(SR)模型通过用户-项目交互学习低维项目ID嵌入,通常忽视文本信息如项目标题或描述。近年来,大语言模型(LLM)的进展促使研究者采用高维语义嵌入编码项目文本信息,并设计转换方法将此类嵌入注入SR模型。这些嵌入转换策略可分为两类,均存在明显缺点:1)基于适配器的方法导致显著的维度塌缩,信息集中于少数主导维度;2)基于SVD的方法僵化且手动,仅考虑少数主频谱成分而丢弃剩余频谱中的丰富信息。为解决这些限制,我们提出了SpecTran,一种在频域操作的变压器适配器,关注完整频谱以选择和聚合信息性成分。可学习的频谱位置编码将奇异值线索作为归纳偏置,引导注意力聚焦于显著的频谱成分,并促进嵌入维度的多样性。在四个真实世界数据集和三个SR基座上,它始终优于强基线,平均提升9.17%。

英文摘要:

Traditional sequential recommendation (SR) models learn low-dimensional item ID embeddings from user-item interactions, often overlooking textual information such as item titles or descriptions. Recent advances in Large Language Models (LLMs) have inspired a surge of research that encodes item textual information with high-dimensional semantic embeddings, and designs transformation methods to inject such embeddings into SR models. These embedding transformation strategies can be categorized into two types, both of which exhibits notable drawbacks: 1) adapter-based methods suffer from pronounced dimension collapse, concentrating information into a few dominant dimensions; 2) SVD-based methods are rigid and manual, considering only a few principal spectral components while discarding rich information in the remaining spectrum. To address these limitations, we propose SpecTran, a spectral-aware transformer-based adapter that operates in the spectral domain, attending to the full spectrum to select and aggregates informative components. A learnable spectral-position encoding injects singular-value cues as an inductive bias, guiding attention toward salient spectral components and promoting diversity across embedding dimensions. Across four real-world datasets and three SR backbones, it consistently outperforms strong baselines, achieving an average improvement of 9.17%.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑