FuXi-Linear: 解锁长期时间感知序列推荐中的线性注意力潜力
FuXi-Linear: Unleashing the Power of Linear Attention in Long-term Time-aware Sequential Recommendation
AI总结：
FuXi-Linear通过引入时间保留通道和线性位置通道，解决长序列推荐中的时间信号干扰和位置信息不足问题，实现高效推荐与显著速度提升。
摘要：
现代推荐系统主要依赖于具有二次复杂度的注意力机制，这限制了它们处理长用户序列的能力并减慢了推理速度。虽然线性注意力是一种有前途的替代方案，但现有研究面临三个关键挑战：（1）时间信号往往被忽视或通过简单的耦合整合，导致时间和语义信号之间相互干扰，同时忽视了行为周期性；（2）现有线性框架提供的位置信息不足；（3）主要关注短序列和浅层架构。为了解决这些问题，我们提出了FuXi-Linear，一种用于高效长序列推荐的线性复杂度模型。我们的方法引入了两个关键组件：（1）一个时间保留通道，通过时间数据独立计算周期性注意力权重，防止时间和语义信号之间的干扰；（2）一个线性位置通道，通过可学习的内核在线性复杂度内整合位置信息。此外，我们证明FuXi-Linear在千长度尺度上表现出稳健的幂律扩展特性，这一特性在先前的线性推荐研究中鲜有探索。在多个千个标记序列上的广泛实验表明，FuXi-Linear在推荐质量上优于最先进的模型，同时在prefill阶段相比竞争基线实现了高达10倍的速度提升，在decode阶段则高达21倍的速度提升。我们的代码已发布在公共仓库https://github.com/USTC-StarTeam/fuxi-linear。
英文摘要：
Modern recommendation systems primarily rely on attention mechanisms with quadratic complexity, which limits their ability to handle long user sequences and slows down inference. While linear attention is a promising alternative, existing research faces three critical challenges: (1) temporal signals are often overlooked or integrated via naive coupling that causes mutual interference between temporal and semantic signals while neglecting behavioral periodicity; (2) insufficient positional information provided by existing linear frameworks; and (3) a primary focus on short sequences and shallow architectures. To address these issues, we propose FuXi-Linear, a linear-complexity model designed for efficient long-sequence recommendation. Our approach introduces two key components: (1) a Temporal Retention Channel that independently computes periodic attention weights using temporal data, preventing crosstalk between temporal and semantic signals; (2) a Linear Positional Channel that integrates positional information through learnable kernels within linear complexity. Moreover, we demonstrate that FuXi-Linear exhibits a robust power-law scaling property at a thousand-length scale, a characteristic largely unexplored in prior linear recommendation studies. Extensive experiments on sequences of several thousand tokens demonstrate that FuXi-Linear outperforms state-of-the-art models in recommendation quality, while achieving up to 10$\times$ speedup in the prefill stage and up to 21$\times$ speedup in the decode stage compared to competitive baselines. Our code has been released in a public repository https://github.com/USTC-StarTeam/fuxi-linear.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
