T-RoPE：用于序列推荐的时间感知旋转位置编码

中文摘要 AI 辅助

大规模推荐系统日益采用大型语言模型背后的序列生成范式，将Transformer及其为文本所做的设计选择（包括旋转位置编码（RoPE））引入推荐领域。在语言模型中，RoPE编码标记索引以进行相对位置推理，但在推荐中，交互索引仅记录事件顺序，不反映经过的时间、跨尺度的行为周期或日历阶段。我们重新审视这一选择，提出T-RoPE，一种用于序列生成推荐的时间感知RoPE，用基于时间戳的角度、可学习的时间系数、多尺度频率库、偏移查询对齐和非平稳键旋转取代仅基于索引的旋转。我们证明标准RoPE（即使应用于时间戳）仍保持时间平移不变性，无法区分季节性上下文，而T-RoPE打破这种不变性同时保留RoPE接口。在五个公开基准上，T-RoPE在每个数据集的每个指标上均取得最佳结果，在稀疏PixelRec数据上HR@10比最强基线提升78-130%，在Amazon Books上各指标提升8-12%。在一个超过60亿交互的工业规模电商数据集上，它比HSTU+Time RAB骨干在每个指标上提升13-82%，消融实验将最大增益归因于多尺度频率（NDCG@50提升56%）和非平稳键（提升4%）。Shop应用中的在线A/B测试在转化率（+0.33%）和订单数（+0.63%）上取得正向提升。我们还提供前向和后向算法，其额外成本与序列长度和头维度呈线性关系，使时间感知RoPE对大型生成式推荐器保持实用。

英文摘要

Large-scale recommenders increasingly adopt the sequential generative recipe behind large language models, bringing the Transformer into recommendation along with design choices made for text, including Rotary Position Embedding (RoPE). In language models, RoPE encodes token indices for relative position reasoning, but in recommendation, an interaction index records only event order, saying nothing about elapsed time, behavioral cycles across scales, or calendar phase. We revisit this choice and propose T-RoPE, a time-aware RoPE for sequential generative recommendation that replaces index-only rotation with timestamp-based angles, learnable temporal coefficients, multiscale frequency banks, shifted query alignment, and non-stationary key rotation. We prove that standard RoPE, even on timestamps, remains time-translation invariant and cannot distinguish seasonal contexts, and that T-RoPE breaks this invariance while preserving the RoPE interface. Across five public benchmarks, T-RoPE achieves the best result on every metric on every dataset, improving over the strongest baseline by 78--130\% in HR@10 on the sparse PixelRec data and 8--12\% across metrics on Amazon Books. On an industrial-scale e-commerce dataset with more than 6B interactions, it improves every metric over the HSTU + Time RAB backbone by 13--82\%, with ablations attributing the largest gains to multiscale frequencies ($+56\%$ NDCG@50) and non-stationary keys ($+4\%$). An online A/B test in the Shop app yields positive lifts in conversion rate ($+0.33\%$) and order count ($+0.63\%$). We also provide forward and backward algorithms whose added cost is linear in sequence length and head dimension, keeping time-aware RoPE practical for large generative recommenders.

发表机构

• Shopify
• Massachusetts Institute of Technology（麻省理工学院）
• Liquid AI

机构由 AI 辅助整理，请以论文原文为准。