arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18283cs.IRcs.AI
收藏

HyTRec:一种用于长行为序列推荐的混合时间感知注意力架构

HyTRec: A Hybrid Temporal-Aware Attention Architecture for Long Behavior Sequential Recommendation

Shanghai Dewu Information Group(上海德 Wu 信息集团);Wuhan University(武汉大学);USTC;Beihang University(北航大学)

Lei Xin, Yuhao Zheng, Ke Cheng, Changjiang Jiang, Zifan Zhang, Fanhu Zeng

AI总结:

HyTRec通过混合注意力架构和时间感知delta网络,提升长行为序列推荐的精度与效率,实现工业级场景下的高命中率改进。

摘要:

在生成性推荐中,建模用户行为的长序列已成为关键前沿。然而,现有解决方案面临两难:线性注意力机制为了效率牺牲了检索精度,因为状态容量有限;而softmax注意力则因计算开销过大而受限。为解决这一挑战,我们提出了HyTRec,一种具有混合注意力架构的模型,该架构明确解耦了长期稳定偏好与短期意图尖峰。通过将大量历史序列分配给线性注意力分支,并保留专门的softmax注意力分支用于近期交互,我们的方法在涉及十万个交互的工业级场景中恢复了精确的检索能力。为缓解线性层中捕捉快速兴趣漂移的滞后,我们进一步设计了时间感知delta网络(TADN),以动态提升新鲜行为信号的同时有效抑制历史噪声。在工业级数据集上的实验证明,我们的模型在保持线性推理速度的同时优于强基线,特别是在为具有超长序列的用户带来超过8%的命中率提升方面效率显著。

英文摘要:

Modeling long sequences of user behaviors has emerged as a critical frontier in generative recommendation. However, existing solutions face a dilemma: linear attention mechanisms achieve efficiency at the cost of retrieval precision due to limited state capacity, while softmax attention suffers from prohibitive computational overhead. To address this challenge, we propose HyTRec, a model featuring a Hybrid Attention architecture that explicitly decouples long-term stable preferences from short-term intent spikes. By assigning massive historical sequences to a linear attention branch and reserving a specialized softmax attention branch for recent interactions, our approach restores precise retrieval capabilities within industrial-scale contexts involving ten thousand interactions. To mitigate the lag in capturing rapid interest drifts within the linear layers, we furthermore design Temporal-Aware Delta Network (TADN) to dynamically upweight fresh behavioral signals while effectively suppressing historical noise. Empirical results on industrial-scale datasets confirm the superiority that our model maintains linear inference speed and outperforms strong baselines, notably delivering over 8% improvement in Hit Rate for users with ultra-long sequences with great efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑