arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.21052cs.IRcs.AIcs.LG
收藏

基于位置感知的序列注意力机制用于准确的下一项推荐

Position-Aware Sequential Attention for Accurate Next Item Recommendations

Skolkovo Institute of Science and Technology(斯克尔科夫科学与技术研究所);AIRI

Timur Nabiev, Evgeny Frolov

AI总结:

本文提出位置核化的自注意力机制,用于提升序列推荐的准确性。

摘要:

顺序自注意力模型通常依赖于加性位置嵌入,这些嵌入在输入时将位置信息注入项目表示中。在没有位置信号的情况下,注意力块在序列位置上是排列等变的,因此除了因果掩码之外没有内在的时间顺序概念。我们认为加性位置嵌入使注意力机制仅对序列顺序表面敏感:位置信息与项目嵌入语义纠缠,深度架构中传播薄弱,并限制了捕捉丰富序列模式的能力。为了解决这些限制,我们引入了一种核化的自注意力机制,其中可学习的位置核在纯位置空间中操作,与语义相似性分离,并直接调节注意力权重。当应用于每个注意力块时,这种核能够实现自适应的多尺度序列建模。在标准的下一项预测基准测试中,我们的位置核注意力在强竞争基线之上 consistently 提高。

英文摘要:

Sequential self-attention models usually rely on additive positional embeddings, which inject positional information into item representations at the input. In the absence of positional signals, the attention block is permutation-equivariant over sequence positions and thus has no intrinsic notion of temporal order beyond causal masking. We argue that additive positional embeddings make the attention mechanism only superficially sensitive to sequence order: positional information is entangled with item embedding semantics, propagates weakly in deep architectures, and limits the ability to capture rich sequential patterns. To address these limitations, we introduce a kernelized self-attention mechanism, where a learnable positional kernel operates purely in the position space, disentangled from semantic similarity, and directly modulates attention weights. When applied per attention block, this kernel enables adaptive multi-scale sequential modeling. Experiments on standard next-item prediction benchmarks show that our positional kernel attention consistently improves over strong competing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑