arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.11451cs.AI
收藏

SIGMA:用于序列推荐的选择性门控 Mamba

SIGMA: Selective Gated Mamba for Sequential Recommendation

City University of Hong Kong(香港城市大学);Xi’an Jiaotong University(西安交通大学);Jinan University(暨南大学);Jilin University(吉林大学)

Ziwei Liu, Qidong Liu, Yejing Wang, Wanyu Wang, Pengyue Jia, Maolin Wang, Zitao Liu, Yi Chang, Xiangyu Zhao

AI总结:

针对Mamba直接用于序列推荐存在的单向性和状态估计不稳定性问题,提出SIGMA框架,结合PF-Mamba双向架构、DS Gate和FE-GRU优化上下文与短期依赖建模,在五个数据集上优于现有模型。

摘要:

在各个领域中,序列推荐系统(SRS)凭借其识别复杂用户偏好的卓越能力,已变得至关重要。通常,SRS 利用基于 transformer 的架构来预测序列中的下一个物品。然而,这些模型固有的二次方计算复杂度往往会导致效率低下,阻碍了实时推荐的实现。Mamba 作为一项最新进展,在时间序列预测中展现出了卓越的性能,显著提升了效率和准确性。但是,将 Mamba 直接集成到 SRS 中会面临几个挑战。其固有的单向性可能会限制模型捕获用户-物品交互完整上下文的能力,而其状态估计的不稳定性可能会削弱其检测交互序列中短期模式的能力。为了克服这些问题,我们引入了一个名为选择性门控 Mamba(SIGMA)的新框架用于序列推荐。该框架利用部分翻转 Mamba(PF-Mamba)构建双向架构,专门用于改进上下文建模。此外,采用对输入敏感的密集选择性门(DS Gate)来优化方向权重,并增强 PF-Mamba 中序列信息的处理。对于短序列建模,我们还开发了特征提取 GRU(FE-GRU)以高效捕获短期依赖关系。实证结果表明,SIGMA 在五个真实世界数据集上优于当前模型。我们的实现代码可在 https://github.com/ziwliu-cityu/SIMGA 获取,以方便复现。

英文摘要:

In various domains, Sequential Recommender Systems (SRS) have become essential due to their superior capability to discern intricate user preferences. Typically, SRS utilize transformer-based architectures to forecast the subsequent item within a sequence. Nevertheless, the quadratic computational complexity inherent in these models often leads to inefficiencies, hindering the achievement of real-time recommendations. Mamba, a recent advancement, has exhibited exceptional performance in time series prediction, significantly enhancing both efficiency and accuracy. However, integrating Mamba directly into SRS poses several challenges. Its inherently unidirectional nature may constrain the model's capacity to capture the full context of user-item interactions, while its instability in state estimation can compromise its ability to detect short-term patterns within interaction sequences. To overcome these issues, we introduce a new framework named Selective Gated Mamba (SIGMA) for Sequential Recommendation. This framework leverages a Partially Flipped Mamba (PF-Mamba) to construct a bidirectional architecture specifically tailored to improve contextual modeling. Additionally, an input-sensitive Dense Selective Gate (DS Gate) is employed to optimize directional weights and enhance the processing of sequential information in PF-Mamba. For short sequence modeling, we have also developed a Feature Extract GRU (FE-GRU) to efficiently capture short-term dependencies. Empirical results indicate that SIGMA outperforms current models on five real-world datasets. Our implementation code is available at https://github.com/ziwliu-cityu/SIMGA to ease reproducibility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑