arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.02638cs.LGcs.AI
收藏

EchoMamba4Rec:融合双向状态空间模型与频谱滤波以提升序列推荐性能

EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation

BNU-HKBU United International College(北师港浸会大学);Research Center for Mathematics, Beijing Normal University(北京师范大学数学研究中心)

Yuda Wang, Xuxin He, Shengxin Zhu

AI总结:

本文提出EchoMamba4Rec,通过双向Mamba状态空间模型与频域FFT滤波及GLU门控机制,高效建模长程序列依赖,显著提升序列推荐的准确性与计算效率。

摘要:

基于历史行为预测用户偏好和序列依赖是序列推荐的核心目标。尽管基于注意力的模型在该领域已展现出有效性,但由于注意力机制固有的二次计算复杂度,它们在推理效率上常常面临困难,尤其是在处理长程行为序列时。受状态空间模型(SSMs)在控制理论中最新进展的启发——这些进展为建模和控制动态系统提供了稳健框架——我们提出了EchoMamba4Rec。控制理论强调利用SSMs通过结构化状态矩阵管理长程依赖并保持推理效率。EchoMamba4Rec在序列推荐中利用这些控制关系,并将双向处理与频域滤波相结合,以更有效地捕获用户交互数据中的复杂模式和依赖关系。我们的模型受益于状态空间模型(SSMs)的学习和并行计算能力,显著提升了计算效率和可扩展性。它包含一个双向Mamba模块,该模块融合了前向和反向Mamba组件,同时利用过去和未来交互的信息。此外,一个滤波层在频域中运行,使用可学习的快速傅里叶变换(FFT)和可学习滤波器,随后通过逆FFT来细化物品嵌入并降低噪声。我们还集成了门控线性单元(GLU)以动态控制信息流,增强模型的表达能力和训练稳定性。实验结果表明,EchoMamba显著优于现有模型,能够提供更准确和个性化的推荐。

英文摘要:

Predicting user preferences and sequential dependencies based on historical behavior is the core goal of sequential recommendation. Although attention-based models have shown effectiveness in this field, they often struggle with inference inefficiency due to the quadratic computational complexity inherent in attention mechanisms, especially with long-range behavior sequences. Drawing inspiration from the recent advancements of state space models (SSMs) in control theory, which provide a robust framework for modeling and controlling dynamic systems, we introduce EchoMamba4Rec. Control theory emphasizes the use of SSMs for managing long-range dependencies and maintaining inferential efficiency through structured state matrices. EchoMamba4Rec leverages these control relationships in sequential recommendation and integrates bi-directional processing with frequency-domain filtering to capture complex patterns and dependencies in user interaction data more effectively. Our model benefits from the ability of state space models (SSMs) to learn and perform parallel computations, significantly enhancing computational efficiency and scalability. It features a bi-directional Mamba module that incorporates both forward and reverse Mamba components, leveraging information from both past and future interactions. Additionally, a filter layer operates in the frequency domain using learnable Fast Fourier Transform (FFT) and learnable filters, followed by an inverse FFT to refine item embeddings and reduce noise. We also integrate Gate Linear Units (GLU) to dynamically control information flow, enhancing the model's expressiveness and training stability. Experimental results demonstrate that EchoMamba significantly outperforms existing models, providing more accurate and personalized recommendations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑