Large pre-trained models have achieved outstanding results in sequence modeling. The Transformer block and its attention mechanism have been the main drivers of the success of these models. Recently, alternative architectures, such as Selective Structured State Space Models (SSMs), have been proposed to address the inefficiencies of Transformers. This paper explores the compression of SSM-based models, particularly Mamba and its hybrids. We study the sensitivity of these models to the removal of selected components at different granularities to reduce the model size and computational overhead, thus improving their efficiency while maintaining accuracy. The proposed solutions, collectively referred to as Mamba-Shedder, achieve a speedup of up to 1.4x during inference, demonstrating that model efficiency can be improved by eliminating several redundancies with minimal impact on the overall model performance. The code is available at https://github.com/IntelLabs/Hardware-Aware-Automated-Machine-Learning.
Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces
University of Toronto（多伦多大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）；University Health Network（大学网络健康中心）；Peter Munk Cardiac Centre（彼得·芒克心脏中心）；Department of Laboratory Medicine and Pathobiology, University of Toronto（多伦多大学检验医学与病理生物学系）；AI Hub, University Health Network（大学网络健康中心人工智能中心）
DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space Models
Beihang University（北京航空航天大学）；National Superior College for Engineers, Beihang University（北京航空航天大学国家卓越工程师学院）；Guangxi Normal University（广西师范大学）；The Third Research Institute of Ministry of Public Security（公安部第三研究所）
The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
The University of Chicago（芝加哥大学）；University of New South Wales（新南威尔士大学）；The University of Hong Kong（香港大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；The Simons Institute for the Theory of Computing at UC Berkeley（加州大学伯克利分校西蒙斯理论计算研究所）
SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
该研究提出 SSUMamba，通过六种空间-光谱连续扫描顺序、双向 SSM 与 3D 卷积高效建模高光谱图像的长程及局部依赖，在降低每批内存消耗的同时提升去噪性能。
Block-Biased Mamba for Long-Range Sequence Processing
Center for Applied Mathematics（应用数学中心）；Cornell University（康奈尔大学）；Lawrence Berkeley National Laboratory（伯克利国家实验室）；International Computer Science Institute（国际计算机科学研究所）
Hybrid Mamba-Attention Neural Architecture for Channel Estimation
Institute for Imaging, Data and Communications, School of Engineering, University of Edinburgh（影像、数据与通信研究所，工程学院，爱丁堡大学）；James Watt School of Engineering, University of Glasgow（詹姆斯·瓦特工程学院，格拉斯哥大学）；National Mobile Communications Research Laboratory, Southeast University（国家移动通信研究中心，东南大学）；Purple Mountain Laboratories, Nanjing（紫金山实验室，南京）；Department of Electrical and Electronic Engineering, The University of Hong Kong（电气与电子工程系，香港大学）；Department of Electrical and Electronic Engineering, University of Manchester（电气与电子工程系，曼彻斯特大学）
DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs
The University of Tokyo（东京大学）；Institute of Science Tokyo（东京科学研究所）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进情报项目中心）；Griffith University（格里菲斯大学）
Mammo-Mamba: A Hybrid State-Space and Transformer Architecture with Sequential Mixture of Experts for Multi-View Mammography
Concordia Institute for Information Systems Engineering (CIISE)（康科德信息系统工程研究所）；Concordia University（康科德大学）；Thomas Jefferson University Hospital（托马斯·杰斐逊大学医院）
Repetitive Contrastive Learning Enhances Mamba's Selectivity in Time Series Prediction
School of Intelligence Science and Technology, Peking University, Beijing（智能科学与技术学院，北京大学，北京）；Computational Intelligence Laboratory（计算智能实验室）；Institute for Artificial Intelligence（人工智能研究院）；State Key Laboratory of General Artificial Intelligence, Peking University, Beijing（通用人工智能国家重点实验室，北京大学，北京）
MambaLRP: Explaining Selective State Space Sequence Models
Technische Universität Berlin（柏林工业大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；Freie Universität Berlin（柏林自由大学）；Korea University（高丽大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Google DeepMind（谷歌DeepMind）
GLADMamba: Unsupervised Graph-Level Anomaly Detection Powered by Selective State Space Model
School of Artificial Intelligence, Jilin University, Changchun, China（吉林大学人工智能学院）；Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China（香港科技大学（广州））；College of Computer Science and Technology, Jilin University, Changchun, China（吉林大学计算机科学与技术学院）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Ministry of Education, China（知识驱动的人机智能工程研究中心，教育部）
