Graph Neural Networks (GNNs) have shown promising potential in graph representation learning. The majority of GNNs define a local message-passing mechanism, propagating information over the graph by stacking multiple layers. These methods, however, are known to suffer from two major limitations: over-squashing and poor capturing of long-range dependencies. Recently, Graph Transformers (GTs) emerged as a powerful alternative to Message-Passing Neural Networks (MPNNs). GTs, however, have quadratic computational cost, lack inductive biases on graph structures, and rely on complex Positional/Structural Encodings (SE/PE). In this paper, we show that while Transformers, complex message-passing, and SE/PE are sufficient for good performance in practice, neither is necessary. Motivated by the recent success of State Space Models (SSMs), such as Mamba, we present Graph Mamba Networks (GMNs), a general framework for a new class of GNNs based on selective SSMs. We discuss and categorize the new challenges when adapting SSMs to graph-structured data, and present four required and one optional steps to design GMNs, where we choose (1) Neighborhood Tokenization, (2) Token Ordering, (3) Architecture of Bidirectional Selective SSM Encoder, (4) Local Encoding, and dispensable (5) PE and SE. We further provide theoretical justification for the power of GMNs. Experiments demonstrate that despite much less computational cost, GMNs attain an outstanding performance in long-range, small-scale, large-scale, and heterophilic benchmark datasets.
MAPN: Enhancing Heterogeneous Sparse Graph Representation by Mamba-based Asynchronous Aggregation
School of Computer Science, Fudan University(复旦大学计算机科学学院);School of Software, Fudan University(复旦大学软件学院);Shanghai Key Laboratory of Data Science(上海数据科学重点实验室)
MOL-Mamba: Enhancing Molecular Representation with Structural & Electronic Insights
School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机与信息工程学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);Anhui University(安徽大学);University of Science and Technology of China(中国科学技术大学)
DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs
The University of Tokyo(东京大学);Institute of Science Tokyo(东京科学研究所);RIKEN Center for Advanced Intelligence Project(理化学研究所先进情报项目中心);Griffith University(格里菲斯大学)
DG-Mamba: Robust and Efficient Dynamic Graph Structure Learning with Selective State Space Models
Beihang University(北京航空航天大学);National Superior College for Engineers, Beihang University(北京航空航天大学国家卓越工程师学院);Guangxi Normal University(广西师范大学);The Third Research Institute of Ministry of Public Security(公安部第三研究所)
Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces
University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(矢量人工智能研究所);University Health Network(大学网络健康中心);Peter Munk Cardiac Centre(彼得·芒克心脏中心);Department of Laboratory Medicine and Pathobiology, University of Toronto(多伦多大学检验医学与病理生物学系);AI Hub, University Health Network(大学网络健康中心人工智能中心)
Graph Mamba Survival Analysis Based on Topology-Aware ordering
School of Mathematics and Statistics(数学与统计学学院);West China Science and Technology Innovation Harbor(西部科学与技术创新港);School of Computer Science and Technology(计算机科学与技术学院)
MSGM: A Multi-Scale Spatiotemporal Graph Mamba for EEG Emotion Recognition
School of Electronics and Communication Engineering, Sun Yat-sen University(中山大学电子与通信工程学院);College of Communication Engineering, Jilin University(吉林大学通信工程学院);School of Pharmaceutical Sciences (Shenzhen), Sun Yat-sen University(中山大学药学院(深圳));Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Generalizable Trajectory Prediction via Inverse Reinforcement Learning with Mamba-Graph Architecture
Department of Mathematics, The University of Hong Kong (HKU)(香港大学数学系);Department of Data and Systems Engineering, HKU(香港大学数据与系统工程系);Musketeers Foundation Institute of Data Science, HKU(穆斯quettes基金会数据科学研究所)
NeuroMambaLLM: Dynamic Graph Learning of fMRI Functional Connectivity in Autistic Brains Using Mamba and Language Model Reasoning
Department of Electrical and Computer Engineering, McMaster University(麦基尔大学电气与计算机工程系);Department of Biomedical Engineering, University of Southern California(南加州大学生物医学工程系);Department of Electrical and Computer Engineering, University of Rochester(罗切斯特大学电气与计算机工程系);BIOSEN Group(BIOSEN集团)
GraphMamba: An Efficient Graph Structure Learning Vision Mamba for Hyperspectral Image Classification
College of Electrical and Information Engineering, Hunan University(湖南大学电气与信息工程学院);Peng Cheng Laboratory(鹏城实验室);School of Information Engineering, Zhongnan University of Economics and Law(中南财经政法大学信息工程学院);Emergency Management Research Center, Zhongnan University of Economics and Law(中南财经政法大学应急管理研究中心)
GLADMamba: Unsupervised Graph-Level Anomaly Detection Powered by Selective State Space Model
School of Artificial Intelligence, Jilin University, Changchun, China(吉林大学人工智能学院);Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China(香港科技大学(广州));College of Computer Science and Technology, Jilin University, Changchun, China(吉林大学计算机科学与技术学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Ministry of Education, China(知识驱动的人机智能工程研究中心,教育部)
MambaLRP: Explaining Selective State Space Sequence Models
Technische Universität Berlin(柏林工业大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究所);Freie Universität Berlin(柏林自由大学);Korea University(高丽大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Google DeepMind(谷歌DeepMind)
Tsinghua University(清华大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);Shanghai Jiao Tong University(上海交通大学);The Chinese University of Hong Kong(香港中文大学);Shenzhen University(深圳大学);Peng Cheng Laboratory(鹏城实验室);ETH Zürich(苏黎世联邦理工学院)
Efficient Unstructured Pruning of Mamba State-Space Models for Resource-Constrained Environments
Department of Computer Science, Iowa State University(计算机科学系,爱荷华州立大学);Department of Civil, Construction and Environmental Engineering, Iowa State University(土木、建设与环境工程系,爱荷华州立大学)
Mammo-Mamba: A Hybrid State-Space and Transformer Architecture with Sequential Mixture of Experts for Multi-View Mammography
Concordia Institute for Information Systems Engineering (CIISE)(康科德信息系统工程研究所);Concordia University(康科德大学);Thomas Jefferson University Hospital(托马斯·杰斐逊大学医院)
The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity
The University of Chicago(芝加哥大学);University of New South Wales(新南威尔士大学);The University of Hong Kong(香港大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);The Simons Institute for the Theory of Computing at UC Berkeley(加州大学伯克利分校西蒙斯理论计算研究所)
Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking
School of Computer Science and Technology, Anhui University(安徽大学计算机科学与技术学院);School of Electronic and Information Engineering, Anhui University(安徽大学电子与信息工程学院)