Selective state space models (SSMs), such as Mamba, highly excel at capturing long-range dependencies in 1D sequential data, while their applications to 2D vision tasks still face challenges. Current visual SSMs often convert images into 1D sequences and employ various scanning patterns to incorporate local spatial dependencies. However, these methods are limited in effectively capturing the complex image spatial structures and the increased computational cost caused by the lengthened scanning paths. To address these limitations, we propose Spatial-Mamba, a novel approach that establishes neighborhood connectivity directly in the state space. Instead of relying solely on sequential state transitions, we introduce a structure-aware state fusion equation, which leverages dilated convolutions to capture image spatial structural dependencies, significantly enhancing the flow of visual contextual information. Spatial-Mamba proceeds in three stages: initial state computation in a unidirectional scan, spatial context acquisition through structure-aware state fusion, and final state computation using the observation equation. Our theoretical analysis shows that Spatial-Mamba unifies the original Mamba and linear attention under the same matrix multiplication framework, providing a deeper understanding of our method. Experimental results demonstrate that Spatial-Mamba, even with a single scan, attains or surpasses the state-of-the-art SSM-based models in image classification, detection and segmentation. Source codes and trained models can be found at https://github.com/EdwardChasel/Spatial-Mamba.
UAVD-Mamba: Deformable Token Fusion Vision Mamba for Multimodal UAV Detection
College of Mechanical and Vehicle Engineering, Hunan University(湖南大学机械与车辆工程学院);School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院)
MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition
Computer Vision and Pattern Recognition Laboratory, Department of Computational Engineering, Lappeenranta-Lahti University of Technology LUT(计算机视觉与模式识别实验室,计算工程系,拉佩兰塔-拉赫蒂理工大学);School of Electrical and Information Engineering, Tianjin University(电气与信息工程学院,天津大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电子与电气工程学院,南洋理工大学);Faculty of Information Technology, Brno University of Technology(信息技术学院,布拉格技术大学)
MambaDSF: Multi-Scale SSM with Dilated Feature Fusion for Sonar Small Target Detection
School of Information Science and Engineering, Ocean University of China(中国海洋大学信息科学与工程学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion
University of Science and Technology of China(中国科学技术大学);Institute of Intelligent Machines(智能机器研究所);Hefei Institutes of Physical Science, Chinese Academy of Sciences(中国科学院合肥物质科学研究院);Intelligent Agriculture Engineering Laboratory of Anhui Province, Institute of Intelligent Machines(安徽省智能农业工程实验室,智能机器研究所)
MambaTrans: Multimodal Fusion Image Translation via Large Language Model Priors for Downstream Visual Tasks
School of Physics and Optoelectronic Engineering(物理与光电工程学院);Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology(粤港澳联合智能微纳光电技术实验室);School of Information Engineering and Automation(信息工程与自动化学院)
本文提出MambaTrans,一种多模态融合图像模态翻译器,利用多模态大语言模型描述和语义分割掩码作为输入,通过Multi-Model State Space Block捕获跨模态依赖,在不调整预训练模型参数的情况下提升下游视觉任务性能。
Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion
School of Future Technology, School of Artificial Intelligence, Dalian University of Technology(大连理工大学未来技术学院、人工智能学院);Affiliated Zhongshan Hospital of Dalian University(大连大学附属中山医院);Central Hospital of Dalian University of Technology(大连理工大学中心医院);School of Information and Communication Engineering, Dalian University of Technology(大连理工大学信息与通信工程学院)
Mamba-Driven Topology Fusion for Monocular 3D Human Pose Estimation
College of Sciences, Northeastern University(东北大学理学院);Key Laboratory of Differential Equations and Their Applications, Northeastern University(东北大学偏微分方程及其应用重点实验室);Nanjing University of Science and Technology(南京理工大学)
D2-Mamba: Dual-Scale Fusion and Dual-Path Scanning with SSMs for Shadow Removal
School of Artificial Intelligence, Hebei University of Technology(人工智能学院,河北工业大学);School of Electrical and Computer Engineering, University of Texas at Austin(电气与计算机工程学院,德克萨斯大学奥斯汀分校);School of Health Sciences and Biomedical Engineering, Hebei University of Technology(健康科学与生物医学工程学院,河北工业大学)
MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
Hangzhou City University(杭州城市大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Hangzhou Normal University(杭州师范大学);Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance(浙江省实时智能安防技术工程研究中心)
CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration
School of Physics(物理学院);Optoelectronic Engineering, Foshan University, Foshan 528225, China(光学电子工程学院,佛山大学,佛山528225,中国);Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China(粤港澳联合智能微纳光电子技术实验室,佛山528225,中国);School of Mechanical Electronic(机械电子学院);Information Engineering, China University of Mining and Technology, Beijing 100083, China(信息工程学院,中国矿业大学,北京100083,中国)
Laplace-Mamba: Laplace Frequency Prior-Guided Mamba-CNN Fusion Network for Image Dehazing
School of Computer Science and Technology, Anhui University of Technology(安徽理工大学计算机科学与技术学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);College of Artificial Intelligence, Taiyuan University of Technology(太原科技大学人工智能学院)
Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies
Univ. of Buenos Aires. Faculty of Exact and Natural Sciences. Dept. of Computer Science (DC)(布宜诺斯艾利斯大学。精确与自然科学学院。计算机科学系);CONICET-Univ. of Buenos Aires. Institute of Computer Sciences (ICC)(布宜诺斯艾利斯大学CONICET联合体。计算机科学研究所)
XFMamba: Cross-Fusion Mamba for Multi-View Medical Image Classification
Digital Environment Research Institute (DERI), Queen Mary University of London(伦敦大学玛丽女王学院数字环境研究所(DERI));School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院);Blizard Institute - Faculty of Medicine and Dentistry, Queen Mary University of London(伦敦大学玛丽女王学院医学与牙科学院布利扎德研究所);University of Cambridge(剑桥大学)
Mamba Based Feature Extraction And Adaptive Multilevel Feature Fusion For 3D Tumor Segmentation From Multi-modal Medical Image
School of Computer Science and Engineering, Central South University, Changsha, 410083, China(计算机科学与工程学院,中南大学,长沙,410083,中国);Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Central South University, Changsha, 410083, China(机器视觉与智能医学工程研究中心,中南大学,长沙,410083,中国);Department of Radiation Oncology, Washington University in St. Louis, USA(放射肿瘤科,华盛顿大学圣路易斯分校,美国);Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France(核医学科,亨利·贝克勒尔癌症中心,鲁昂,法国);University of Rouen-Normandy, AMIS - QuantIF UR 4108, F-76000, Rouen, France(鲁昂-诺曼底大学,AMIS - QuantIF UR 4108,法国,F-76000,鲁昂,法国)
ME-Mamba: Multi-Expert Mamba with Efficient Knowledge Capture and Fusion for Multimodal Survival Analysis
Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai 200032, China(复旦大学基础医学学院数字医学研究中心);Shanghai Key Lab of Medical Image Computing(上海医学图像计算重点实验室)
Tree-Mamba: A Tree-Aware Mamba for Underwater Monocular Depth Estimation
Key Laboratory of Knowledge Automation for Industrial Processes, Ministry of Education(知识自动化工业过程关键实验室,教育部);School of Automation and Electrical Engineering, University of Science and Technology Beijing(北京科技大学自动化与电气工程学院);School of Ophthalmology and Optometry, Wenzhou Medical University(温州医科大学眼视光学院);Department of Automation, Tsinghua University(清华大学自动化系);Deepinfar Ocean Technology Inc.(深海远航技术有限公司);School of Data Science, Lingnan University(岭南大学数据科学学院);College of Computer Science, Nankai University(南开大学计算机学院)
MM-DETR: An Efficient Multimodal Detection Transformer with Mamba-Driven Dual-Granularity Fusion and Frequency-Aware Modality Adapters
School of Information and Electronics, Beijing Institute of Technology(信息与电子学院,北京理工大学);Beijing Institute of Technology Chongqing Innovation Center(北京理工大学重庆创新中心);National Key Laboratory for Space-Born Intelligent Information Processing(空间智能信息处理国家级重点实验室);School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学)
AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology(大连理工大学信息与通信工程学院);School of Innovation and Entrepreneurship, Dalian University of Technology(大连理工大学创新创业学院);School of Artificial Intelligence, Dalian University of Technology(大连理工大学人工智能学院)
RemoteDet-Mamba: A Hybrid Mamba-CNN Network for Multi-modal Object Detection in Remote Sensing Images
School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China(计算机科学学院,北京邮电大学);School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, China(电子工程学院,北京邮电大学)
MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), CASIA(多模态人工智能系统国家重点实验室(MAIS),中国科学院自动化所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Anyverse Intelligence;Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information(北京多模态信息超级智能安全重点实验室);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)
CS-MUNet: A Channel-Spatial Dual-Stream Mamba Network for Multi-Organ Segmentation
School of Software Engineering(软件工程学院);The Chinese University of Hong Kong(香港中文大学);Yunnan Provincial Key Laboratory of Software Engineering(云南省软件工程重点实验室)
MambaVSR: Content-Aware Scanning State Space Model for Video Super-Resolution
Institute of Information Science, Beijing Jiaotong University(北京交通大学信息科学学院);Visual Intelligence + X International Cooperation Joint Laboratory of MOE, Beijing Jiaotong University(教育部视觉智能+X国际合作联合实验室);School of Computer and Communication Engineering, University of Science and Technology Beijing(北京科技大学计算机与通信工程学院)
DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University(复旦大学脑智能科学与技术研究院);School of Computer Science, Fudan University(复旦大学计算机科学技术学院)
Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba
School of Communication and Information Engineering, Shanghai University(上海大学通信与信息工程学院);College of Electronic Science, Air Force Aviation University(空军航空大学电子科学学院);Intelligent Visual Information Perception Laboratory, Institute of Advanced Technology, Nanjing University of Posts and Telecommunications(南京邮电大学先进技术研究院智能视觉信息感知实验室);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)