U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation
Peter Munk Cardiac Centre, University Health Network(彼得·蒙克心脏中心,大学健康网络);University of Toronto(多伦多大学);Vector Institute for Artificial Intelligence(向量人工智能研究所)
卷积神经网络(CNNs)和Transformers一直是生物医学图像分割最受欢迎的架构,但由于固有的局部性或计算复杂性,它们在处理长距离依赖方面的能力都有限。为了应对这一挑战,我们引入了U-Mamba,一种用于生物医学图像分割的通用网络。受State Space Sequence Models(SSMs)启发,这是一种以处理长序列的强大能力而闻名的深度序列模型新家族,我们设计了一个混合CNN-SSM块,将卷积层的局部特征提取能力与SSMs捕获长距离依赖的能力相结合。此外,U-Mamba具有自配置机制,使其能够无需人工干预即可自动适应各种数据集。我们在四项不同的任务上进行了广泛实验,包括CT和MR图像中的3D腹部器官分割、内窥镜图像中的器械分割以及显微镜图像中的细胞分割。结果显示,U-Mamba在所有任务中均优于基于CNN和基于Transformer的最先进分割网络。这为生物医学图像分析中的高效长距离依赖建模开辟了新途径。代码、模型和数据已在https://wanglab.ai/u-mamba.html公开提供。
英文摘要:
Convolutional Neural Networks (CNNs) and Transformers have been the most popular architectures for biomedical image segmentation, but both of them have limited ability to handle long-range dependencies because of inherent locality or computational complexity. To address this challenge, we introduce U-Mamba, a general-purpose network for biomedical image segmentation. Inspired by the State Space Sequence Models (SSMs), a new family of deep sequence models known for their strong capability in handling long sequences, we design a hybrid CNN-SSM block that integrates the local feature extraction power of convolutional layers with the abilities of SSMs for capturing the long-range dependency. Moreover, U-Mamba enjoys a self-configuring mechanism, allowing it to automatically adapt to various datasets without manual intervention. We conduct extensive experiments on four diverse tasks, including the 3D abdominal organ segmentation in CT and MR images, instrument segmentation in endoscopy images, and cell segmentation in microscopy images. The results reveal that U-Mamba outperforms state-of-the-art CNN-based and Transformer-based segmentation networks across all tasks. This opens new avenues for efficient long-range dependency modeling in biomedical image analysis. The code, models, and data are publicly available at https://wanglab.ai/u-mamba.html.
MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));HKUST(GZ) - SmartMore Joint Lab(香港科技大学(广州)与视博医疗联合实验室);The Hong Kong University of Science and Technology(香港科技大学);Southern University of Science and Technology(南方科技大学);SmartMore(视博医疗)
Graph Mamba Survival Analysis Based on Topology-Aware ordering
School of Mathematics and Statistics(数学与统计学学院);West China Science and Technology Innovation Harbor(西部科学与技术创新港);School of Computer Science and Technology(计算机科学与技术学院)
SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoising
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
该研究提出 SSUMamba,通过六种空间-光谱连续扫描顺序、双向 SSM 与 3D 卷积高效建模高光谱图像的长程及局部依赖,在降低每批内存消耗的同时提升去噪性能。
MambaEviScrib: Mamba and Evidence-Guided Consistency Enhance CNN Robustness for Scribble-Based Weakly Supervised Ultrasound Image Segmentation
School of Communication and Information Engineering, Shanghai University(上海大学通信与信息工程学院);The SMART (Smart Medicine and AI-based Radiology Technology) Lab, Shanghai Institute for Advanced Communication and Data Science, Shanghai University(上海大学先进通信与数据科学研究院智能医学与AI放射技术实验室);School of Health Science and Engineering, University of Shanghai for Science and Technology(上海科技大学健康科学与工程学院);Department of Pediatric Cardiology, Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine(上海交通大学医学院儿童心脏科部);School of Medical Instruments, Shanghai University of Medicine and Health Sciences(医学仪器学院,上海医学院与健康科学大学)
LKM-UNet: Large Kernel Vision Mamba UNet for Medical Image Segmentation
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Institute of Wenzhou, Zhejiang University(浙江大学温州研究院);The Second Affiliated Hospital, Zhejiang University School of Medicine(浙江大学医学院附属第二医院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Notre Dame(圣母大学)
MS-UMamba: An Improved Vision Mamba Unet for Fetal Abdominal Medical Image Segmentation
Guangxi Key Laboratory of Machine Vision and Intelligent Control(广西机器视觉与智能控制重点实验室);Wuzhou University(梧州大学);College of Electronical and Information Engineering(电子信息工程学院);Wuzhou Medical College(梧州医学院);School of Microelectronics(微电子学院);Southern Medical University(南方医科大学)
VIMCAN: Visual-Inertial 3D Human Pose Estimation with Hybrid Mamba-Cross-Attention Network
Beihang University(北航);Peng Cheng Laboratory(鹏城实验室);Capital University of Physical Education and Sports(首都体育学院);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院)
CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息与电子工程学院);School of Aeronautics and Astronautics, Zhejiang University(浙江大学航空航天学院);College of Computer Science, Zhejiang University(浙江大学计算机学院)
ShapeMamba-EM: Fine-Tuning Foundation Model with Local Shape Descriptors and Mamba Blocks for 3D EM Image Segmentation
National Engineering Research Center of Visual Technology(国家视觉工程研究中心);School of Computer Science, Peking University(北京大学计算机学院);State Key Laboratory of Multimedia Information Processing(多媒体信息处理国家重点实验室);National Biomedical Imaging Center, Peking University(北京大学生物医学成像中心);Peng Cheng Laboratory(鹏城实验室)
ACM-UNet: Adaptive Integration of CNNs and Mamba for Efficient Medical Image Segmentation
Wuhan University of Technology(武汉理工大学);Chinese Academy of Medical Sciences and Peking Union Medical College(中国医学科学院北京协和医学院);Zhongnan Hospital of Wuhan University(武汉大学中南医院)
Chemical Language Models for Natural Products: A State-Space Model Approach
Spoken Language Systems, Saarland Informatics Campus, Saarland University, Germany(萨尔兰大学语音语言系统,萨尔兰信息学校区);Data Driven Drug Design, Center for Bioinformatics, Saarland University, Germany(数据驱动药物设计,生物信息学中心,萨尔兰大学)
SegMaFormer: A Hybrid State-Space and Transformer Model for Efficient Segmentation
Department of Software Engineering, Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT)(胡志明市技术大学软件工程系);Vietnam National University Ho Chi Minh City(越南胡志明市国家大学)
DyGMamba: Efficiently Modeling Long-Term Temporal Dependency on Continuous-Time Dynamic Graphs with State Space Models
University of Cambridge(剑桥大学);Technical University of Munich(慕尼黑技术大学);University of Oxford(牛津大学);University of Stuttgart(斯图加特大学);LMU Munich(慕尼黑大学);Munich Center for Machine Learning(慕尼黑机器学习中心)
xLSTM-UNet can be an Effective 2D & 3D Medical Image Segmentation Backbone with Vision-LSTM (ViL) better than its Mamba Counterpart
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);KOKONI, Moxin (Huzhou) Tech. Co., LTD.(墨芯(湖州)科技有限公司KOKONI);School of Information Engineering, Huzhou University(湖州师范学院信息工程学院);Information Systems Technology and Design Pillar, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计 pillar);School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院)
SpectralMamba-UNet: Frequency-Disentangled State Space Modeling for Texture-Structure Consistent Medical Image Segmentation
College of Computer Science(计算机科学学院);Sichuan Normal University(四川师范大学);Zhejiang University(浙江大学);Ant Group(蚂蚁集团);State Key Laboratory of Public Big Data(公共大数据国家重点实验室);Guizhou University(贵州大学)
A Multi-scale Linear-time Encoder for Whole-Slide Image Analysis
Department of Electrical and Computer Engineering, University of Houston(电子与计算机工程系,休斯顿大学);Department of Information Science Technology, University of Houston(信息科学与技术系,休斯顿大学);Department of Neurology, College of Medicine, University of Florida(神经病学系,医学学院,佛罗里达大学)
A Novel Hybrid Approach for Retinal Vessel Segmentation with Dynamic Long-Range Dependency and Multi-Scale Retinal Edge Fusion Enhancement
School of Artificial Intelligence(人工智能学院);Hebei University of Technology(河北工业大学);NHC Key Lab of Hormones and Development and Tianjin Key Lab of Metabolic Diseases(国家卫生健康委员会激素与发育重点实验室和天津代谢疾病重点实验室);Tianjin Medical University Chu Hsien-I Memorial Hospital & Institute of Endocrinology(天津医科大学楚生一纪念医院及内分泌研究所)
DH-Mamba: Exploring Dual-domain Hierarchical State Space Models for MRI Reconstruction
School of Basic Medical Science, Fudan University(复旦大学基础医学院);Academy of Engineering and Technology, Fudan University(复旦大学工程与技术学院);Shandong Computer Science Center (National Supercomputer Center in Jinan)(山东计算机科学中心(国家济南超级计算中心))
ClinicalFMamba: Advancing Clinical Assessment using Mamba-based Multimodal Neuroimaging Fusion
TD Bank Group, Toronto, Canada(TD银行集团,加拿大);Department of Computer Science, University of Toronto, Toronto, Canada(多伦多大学计算机科学系,加拿大);Department of Medical Imaging, University of Toronto, Toronto, Canada(多伦多大学医学影像系,加拿大)