单图像超分辨率(SR)长期以来一直是计算机视觉领域的一项挑战。尽管深度学习的出现催生了众多旨在解决这一长期问题的方法,但当前方法在建模长序列信息方面仍面临困难,从而限制了其有效捕获全局像素交互的能力。为应对这一挑战并获得更优的 SR 结果,我们提出了 Mamba 逐像素序列交互网络(MPSI),旨在加强信息长程连接的建立,尤其关注逐像素序列交互。我们提出 Channel-Mamba Block(CMB),通过有效建模长序列信息来捕获全面的像素交互信息。此外,在现有 SR 方法中,仍存在忽视前序层所提取特征的问题,导致有价值的特征信息丢失。尽管某些现有模型努力保留这些特征,但它们往往难以在所有层之间建立连接。为克服这一局限,MPSI 引入了 Mamba channel recursion module(MCRM),最大限度保留来自早期层的有价值特征信息,从而促进从多个层级获取像素序列交互信息。通过大量实验,我们证明 MPSI 在图像重建结果方面优于现有超分辨率方法,达到了最先进性能。
英文摘要:
Single image super-resolution (SR) has long posed a challenge in the field of computer vision. While the advent of deep learning has led to the emergence of numerous methods aimed at tackling this persistent issue, the current methodologies still encounter challenges in modeling long sequence information, leading to limitations in effectively capturing the global pixel interactions. To tackle this challenge and achieve superior SR outcomes, we propose the Mamba pixel-wise sequential interaction network (MPSI), aimed at enhancing the establishment of long-range connections of information, particularly focusing on pixel-wise sequential interaction. We propose the Channel-Mamba Block (CMB) to capture comprehensive pixel interaction information by effectively modeling long sequence information. Moreover, in the existing SR methodologies, there persists the issue of the neglect of features extracted by preceding layers, leading to the loss of valuable feature information. While certain existing models strive to preserve these features, they frequently encounter difficulty in establishing connections across all layers. To overcome this limitation, MPSI introduces the Mamba channel recursion module (MCRM), which maximizes the retention of valuable feature information from early layers, thereby facilitating the acquisition of pixel sequence interaction information from multiple-level layers. Through extensive experimentation, we demonstrate that MPSI outperforms existing super-resolution methods in terms of image reconstruction results, attaining state-of-the-art performance.
Traffic Image Restoration under Adverse Weather via Frequency-Aware Mamba
School of Communication and Information Engineering, Shanghai University(上海大学通信与信息工程学院);College of Electronic Science, Air Force Aviation University(空军航空大学电子科学学院);Intelligent Visual Information Perception Laboratory, Institute of Advanced Technology, Nanjing University of Posts and Telecommunications(南京邮电大学先进技术研究院智能视觉信息感知实验室);School of Computer Science and Technology, East China Normal University(华东师范大学计算机科学与技术学院)
Wavelet-based Mamba with Fourier Adjustment for Low-light Image Enhancement
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology(上海理工大学光电信息与计算机工程学院);School of computer science and artificial intelligence, Liaoning Normal University(辽宁师范大学计算机与人工智能学院);College of computer science and technology, Jilin University(吉林大学计算机科学与技术学院);School of Computer, National University of Dense Technology(国防科技大学计算机学院)
M2IR: Proactive All-in-One Image Restoration via Mamba-style Modulation and Mixture-of-Experts
School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);School of Computer Science and Technology, Anhui University of Technology(安徽工业大学计算机科学与技术学院);Shenzhen Ubiquitous Data Enabling Key Laboratory, Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院深圳 ubiquitous 数据赋能重点实验室);College of Artificial Intelligence, Taiyuan University of Technology(太原科技大学人工智能学院)
Semi-LLIE: Semi-supervised Contrastive Learning with Mamba-based Low-light Image Enhancement
Northwestern Polytechnical University(西北工业大学);Science and Technology on Electromechanical Dynamic Control Laboratory(机电动态控制重点实验室);Key Laboratory of Intelligent Interaction and Applications (Northwestern Polytechnical University), Ministry of Industry and Information Technology(工业和信息化部智能交互与应用重点实验室(西北工业大学))
HSRMamba: Contextual Spatial-Spectral State Space Model for Single Image Hyperspectral Super-Resolution
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University(国家多媒体软件工程研究中心,计算机学院,武汉大学);Aerospace Information Research Institute, Henan Academy of Sciences(航天信息研究所,河南科学院)
ECP-Mamba: An Efficient Multi-scale Self-supervised Contrastive Learning Method with State Space Model for PolSAR Image Classification
School of Information and Communications Engineering, Xi’an Jiaotong University(西安交通大学信息与通信工程学院);Department of Mathematics and Fundamental Research, Peng Cheng Laboratory(鹏城实验室数学与基础研究部);School of Mathematics and Statistics, Xi’an Jiaotong University(西安交通大学数学与统计学院)
3DSS-Mamba: 3D-Spectral-Spatial Mamba for Hyperspectral Image Classification
Nanjing University of Information Science and Technology(南京信息工程大学);Institute of Optics and Electronics(光学电子研究所);China University of Geosciences(中国地质大学);University of Extremadura(埃斯特雷马杜拉大学)
MambaMorph: a Mamba-based Framework for Medical MR-CT Deformable Registration
Image Processing Center, Beihang University, Beijing, China(北京航空航天大学图像处理中心);Robotics Institute, Beihang University, Beijing, China(北京航空航天大学机器人研究所);Tongji Hospital, Huazhong University of Science and Technology, Wuhan, China(华中科技大学同济医院)
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment
Peng Cheng Laboratory(鹏城实验室);University of Chinese Academy of Sciences(中国科学院大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Sun Yat-sen University(中山大学)
MambaHash: Visual State Space Deep Hashing Model for Large-Scale Image Retrieval
School of Computer Science, Inner Mongolia University, Provincial Key Laboratory of Mongolian Information Processing Technology, National and Local Joint Engineering Research Center of Mongolian Information Processing(内蒙古大学计算机学院、蒙古信息处理技术省级重点实验室、蒙古信息处理国家与地方联合工程研究中心)
{S$^3$-Mamba}: Small-Size-Sensitive Mamba for Lesion Segmentation
Shenzhen University(深圳大学);University of Nottingham Ningbo China(宁波诺丁汉大学);Guangxi Medical University(广西医科大学);City University of Hong Kong(香港城市大学);University of Nottingham(诺丁汉大学)
Mamba-Sea: A Mamba-based Framework with Global-to-Local Sequence Augmentation for Generalizable Medical Image Segmentation
Shanghai Jiao Tong University School of Medicine(上海交通大学医学院);National Institute of Healthcare Data Science(健康数据科学国家研究院);State Key Laboratory for Novel Software Technology(新型软件技术国家重点实验室);School of Computer Science and Engineering(计算机科学与工程学院);Key Lab of Computer Network and Information Integration(计算机网络与信息集成重点实验室);School of Electrical and Information Engineering(电气与信息工程学院);The University of Sydney(悉尼大学)
PE-Mamba: Bidirectional Selective Layer Aggregation for AI-Generated Image Detection
College of Innovation & Technology, University of Michigan-Flint(密歇根大学弗林特分校创新与技术学院);School of Electronics and Information Engineering, Korea Aerospace University(韩国航空大学电子与信息工程学院)
A Lightweight and Effective Image Tampering Localization Network with Vision Mamba
School of Computer and Cyber Sciences, Communication University of China(中国传媒大学计算机与网络空间科学学院);State Key Laboratory of Media Convergence and Communication, Communication University of China(中国传媒大学媒体融合与传播国家重点实验室);School of Information Engineering, Changsha Medical University(长沙医学院信息工程学院)
BiSegMamba: Efficient Bidirectional Tri-Oriented Mamba for 3D Medical Image Segmentation
School of Computer Science and Engineering, Beihang University(北航计算机科学与工程学院);State Key Laboratory of Virtual Reality Technology and Systems, Beihang University(北航虚拟现实技术与系统国家重点实验室)
A Hybrid Transformer-Mamba Network for Single Image Deraining
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络空间科学与技术学院);Yunnan Normal University(云南师范大学)
CMAMBADEPTH: Self-supervised Monocular Depth Estimation with Channel Mamba and Hybrid Attention
Harbin Engineering University(哈尔滨工程大学);Key Laboratory of Advanced Marine Communication and Information Technology(先进海洋通信与信息技术重点实验室);University of Toronto(多伦多大学);Kanagawa University(神奈川大学)
Scan Clusters, Not Pixels: A Cluster-Centric Paradigm for Efficient Ultra-high-definition Image Restoration
National University of Defense Technology(国防科技大学);HKUST(GZ)(香港理工大学);Qilu University of Technology(青岛科技大学);Technical University of Munich(慕尼黑技术大学);TeleAI, China Telecom(TeleAI,中国电信);Beihang University(北航)
Selective Structured State Space for Multispectral-fused Small Target Detection
Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);National Space Science Center, Chinese Academy of Sciences(中国科学院国家空间科学中心);School of Computer Science and Technology, University of Chinese Academy of Sciences(中国科学院大学计算机科学与技术学院);School of Astronomy and Space Science, University of Chinese Academy of Sciences(中国科学院大学天文与空间科学学院);School of Computing, National University of Singapore(新加坡国立大学计算机学院)
3D Segment Anything Model with Visual Mamba for Diagnosing Placenta Accreta Spectrum
Department of Obstetrics and Gynecology, The Third Affiliated Hospital, Guangzhou Medical University(妇产科系,广州医科大学第三附属医院);Department of Obstetrics, Guangzhou Women and Children’s Medical Center, Guangzhou Medical University(妇产科,广州妇女儿童医疗中心,广州医科大学);Department of Radiology, The Third Affiliated Hospital, Guangzhou Medical University(放射科,广州医科大学第三附属医院);School of Future Technology, Dalian University of Technology(未来技术学院,大连理工大学)
SpectralMamba: Efficient Mamba for Hyperspectral Image Classification
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);School of Mathematics and Statistics, Southeast University(东南大学数学学院);Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK(格勒诺布尔阿尔卑斯大学、法国国家信息与自动化研究所、法国国家科学研究中心、格勒诺布尔理工学院、约瑟夫·傅里叶大学)
Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);University of Chinese Academy of Sciences(中国科学院大学);Guangzhou University(广州大学)
SAMamba: Adaptive State Space Modeling with Hierarchical Vision for Infrared Small Target Detection
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)(教育部计算电力网络与信息安全重点实验室,山东计算机科学中心(济南国家超算中心),齐鲁工业大学(山东省科学院));The State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science(山东省计算机网络重点实验室,山东省计算机科学基础研究中心)