Mamba与Vision Mamba(Vim,视觉Mamba)模型已展现出作为Transformer架构基础方法替代方案的潜力。本研究提出Fast Mamba for Vision(Famba-V,快速视觉Mamba),这是一种提升Vim模型训练效率的跨层令牌融合技术。Famba-V的核心思路是基于一套跨层策略,识别并融合不同Vim层中的相似令牌,而非像现有研究提出的那样在所有层统一应用令牌融合。我们在CIFAR-100数据集上评估了Famba-V的性能。结果表明,Famba-V能够通过减少训练时间和训练期间的峰值内存占用,提升Vim模型的训练效率。此外,所提出的跨层策略使Famba-V实现了更优的准确率-效率权衡。这些结果共同证明,Famba-V是一种颇具前景的Vim模型效率增强技术。
英文摘要:
Mamba and Vision Mamba (Vim) models have shown their potential as an alternative to methods based on Transformer architecture. This work introduces Fast Mamba for Vision (Famba-V), a cross-layer token fusion technique to enhance the training efficiency of Vim models. The key idea of Famba-V is to identify and fuse similar tokens across different Vim layers based on a suit of cross-layer strategies instead of simply applying token fusion uniformly across all the layers that existing works propose. We evaluate the performance of Famba-V on CIFAR-100. Our results show that Famba-V is able to enhance the training efficiency of Vim models by reducing both training time and peak memory usage during training. Moreover, the proposed cross-layer strategies allow Famba-V to deliver superior accuracy-efficiency trade-offs. These results all together demonstrate Famba-V as a promising efficiency enhancement technique for Vim models.
ME-Mamba: Multi-Expert Mamba with Efficient Knowledge Capture and Fusion for Multimodal Survival Analysis
Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai 200032, China(复旦大学基础医学学院数字医学研究中心);Shanghai Key Lab of Medical Image Computing(上海医学图像计算重点实验室)
DEPFusion: Dual-Domain Enhancement and Priority-Guided Mamba Fusion for UAV Multispectral Object Detection
School of Information Engineering, Guangdong University of Technology(广东工业大学信息工程学院);School of Electronic and Information Engineering, South China University of Technology(华南理工大学电子信息学院);School of Artificial Intelligence, XiDian University(西安电子科技大学人工智能学院)
COMO: Cross-Mamba Interaction and Offset-Guided Fusion for Multimodal Object Detection
School of Resource and Environmental Sciences, Wuhan University(武汉大学资源与环境科学学院);State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University(武汉大学测绘遥感信息工程国家重点实验室);School of Mathematics and Statistics, University of Glasgow(格拉斯哥大学数学与统计学院);School of Geophysics and Geomatics, China University of Geoscience(中国地质大学(武汉)地球物理与空间信息学院)
An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusion
Labcin - Nature Inspired Computing Lab, Federal University of Esp\'irito Santo, Vit\'oria, Brazil PPGI - Graduate Program in Computer Science, Federal University of Esp\'irito Santo, Vit\'oria, Brazil
RUFNet: Query-Guided Support Mask Refinement and Uncertainty Fusion based on Hybrid Mamba for Few-Shot Brain Tumor Segmentation
Department of Language Science and Technology, The Hong Kong Polytechnic University(香港理工大学语言科学与技术系);School of Artificial Intelligence, Chongqing University of Technology(重庆理工大学人工智能学院);Affiliated Lianyungang Hospital of Xuzhou Medical University(徐州医科大学附属连云港医院);College of Mechatronics and Control Engineering, Shenzhen University(深圳大学机电与控制工程学院)
UAVD-Mamba: Deformable Token Fusion Vision Mamba for Multimodal UAV Detection
College of Mechanical and Vehicle Engineering, Hunan University(湖南大学机械与车辆工程学院);School of Electronic Information and Communications, Huazhong University of Science and Technology(华中科技大学电子信息与通信学院)
MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition
Computer Vision and Pattern Recognition Laboratory, Department of Computational Engineering, Lappeenranta-Lahti University of Technology LUT(计算机视觉与模式识别实验室,计算工程系,拉佩兰塔-拉赫蒂理工大学);School of Electrical and Information Engineering, Tianjin University(电气与信息工程学院,天津大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电子与电气工程学院,南洋理工大学);Faculty of Information Technology, Brno University of Technology(信息技术学院,布拉格技术大学)
MambaDSF: Multi-Scale SSM with Dilated Feature Fusion for Sonar Small Target Detection
School of Information Science and Engineering, Ocean University of China(中国海洋大学信息科学与工程学院);School of Information and Communication Technology, Griffith University(格里菲斯大学信息与通信技术学院)
Shuffle Mamba: State Space Models with Random Shuffle for Multi-Modal Image Fusion
University of Science and Technology of China(中国科学技术大学);Institute of Intelligent Machines(智能机器研究所);Hefei Institutes of Physical Science, Chinese Academy of Sciences(中国科学院合肥物质科学研究院);Intelligent Agriculture Engineering Laboratory of Anhui Province, Institute of Intelligent Machines(安徽省智能农业工程实验室,智能机器研究所)
Spatial-Frequency Enhanced Mamba for Multi-Modal Image Fusion
School of Future Technology, School of Artificial Intelligence, Dalian University of Technology(大连理工大学未来技术学院、人工智能学院);Affiliated Zhongshan Hospital of Dalian University(大连大学附属中山医院);Central Hospital of Dalian University of Technology(大连理工大学中心医院);School of Information and Communication Engineering, Dalian University of Technology(大连理工大学信息与通信工程学院)
Mamba-Driven Topology Fusion for Monocular 3D Human Pose Estimation
College of Sciences, Northeastern University(东北大学理学院);Key Laboratory of Differential Equations and Their Applications, Northeastern University(东北大学偏微分方程及其应用重点实验室);Nanjing University of Science and Technology(南京理工大学)
DepMamba: Progressive Fusion Mamba for Multimodal Depression Detection
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University(复旦大学脑智能科学与技术研究院);School of Computer Science, Fudan University(复旦大学计算机科学技术学院)
D2-Mamba: Dual-Scale Fusion and Dual-Path Scanning with SSMs for Shadow Removal
School of Artificial Intelligence, Hebei University of Technology(人工智能学院,河北工业大学);School of Electrical and Computer Engineering, University of Texas at Austin(电气与计算机工程学院,德克萨斯大学奥斯汀分校);School of Health Sciences and Biomedical Engineering, Hebei University of Technology(健康科学与生物医学工程学院,河北工业大学)
ClinicalFMamba: Advancing Clinical Assessment using Mamba-based Multimodal Neuroimaging Fusion
TD Bank Group, Toronto, Canada(TD银行集团,加拿大);Department of Computer Science, University of Toronto, Toronto, Canada(多伦多大学计算机科学系,加拿大);Department of Medical Imaging, University of Toronto, Toronto, Canada(多伦多大学医学影像系,加拿大)
MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
Hangzhou City University(杭州城市大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);Hangzhou Normal University(杭州师范大学);Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance(浙江省实时智能安防技术工程研究中心)
CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration
School of Physics(物理学院);Optoelectronic Engineering, Foshan University, Foshan 528225, China(光学电子工程学院,佛山大学,佛山528225,中国);Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China(粤港澳联合智能微纳光电子技术实验室,佛山528225,中国);School of Mechanical Electronic(机械电子学院);Information Engineering, China University of Mining and Technology, Beijing 100083, China(信息工程学院,中国矿业大学,北京100083,中国)
Laplace-Mamba: Laplace Frequency Prior-Guided Mamba-CNN Fusion Network for Image Dehazing
School of Computer Science and Technology, Anhui University of Technology(安徽理工大学计算机科学与技术学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);College of Artificial Intelligence, Taiyuan University of Technology(太原科技大学人工智能学院)
MMR-Mamba: Multi-Modal MRI Reconstruction with Mamba and Spatial-Frequency Information Fusion
The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);Stanford University(斯坦福大学)
Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies
Univ. of Buenos Aires. Faculty of Exact and Natural Sciences. Dept. of Computer Science (DC)(布宜诺斯艾利斯大学。精确与自然科学学院。计算机科学系);CONICET-Univ. of Buenos Aires. Institute of Computer Sciences (ICC)(布宜诺斯艾利斯大学CONICET联合体。计算机科学研究所)
XFMamba: Cross-Fusion Mamba for Multi-View Medical Image Classification
Digital Environment Research Institute (DERI), Queen Mary University of London(伦敦大学玛丽女王学院数字环境研究所(DERI));School of Electronic Engineering and Computer Science, Queen Mary University of London(伦敦大学玛丽女王学院电子工程与计算机科学学院);Blizard Institute - Faculty of Medicine and Dentistry, Queen Mary University of London(伦敦大学玛丽女王学院医学与牙科学院布利扎德研究所);University of Cambridge(剑桥大学)
FMRFT: Fusion Mamba and DETR for Query Time Sequence Intersection Fish Tracking
National Innovation Center for Digital Fishery(国家数字渔业创新中心);Key Laboratory of Smart Farming Technologies for Aquatic Animal and Livestock, Ministry of Agriculture and Rural Affairs(农业农村部水产动物与畜禽智慧养殖技术重点实验室);College of Information and Electrical Engineering, China Agricultural University(中国农业大学信息与电气工程学院);College of Engineering, China Agricultural University(中国农业大学工学院);Smart City College, Beijing Union University(北京联合大学智慧城市学院)
Mamba Based Feature Extraction And Adaptive Multilevel Feature Fusion For 3D Tumor Segmentation From Multi-modal Medical Image
School of Computer Science and Engineering, Central South University, Changsha, 410083, China(计算机科学与工程学院,中南大学,长沙,410083,中国);Hunan Engineering Research Center of Machine Vision and Intelligent Medicine, Central South University, Changsha, 410083, China(机器视觉与智能医学工程研究中心,中南大学,长沙,410083,中国);Department of Radiation Oncology, Washington University in St. Louis, USA(放射肿瘤科,华盛顿大学圣路易斯分校,美国);Department of Nuclear Medicine, Henri Becquerel Cancer Center, Rouen, France(核医学科,亨利·贝克勒尔癌症中心,鲁昂,法国);University of Rouen-Normandy, AMIS - QuantIF UR 4108, F-76000, Rouen, France(鲁昂-诺曼底大学,AMIS - QuantIF UR 4108,法国,F-76000,鲁昂,法国)
MM-DETR: An Efficient Multimodal Detection Transformer with Mamba-Driven Dual-Granularity Fusion and Frequency-Aware Modality Adapters
School of Information and Electronics, Beijing Institute of Technology(信息与电子学院,北京理工大学);Beijing Institute of Technology Chongqing Innovation Center(北京理工大学重庆创新中心);National Key Laboratory for Space-Born Intelligent Information Processing(空间智能信息处理国家级重点实验室);School of Automation, Beijing Institute of Technology(自动化学院,北京理工大学)
AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology(大连理工大学信息与通信工程学院);School of Innovation and Entrepreneurship, Dalian University of Technology(大连理工大学创新创业学院);School of Artificial Intelligence, Dalian University of Technology(大连理工大学人工智能学院)
Interactive Spatial-Frequency Fusion Mamba for Multi-Modal Image Fusion
School of Future Technology, Dalian University of Technology and the Key Laboratory of Data Science and Smart Education (Hainan Normal University), Ministry of Education(未来技术学院,大连理工大学和数据科学与智能教育关键实验室(海南师范大学),教育部);Affiliated Zhongshan Hospital of Dalian University(大连大学附属中山医院);School of Computer Science and Artificial Intelligence, Wuhan University of Technology(计算机科学与人工智能学院,武汉理工大学);Central Hospital of Dalian University of Technology(大连理工大学中心医院);School of Information and Communication Engineering, Dalian University of Technology(信息与通信工程学院,大连理工大学)
Vision-Proprioception Fusion with Mamba2 in End-to-End Reinforcement Learning for Motion Control
School of Computer Science and Statistics, Trinity College Dublin(都柏林三一学院计算机科学与统计学系);Faculty of Engineering and Information Technology, University of Technology Sydney(新南威尔士大学理工学院)
MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), CASIA(多模态人工智能系统国家重点实验室(MAIS),中国科学院自动化所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Anyverse Intelligence;Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information(北京多模态信息超级智能安全重点实验室);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)
Axial-Relation Guided Fusion State Space Model for Optical-Elevation Sensing Image Segmentation
State Key Laboratory of Physical Oceanography, Ocean University of China(中国海洋大学物理海洋学国家重点实验室);Department of Electrical and Computer Engineering, Mississippi State University(密苏里州立大学电气与计算机工程系)