In this paper, we introduce Matten, a cutting-edge latent diffusion model with Mamba-Attention architecture for video generation. With minimal computational cost, Matten employs spatial-temporal attention for local video content modeling and bidirectional Mamba for global video content modeling. Our comprehensive experimental evaluation demonstrates that Matten has competitive performance with the current Transformer-based and GAN-based models in benchmark performance, achieving superior FVD scores and efficiency. Additionally, we observe a direct positive correlation between the complexity of our designed model and the improvement in video quality, indicating the excellent scalability of Matten.
Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机科学学院，北京大学）；UCAS-Terminus AI Lab, University of Chinese Academy of Sciences（中国科学院大学Terminus AI实验室，中国科学院大学）；School of Intelligent Science and Technology, Nanjing University（智能科学与技术学院，南京大学）；Department of Information Technology and Electrical Engineering, ETH Zurich（信息科技与电气工程系，苏黎世联邦理工学院）；Department of Electrical Engineering, KU Leuven（电气工程系，鲁汶大学）；INSAIT, Sofia Un.（INSAIT，索菲亚大学）；Department of Information Engineering and Computer Science (DISI), University of Trento（信息工程与计算机科学系（DISI），特伦托大学）
MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos
IRMV Lab, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University and State Key Laboratory of Avionics Integration and Aviation System-of-Systems Synthesis, Shanghai Key Laboratory of Navigation and Location Based Services（IRMV实验室，自动化与智能感知学院，上海交通大学，航空集成与航空系统-of-Systems综合国家重点实验室，导航与定位服务重点实验室）；College of Intelligence Science and Technology, National University of Defense Technology（智能科学与技术学院，国防科技大学）；Department of Electronic Engineering, Shanghai Jiao Tong University（电子工程学院，上海交通大学）
Mamba-Enhanced Implicit Motion Learning for Audio-Driven Portrait Animation
Fujian Provincial Natural Science Foundation of China（福建省自然科学基金委员会）；Giant Interactive Group Inc.（巨匠互动集团有限公司）；National Natural Science Foundation of China（国家自然科学基金委员会）
Not Like Transformers: Drop the Beat Representation for Dance Generation with Mamba-Based Diffusion Model
Ulsan National Institute of Science and Technology, South Korea（乌山国立科学技术研究院，韩国）；Daegu Gyeongbuk Institute of Science and Technology, South Korea（大邱庆北科学技术研究院，韩国）
QueryMamba: A Mamba-Based Encoder-Decoder Architecture with a Statistical Verb-Noun Interaction Module for Video Action Forecasting @ Ego4D Long-Term Action Anticipation Challenge 2024
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Fraunhofer IOSB（弗劳恩霍夫IOSB研究所）
Vivim: a Video Vision Mamba for Medical Video Segmentation
The Hong Kong University of Science and Technology, Guangzhou（香港科技大学（广州））；The University of Hong Kong（香港大学）；Guangdong Provincial People’s Hospital（广东省人民医院）；Institute of High Performance Computing, A*STAR（新加坡高性能计算研究院（新加坡科研局下属机构））；Hong Kong University of Science and Technology（香港科技大学）
MamFusion: Multi-Mamba with Temporal Fusion for Partially Relevant Video Retrieval
Hangzhou City University（杭州城市大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Hangzhou Normal University（杭州师范大学）；Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance（浙江省实时智能安防技术工程研究中心）
MambaVSR: Content-Aware Scanning State Space Model for Video Super-Resolution
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学学院）；Visual Intelligence + X International Cooperation Joint Laboratory of MOE, Beijing Jiaotong University（教育部视觉智能+X国际合作联合实验室）；School of Computer and Communication Engineering, University of Science and Technology Beijing（北京科技大学计算机与通信工程学院）
Safe-Sora: Safe Text-to-Video Generation via Graphical Watermarking
Tsinghua University（清华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；South China University of Technology（华南理工大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东人工智能与数字经济实验室）；Singapore Management University（新加坡管理学院）
Geometric Transformation-Embedded Mamba for Learned Video Compression
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi'an Jiaotong University（人机混合增强智能国家重点实验室，人工智能与机器人研究院，西安交通大学）；School of Information and Communications Engineering, Xi'an Jiaotong University（信息与通信工程学院，西安交通大学）
Cross-Modal Action Recognition in Egocentric Video Using Mamba: Integrating RGB and Hand Skeleton Streams via CLS Token Fusion Strategies
Univ. of Buenos Aires. Faculty of Exact and Natural Sciences. Dept. of Computer Science (DC)（布宜诺斯艾利斯大学。精确与自然科学学院。计算机科学系）；CONICET-Univ. of Buenos Aires. Institute of Computer Sciences (ICC)（布宜诺斯艾利斯大学CONICET联合体。计算机科学研究所）
FTDMamba: Frequency-Assisted Temporal Dilation Mamba for Unmanned Aerial Vehicle Video Anomaly Detection
Information Materials and Intelligent Sensing Laboratory of Anhui Province（安徽省信息材料与智能感知实验室）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation（安徽省多模态认知计算重点实验室）；School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）计算机科学与工程系）
Trajectory-aware Shifted State Space Models for Online Video Super-Resolution
Pengcheng Labartory（鹏城实验室）；University of Bristol（英国布里斯托大学）；University of Electronic Science and Technology of China（电子科技大学）；Shenzhen Graduate School, Peking University（北京大学深圳研究生院）
H-MBA: Hierarchical MamBa Adaptation for Multi-Modal Video Understanding in Autonomous Driving
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；School of Artificial Intelligence, University of Chinese Academy of Science（中国科学院大学人工智能学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
STNMamba: Mamba-based Spatial-Temporal Normality Learning for Video Anomaly Detection
Academy for Engineering and Technology, Fudan University（复旦大学工程技术研究院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）
AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Innovation and Entrepreneurship, Dalian University of Technology（大连理工大学创新创业学院）；School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院）
