通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2409.00410cs.CV
一种用于单图像去雨的混合Transformer-Mamba网络
A Hybrid Transformer-Mamba Network for Single Image Deraining
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间科学与技术学院）；Yunnan Normal University（云南师范大学）
Existing deraining Transformers employ self-attention mechanisms with fixed-range windows or along channel dimensions, limiting the exploitation of non-local receptive fields. In response to this issue, we introduce a novel dual-branch hybrid Transformer-Mamba network, denoted as TransMamba, aimed at effectively capturing long-range rain-related dependencies. Based on the prior of distinct spectral-domain features of rain degradation and background, we design a spectral-banded Transformer blocks on the first branch. Self-attention is executed within the combination of the spectral-domain channel dimension to improve the ability of modeling long-range dependencies. To enhance frequency-specific information, we present a spectral enhanced feed-forward module that aggregates features in the spectral domain. In the second branch, Mamba layers are equipped with cascaded bidirectional state space model modules to additionally capture the modeling of both local and global information. At each stage of both the encoder and decoder, we perform channel-wise concatenation of dual-branch features and achieve feature fusion through channel reduction, enabling more effective integration of the multi-scale information from the Transformer and Mamba branches. To better reconstruct innate signal-level relations within clean images, we also develop a spectral coherence loss. Extensive experiments on diverse datasets and real-world images demonstrate the superiority of our method compared against the state-of-the-art approaches.
PRE-Mamba: A 4D State Space Model for Ultra-High-Frequent Event Camera Deraining
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Harbin Institute of Technology（哈尔滨工业大学）；Carnegie Mellon University（卡内基梅隆大学）；Pengcheng Laboratory（鹏城实验室）
Australian National University（澳大利亚国立大学）；Data61-CSIRO（澳大利亚联邦科学与工业研究组织数据61研究所）
Over parameterization is a common technique in deep learning to help models learn and generalize sufficiently to the given task; nonetheless, this often leads to enormous network structures and consumes considerable computing resources during training. Recent powerful transformer-based deep learning models on vision tasks usually have heavy parameters and bear training difficulty. However, many dense-prediction low-level computer vision tasks, such as rain streak removing, often need to be executed on devices with limited computing power and memory in practice. Hence, we introduce a recursive …
Nanjing University of Science and Technology（南京理工大学）；China Electronics Technology Group Corporation（中国电子科技集团有限公司）
Transformers-based methods have achieved significant performance in image deraining as they can model the non-local information which is vital for high-quality image reconstruction. In this paper, we find that most existing Transformers usually use all similarities of the tokens from the query-key pairs for the feature aggregation. However, if the tokens from the query are different from those of the key, the self-attention values estimated from these tokens also involve in feature aggregation, which accordingly interferes with the clear image restoration. To overcome this problem, we propose …
Wuhan University（武汉大学）；University of Central Florida（中佛罗里达大学）；Shenzhen University（深圳大学）；National Tsing Hua University（台湾清华大学）
Convolutional neural network (CNN) and Transformer have achieved great success in multimedia applications. However, little effort has been made to effectively and efficiently harmonize these two architectures to satisfy image deraining. This paper aims to unify these two architectures to take advantage of their learning merits for image deraining. In particular, the local connectivity and translation equivariance of CNN and the global aggregation ability of self-attention (SA) in Transformer are fully exploited for specific local context and global structure representations. Based on the obser…
Exploiting Regional Information Transformer for Single Image Deraining
The Chinese University of Hong Kong（香港中文大学）；Zhejiang University（浙江大学）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）
Cross Paradigm Representation and Alignment Transformer for Image Deraining
Nanjing Agricultural University（南京农业大学）；Soochow University（苏州大学）；Xiangtan University（湘潭大学）；Shanghai University（上海大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）；Westlake University（西湖大学）
Rain streaks degrade the image quality and seriously affect the performance of subsequent computer vision tasks, such as autonomous driving, social security, etc. Therefore, removing rain streaks from a given rainy images is of great significance. Convolutional neural networks(CNN) have been widely used in image deraining tasks, however, the local computational characteristics of convolutional operations limit the development of image deraining tasks. Recently, the popular transformer has global computational features that can further facilitate the development of image deraining tasks. In thi…
JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model
University of Alberta, Multimedia Research Center (MRC)（阿尔伯塔大学多媒体研究中心）；University of Florence, Media Integration and Communication Center (MICC)（佛罗伦萨大学媒体整合与通信中心）
Transformer-Progressive Mamba Network for Lightweight Image Super-Resolution
Bell Honors School, Nanjing University of Posts and Telecommunications（南京邮电大学贝尔荣誉学院）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学集成服务网络国家重点实验室）；Pattern Recognition and Intelligent System Laboratory, School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院模式识别与智能系统实验室）；Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院智能感知与高维信息系统教育部重点实验室）；Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University（国立交通大学电子工程系与通信工程研究所）
Hybrid Transformer-Mamba for Weakly Supervised Volumetric Medical Segmentation
University of Western Australia（西澳大学）；Harry Perkins Institute of Medical Research（哈利·佩金斯医学研究所）；National Imaging Facility（国家成像设施）；Fiona Stanley Hospital（菲奥娜·斯蒂尔医院）；Victor Chang Cardiac Research Institute（维多利亚·张心脏研究中心）
TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Hong Kong Polytechnic University（香港理工大学）；Zhuhai Campus of Sun Yat-sen University（中山大学珠海校区）；University of Adelaide（阿德莱德大学）；Peking University（北京大学）；Huawei Noah’s Ark Lab（华为诺亚实验室）
MaTVLM: Hybrid Mamba-Transformer for Efficient Vision-Language Modeling
School of EIC, Huazhong University of Science & Technology（华中科技大学电子信息与通信学院）；Institute of Artificial Intelligence, Huazhong University of Science & Technology（华中科技大学人工智能研究院）
Multi-dimensional Visual Prompt Enhanced Image Restoration via Mamba-Transformer Aggregation
School of Digital Industry, Jiangxi Normal University（江西师范大学数字产业学院）；School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）
Hybrid Vision Transformer-Mamba Framework for Autism Diagnosis via Eye-Tracking Analysis
Faculty of Science and Technology, Tissemsilt University（科技与技术学院，蒂塞姆西尔大学）；College of Engineering and Information Technology University of Dubai（工程与信息科技学院，迪拜大学）；College of Medicine and Health Sciences, Mohammed Bin Rashid University Dubai, UAE（医学与健康科学学院，迪拜穆罕默德·本·拉希德大学）；School of Clinical Medicine University of New South Wales, Australia（临床医学学院，新南威尔士大学）
InfiniMotion: Mamba Boosts Memory in Transformer for Arbitrary Long Motion Generation
Monash University（莫纳什大学）；The Australian National University（澳大利亚国立大学）；The University of Adelaide（阿德莱德大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Peking University（北京大学）
GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions
Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Hong Kong University of Science and Technology（香港科技大学）；Rakuten Institute of Technology（乐天理工研究院）；Imperial College London（伦敦帝国学院）；KAIST（韩国科学技术院）；Tencent（腾讯）；Australian National University（澳大利亚国立大学）；Nanjing University（南京大学）
MambaPanoptic: A Vision Mamba-based Structured State Space Framework for Panoptic Segmentation
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Polytechnic University of Milan（米兰理工大学）；University of Stuttgart（斯图加特大学）；Wuhan University（武汉大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）
