Transformer architectures are now central to sequence modeling tasks. At its heart is the attention mechanism, which enables effective modeling of long-term dependencies in a sequence. Recently, transformers have been successfully applied in the computer vision domain, where 2D images are first segmented into patches and then treated as 1D sequences. Such linearization, however, impairs the notion of spatial locality in images, which bears important visual clues. To bridge the gap, we propose ripple attention, a sub-quadratic attention mechanism for vision transformers. Built upon the recent kernel-based efficient attention mechanisms, we design a novel dynamic programming algorithm that weights contributions of different tokens to a query with respect to their relative spatial distances in the 2D space in linear observed time. Extensive experiments and analyses demonstrate the effectiveness of ripple attention on various visual tasks.
Walkable to Whom? Capturing Subjective Variability in Walkability Perception Using Multimodal Deep Learning
School of Civil and Environmental Engineering, University of New South Wales (UNSW)（新南威尔士大学土木与环境工程学院）；Research Centre for Integrated Transport Innovation (rCITI)（综合交通创新研究中心）
The lower the distortion of an estimator, the more the distribution of its outputs generally deviates from the distribution of the signals it attempts to estimate. This phenomenon, known as the perception-distortion tradeoff, has captured significant attention in image restoration, where it implies that fidelity to ground truth images comes at the expense of perceptual quality (deviation from statistics of natural images). However, despite the increasing popularity of performing comparisons on the perception-distortion plane, there remains an important open question: what is the minimal distor…
Attention mechanisms in biological perception are thought to select subsets of perceptual information for more sophisticated processing which would be prohibitive to perform on all sensory inputs. In computer vision, however, there has been relatively little exploration of hard attention, where some information is selectively ignored, in spite of the success of soft attention, where information is re-weighted and aggregated, but never filtered out. Here, we introduce a new approach for hard attention and find it achieves very competitive performance on a recently-released visual question answe…
Fourier-RWKV: A Multi-State Perception Network for Efficient Image Dehazing
Institute of Intelligent Information Processing, Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen Key Laboratory of Modern Communications and Information Processing, Shenzhen University（智能信息处理研究院）；College of Engineering and Computer Science, Australian National University（工程与计算机科学学院）
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
This paper investigates the vision-based autonomous driving with deep learning and reinforcement learning methods. Different from the end-to-end learning method, our method breaks the vision-based lateral control system down into a perception module and a control module. The perception module which is based on a multi-task learning neural network first takes a driver-view image as its input and predicts the track features. The control module which is based on reinforcement learning then makes a control decision based on these features. In order to improve the data efficiency, we propose visual…
Beijing University of Posts and Telecommunications（北京邮电大学）
Traffic prediction has gradually attracted the attention of researchers because of the increase in traffic big data. Therefore, how to mine the complex spatio-temporal correlations in traffic data to predict traffic conditions more accurately become a difficult problem. Previous works combined graph convolution networks (GCNs) and self-attention mechanism with deep time series models (e.g. recurrent neural networks) to capture the spatio-temporal correlations separately, ignoring the relationships across time and space. Besides, GCNs are limited by over-smoothing issue and self-attention is li…
S2AFormer: Strip Self-Attention for Efficient Vision Transformer
Faculty of Engineering and Information Technology, University of Technology Sydney（工程与信息技术学院，悉尼大学）；Bionic Vision System Laboratory, State Key Laboratory of Transducer Technology, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（生物视觉系统实验室，传感技术国家重点实验室，上海微系统与信息技术研究所，中国科学院）；PCA Lab, Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（PCA实验室，教育部高维信息智能感知与系统重点实验室，南京理工大学计算机科学与工程学院）；Research Center for Industries of the Future and the School of Engineering, Westlake University（未来产业研究中心和工程学院，西湖大学）；OPPO Research, Seattle, WA 98101 USA（OPPO研究，美国西雅图华盛顿州98101）
ShadowMamba: State-Space Model with Boundary-Region Selective Scan for Shadow Removal
Department of Electrical Engineering, Faculty of Engineering, Universiti Malaya（马来大学电气工程系）；Faculty of Engineering and Information Technology, Southern University College（南方学院工程与信息技术学院）
LinearSR: Unlocking Linear Attention for Stable and Efficient Image Super-Resolution
Shanghai Jiao Tong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；The Australian National University（澳大利亚国立大学）
AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Innovation and Entrepreneurship, Dalian University of Technology（大连理工大学创新创业学院）；School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院）
Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving
Shanghai Jiao Tong University（上海交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；Carnegie Mellon University（卡内基梅隆大学）；Sichuan University（四川大学）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）
LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Institute of Visual Intelligence, Pengcheng Laboratory（鹏城实验室视觉智能研究院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Bytedance Inc.（字节跳动公司）
PromptSR: Cascade Prompting for Lightweight Image Super-Resolution
School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore（南洋理工大学电子与电气工程学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China（华中科技大学电子信息与通信学院）；Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong（香港理工大学电子与电气工程系）
