Long-range spatiotemporal dependencies capturing plays an essential role in improving video features for action recognition. The non-local block inspired by the non-local means is designed to address this challenge and have shown excellent performance. However, the non-local block brings significant increase in computation cost to the original network. It also lacks the ability to model regional correlation in videos. To address the above limitations, we propose Pyramid Non-Local (PNL) module, which extends the non-local block by incorporating regional correlation at multiple scales through a pyramid structured module. This extension upscales the effectiveness of non-local operation by attending to the interaction between different regions. Empirical results prove the effectiveness and efficiency of our PNL module, which achieves state-of-the-art performance of 83.09% on the Mini-Kinetics dataset, with decreased computation cost compared to the non-local block.
Most of the existing Siamese-based trackers treat tracking problem as a parallel task of classification and regression. However, some studies show that the sibling head structure could lead to suboptimal solutions during the network training. Through experiments we find that, without regression, the performance could be equally promising as long as we delicately design the network to suit the training objective. We introduce a novel voting-based classification-only tracking algorithm named Pyramid Correlation based Deep Hough Voting (short for PCDHV), to jointly locate the top-left and bottom-…
In this paper, we propose a spatio-temporal contextual network, STC-Flow, for optical flow estimation. Unlike previous optical flow estimation approaches with local pyramid feature extraction and multi-level correlation, we propose a contextual relation exploration architecture by capturing rich long-range dependencies in spatial and temporal dimensions. Specifically, STC-Flow contains three key context modules - pyramidal spatial context module, temporal context correlation module and recurrent residual contextual upsampling module, to build the relationship in each stage of feature extractio…
Monash University（莫纳什大学）；University of Technology Sydney（悉尼科技大学）；RMIT University（皇家墨尔本理工大学）
The task of action detection aims at deducing both the action category and localization of the start and end moment for each action instance in a long, untrimmed video. While vision Transformers have driven the recent advances in video understanding, it is non-trivial to design an efficient architecture for action detection due to the prohibitively expensive self-attentions over a long sequence of video clips. To this end, we present an efficient hierarchical Spatio-Temporal Pyramid Transformer (STPT) for action detection, building upon the fact that the early self-attention layers in Transfor…
T-SiamTPN: Temporal Siamese Transformer Pyramid Networks for Robust and Efficient UAV Tracking
Department of Electrical Engineering, Amirkabir University of Technology (AUT)（电气工程系，阿米尔卡比尔技术大学）；Qatar Computing Research Institute Hamad Bin Khalifa University（卡塔尔计算研究所，哈马德·本·卡西姆大学）
Pyramid Forcing: Head-Aware Pyramid KV Cache Policy for High-Quality Long Video Generation
Peking University（北京大学）；South China University of Technology（华南理工大学）；Xinjiang University（新疆大学）；Beihang University（北京航空航天大学）；Zhongguancun Academy（中关村学院）
Existing neural networks proposed for low-level image processing tasks are usually implemented by stacking convolution layers with limited kernel size. Every convolution layer merely involves in context information from a small local neighborhood. More contextual features can be explored as more convolution layers are adopted. However it is difficult and costly to take full advantage of long-range dependencies. We propose a novel non-local module, Pyramid Non-local Block, to build up connection between every pixel and all remain pixels. The proposed module is capable of efficiently exploiting …
Deep neural networks have been a prevailing technique in the field of medical image processing. However, the most popular convolutional neural networks (CNNs) based methods for medical image segmentation are imperfect because they model long-range dependencies by stacking layers or enlarging filters. Transformers and the self-attention mechanism are recently proposed to effectively learn long-range dependencies by modeling all pairs of word-to-word attention regardless of their positions. The idea has also been extended to the computer vision field by creating and treating image patches as emb…
Tianjin University（天津大学）；Hefei University of Technology（合肥工业大学）；Tianjin University of Technology（天津理工大学）
Temporal action localization plays an important role in video analysis, which aims to localize and classify actions in untrimmed videos. The previous methods often predict actions on a feature space of a single-temporal scale. However, the temporal features of a low-level scale lack enough semantics for action classification while a high-level scale cannot provide rich details of the action boundaries. To address this issue, we propose to predict actions on a feature space of multi-temporal scales. Specifically, we use refined feature pyramids of different scales to pass semantics from high-le…
Building extraction from aerial images has several applications in problems such as urban planning, change detection, and disaster management. With the increasing availability of data, Convolutional Neural Networks (CNNs) for semantic segmentation of remote sensing imagery has improved significantly in recent years. However, convolutions operate in local neighborhoods and fail to capture non-local features that are essential in semantic understanding of aerial images. In this work, we propose to improve building segmentation of different sizes by capturing long-range dependencies using context…
P2AT: Pyramid Pooling Axial Transformer for Real-time Semantic Segmentation
School of Computer Science, Zhejiang Normal University（浙江师范大学计算机科学学院）；School of Mathematical Medicine, Zhejiang Normal University（浙江师范大学数学医学院）；Zhejiang Institute of Photoelectronics & Zhejiang Institute for Advanced Light Source, Zhejiang Normal University（浙江师范大学光电研究所与浙江先进光源研究所）；School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications（重庆邮电大学通信与信息工程学院）
Tencent（腾讯）；School of Software Engineering, South China University of Technology（华南理工大学软件学院）
For object detection, how to address the contradictory requirement between feature map resolution and receptive field on high-resolution inputs still remains an open question. In this paper, to tackle this issue, we build a novel architecture, called Attention-guided Context Feature Pyramid Network (AC-FPN), that exploits discriminative information from various large receptive fields via integrating attention-guided multi-path features. The model contains two modules. The first one is Context Extraction Module (CEM) that explores large contextual information from multiple receptive fields. As …
Northern Arizona University（北亚利桑那大学）；Xi’an Network Computing Data Technology Co., Ltd.（西安网络计算数据技术有限公司）
Convolutional layers in Artificial Neural Networks (ANN) treat the channel features equally without feature selection flexibility. While using ANNs for image denoising in real-world applications with unknown noise distributions, particularly structured noise with learnable patterns, modeling informative features can substantially boost the performance. Channel attention methods in real image denoising tasks exploit dependencies between the feature channels, hence being a frequency component filtering mechanism. Existing channel attention modules typically use global statics as descriptors to l…
AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentation
School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Innovation and Entrepreneurship, Dalian University of Technology（大连理工大学创新创业学院）；School of Artificial Intelligence, Dalian University of Technology（大连理工大学人工智能学院）
Coronavirus Disease 2019 (COVID-19) has rapidly spread in 2020, emerging a mass of studies for lung infection segmentation from CT images. Though many methods have been proposed for this issue, it is a challenging task because of infections of various size appearing in different lobe zones. To tackle these issues, we propose a Graph-based Pyramid Global Context Reasoning (Graph-PGCR) module, which is capable of modeling long-range dependencies among disjoint infections as well as adapt size variation. We first incorporate graph convolution to exploit long-term contextual information from multi…
PLA Strategic Support Force Information Engineering University（中国人民解放军战略支援部队信息工程大学）
We present an efficient multi-view stereo (MVS) network for 3D reconstruction from multiview images. While previous learning based reconstruction approaches performed quite well, most of them estimate depth maps at a fixed resolution using plane sweep volumes with a fixed depth hypothesis at each plane, which requires densely sampled planes for desired accuracy and therefore is difficult to achieve high resolution depth maps. In this paper we introduce a coarseto-fine depth inference strategy to achieve high resolution depth. This strategy estimates the depth map at coarsest level, while the d…
Huazhong University of Science and Technology（华中科技大学）；University of Oxford（牛津大学）
The non-local module works as a particularly useful technique for semantic segmentation while criticized for its prohibitive computation and GPU memory occupation. In this paper, we present Asymmetric Non-local Neural Network to semantic segmentation, which has two prominent components: Asymmetric Pyramid Non-local Block (APNB) and Asymmetric Fusion Non-local Block (AFNB). APNB leverages a pyramid sampling module into the non-local block to largely reduce the computation and memory consumption without sacrificing the performance. AFNB is adapted from APNB to fuse the features of different leve…
School of Informatics, Xiamen University（厦门大学信息学院）；College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；University of Luxembourg（卢森堡大学）；School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）
Sea AI Lab（Sea人工智能实验室）；National University of Singapore（新加坡国立大学）
While transformers have shown great potential on video recognition with their strong capability of capturing long-range dependencies, they often suffer high computational costs induced by the self-attention to the huge number of 3D tokens. In this paper, we present a new transformer architecture termed DualFormer, which can efficiently perform space-time attention for video recognition. Concretely, DualFormer stratifies the full space-time attention into dual cascaded levels, i.e., to first learn fine-grained local interactions among nearby 3D tokens, and then to capture coarse-grained global …
Sun Yat-sen University（中山大学）；The University of Hong Kong（香港大学）；Deepwise AI Lab（深智云人工智能实验室）
Recent progress in biomedical image segmentation based on deep convolutional neural networks (CNNs) has drawn much attention. However, its vulnerability towards adversarial samples cannot be overlooked. This paper is the first one that discovers that all the CNN-based state-of-the-art biomedical image segmentation models are sensitive to adversarial perturbations. This limits the deployment of these methods in safety-critical biomedical fields. In this paper, we discover that global spatial dependencies and global contextual information in a biomedical image can be exploited to defend against …
MambaPanoptic: A Vision Mamba-based Structured State Space Framework for Panoptic Segmentation
Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Polytechnic University of Milan（米兰理工大学）；University of Stuttgart（斯图加特大学）；Wuhan University（武汉大学）；Karlsruhe University of Applied Sciences（卡尔斯鲁厄应用科学大学）
Accurate and lightweight dehazing via multi-receptive-field non-local network and novel contrastive regularization
Huanjiang Laboratory, Zhuji, P.R. China and School of Aeronautics and Astronautics, Zhejiang University, Hangzhou, P.R.China（湖南江实验室，浙江诸暨，中国 和 浙江大学航空宇航学院，杭州，中国）；School of Aeronautics and Astronautics, Zhejiang University, Hangzhou, P.R.China（浙江大学航空宇航学院，杭州，中国）；Electrical and Computer Engineering Department, Hellenic Mediterranean University, Heraklion, Crete, Greece（希腊地中海大学电气与计算机工程系，赫拉克利翁，克里特，希腊）
Beihang University（北京航空航天大学）；State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology（高效可重复使用航天运输技术国家重点实验室）；The School of Astronautics（航天学院）；The National Key Laboratory of Integrated Air-Ground Navigation Technologies（集成空天地导航技术国家实验室）；The State Key Laboratory of Media Convergence and Communication（媒体融合与传播国家重点实验室）；The School of Artificial Intelligence（人工智能学院）；Zhongguancun Academy（中关村学院）；The School of Automation Science and Electrical Engineering（自动化科学与电气工程学院）；Beijing Institute of Control and Electronics Technology（北京控制电子技术研究所）；Ningbo Institute of Digital Twin（宁波数字孪生研究所）；Eastern Institute of Technology（东部技术研究所）；Hangzhou Research Institute, School of Artificial Intelligence（杭州研究院，人工智能学院）
研究基于双流 CNN 的 RGB-IR 目标检测问题，提出拉普拉斯解耦特征增强模块 LDFE，通过全局-局部分解等操作融合不同阶段特征，在多个数据集上显著提升目标检测性能，mAP 超 SOTA 方法。
