Extracting and fusing part features have become the key of fined-grained image recognition. Recently, Non-local (NL) module has shown excellent improvement in image recognition. However, it lacks the mechanism to model the interactions between multi-scale part features, which is vital for fine-grained recognition. In this paper, we propose a novel cross-layer non-local (CNL) module to associate multi-scale receptive fields by two operations. First, CNL computes correlations between features of a query layer and all response layers. Second, all response features are weighted according to the correlations and are added to the query features. Due to the interactions of cross-layer features, our model builds spatial dependencies among multi-level layers and learns more discriminative features. In addition, we can reduce the aggregation cost if we set low-dimensional deep layer as query layer. Experiments are conducted to show our model achieves or surpasses state-of-the-art results on three benchmark datasets of fine-grained classification. Our codes can be found at github.com/FouriYe/CNL-ICIP2020.
School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi'an, Shaanxi Province, China（陕西科技大学电子信息与人工智能学院）；School of Electrical Engineering and Computer Science, the University of Queensland, QLD, Australia（昆士兰大学电气工程与计算机科学学院）；Institute for Integrated and Intelligent Systems, Griffith University, QLD, Australia（格里菲斯大学集成与智能系统研究所）；Chengdu University of Technology, Chengdu, Sichuan Province, China（成都理工大学）；Xidian University, Xi'an, Shaanxi Province, China（西安电子科技大学）；The First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi Province, China（西安交通大学第一附属医院）
School of Software and Microelectronics, Peking University（北京大学软件与微电子学院）；Alibaba Group, Inc.（阿里巴巴集团）
Learning subtle representation about object parts plays a vital role in fine-grained visual recognition (FGVR) field. The vision transformer (ViT) achieves promising results on computer vision due to its attention mechanism. Nonetheless, with the fixed size of patches in ViT, the class token in deep layer focuses on the global receptive field and cannot generate multi-granularity features for FGVR. To capture region attention without box annotations and compensate for ViT shortcomings in FGVR, we propose a novel method named Adaptive attention multi-scale Fusion Transformer (AFTrans). The Sele…
From Glance to Focus: Mitigating Recognition Laziness in MLLMs for Fine-Grained Recognition
College of Computer Science, Sichuan University（四川大学计算机学院）；School of Artificial Intelligence, Sichuan University（四川大学人工智能学院）；Tianfu Jincheng Laboratory（天府锦城实验室）
Locating discriminative parts plays a key role in fine-grained visual classification due to the high similarities between different objects. Recent works based on convolutional neural networks utilize the feature maps taken from the last convolutional layer to mine discriminative regions. However, the last convolutional layer tends to focus on the whole object due to the large receptive field, which leads to a reduced ability to spot the differences. To address this issue, we propose a novel Granularity-Aware Convolutional Neural Network (GA-CNN) that progressively explores discriminative feat…
Shanghai Jiao Tong University（上海交通大学）；Foshan Woman and Children’s Hospital（佛山市妇幼保健院）
Understanding the implication of point cloud is still challenging to achieve the goal of classification or segmentation due to the irregular and sparse structure of point cloud. As we have known, PointNet architecture as a ground-breaking work for point cloud which can learn efficiently shape features directly on unordered 3D point cloud and have achieved favorable performance. However, this model fail to consider the fine-grained semantic information of local structure for point cloud. Afterwards, many valuable works are proposed to enhance the performance of PointNet by means of semantic fea…
In image classification task, feature extraction is always a big issue. Intra-class variability increases the difficulty in designing the extractors. Furthermore, hand-crafted feature extractor cannot simply adapt new situation. Recently, deep learning has drawn lots of attention on automatically learning features from data. In this study, we proposed multi-subspace neural network (MSNN) which integrates key components of the convolutional neural network (CNN), receptive field, with subspace concept. Associating subspace with the deep network is a novel designing, providing various viewpoints …
Transformer-Progressive Mamba Network for Lightweight Image Super-Resolution
Bell Honors School, Nanjing University of Posts and Telecommunications（南京邮电大学贝尔荣誉学院）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学集成服务网络国家重点实验室）；Pattern Recognition and Intelligent System Laboratory, School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院模式识别与智能系统实验室）；Key Lab of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院智能感知与高维信息系统教育部重点实验室）；Department of Electrical Engineering and the Institute of Communications Engineering, National Tsing Hua University（国立交通大学电子工程系与通信工程研究所）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；IBM Research AI（IBM 研究院人工智能部门）
Fine-grained action detection is an important task with numerous applications in robotics and human-computer interaction. Existing methods typically utilize a two-stage approach including extraction of local spatio-temporal features followed by temporal modeling to capture long-term dependencies. While most recent papers have focused on the latter (long-temporal modeling), here, we focus on producing features capable of modeling fine-grained motion more efficiently. We propose a novel locally-consistent deformable convolution, which utilizes the change in receptive fields and enforces a local …
Beijing Institute of Technology（北京理工大学）；University of Rochester（罗切斯特大学）
How to model fine-grained spatial-temporal dynamics in videos has been a challenging problem for action recognition. It requires learning deep and rich features with superior distinctiveness for the subtle and abstract motions. Most existing methods generate features of a layer in a pure feedforward manner, where the information moves in one direction from inputs to outputs. And they rely on stacking more layers to obtain more powerful features, bringing extra non-negligible overheads. In this paper, we propose an Adaptive Recursive Circle (ARC) framework, a fine-grained decorator for pure fee…
3D object detection with LiDAR point clouds plays an important role in autonomous driving perception module that requires high speed, stability and accuracy. However, the existing point-based methods are challenging to reach the speed requirements because of too many raw points, and the voxel-based methods are unable to ensure stable speed because of the 3D sparse convolution. In contrast, the 2D grid-based methods, such as PointPillar, can easily achieve a stable and efficient speed based on simple 2D convolution, but it is hard to get the competitive accuracy limited by the coarse-grained po…
Sun Yat-sen University（中山大学）；SenseTime Research（商汤科技研究院）；Guilin University of Electronic Technology（桂林电子科技大学）
Humans can naturally understand an image in depth with the aid of rich knowledge accumulated from daily lives or professions. For example, to achieve fine-grained image recognition (e.g., categorizing hundreds of subordinate categories of birds) usually requires a comprehensive visual concept organization including category labels and part-level attributes. In this work, we investigate how to unify rich professional knowledge with deep neural network architectures and propose a Knowledge-Embedded Representation Learning (KERL) framework for handling the problem of fine-grained image recognitio…
Fine-Grained Visual Classification (FGVC) is known as a challenging task due to subtle differences among subordinate categories. Many current FGVC approaches focus on identifying and locating discriminative regions by using the attention mechanism, but neglect the presence of unnecessary features that hinder the understanding of object structure. These unnecessary features, including 1) ambiguous parts resulting from the visual similarity in object appearances and 2) noninformative parts (e.g., background noise), can have a significant adverse impact on classification results. In this paper, w…
Malaviya National Institute of Technology Jaipur（斋浦尔马拉维亚国家理工学院）；olx people（OLX人事）
The HGR is a quite challenging task as its performance is influenced by various aspects such as illumination variations, cluttered backgrounds, spontaneous capture, etc. The conventional CNN networks for HGR are following two stage pipeline to deal with the various challenges: complex signs, illumination variations, complex and cluttered backgrounds. The existing approaches needs expert expertise as well as auxiliary computation at stage 1 to remove the complexities from the input images. Therefore, in this paper, we proposes an novel end-to-end compact CNN framework: fine grained feature atte…
ECT: Fine-grained Edge Detection with Learned Cause Tokens
Xiamen University（厦门大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院（AIR））；Tsinghua University（清华大学）；Beihang University（北京航空航天大学）；Peking University（北京大学）；Intel Labs（英特尔实验室）
Peking University（北京大学）；China North Vehicle Research Institute（中国北方车辆研究所）
Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot t…
In fine-grained image recognition (FGIR), the localization and amplification of region attention is an important factor, which has been explored a lot by convolutional neural networks (CNNs) based approaches. The recently developed vision transformer (ViT) has achieved promising results on computer vision tasks. Compared with CNNs, Image sequentialization is a brand new manner. However, ViT is limited in its receptive field size and thus lacks local attention like CNNs due to the fixed size of its patches, and is unable to generate multi-scale features to learn discriminative region attention.…
School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Instrumentation Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学仪器科学与工程学院）；Harbin Institute of Technology Suzhou Research Institute（哈尔滨工业大学苏州研究院）；Ningbo Haichuang Group Co., Ltd.（宁波海创集团有限公司）；School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Information Engineering, North China University of Water Resources and Electric Power（中国北方水利电力大学信息工程学院）；Department of Geo-spatial Information, Information Engineering University（信息工程大学地理信息学院）；Innovation Center of Yangtze River Delta, Zhejiang University（长三角创新中心，浙江大学）
SenseTime X-Lab（商汤科技X实验室）；The Chinese University of Hong Kong（香港中文大学）；Beihang University（北京航空航天大学）；National Computer network Emergency Response technical Team/Coordination Center of China（国家计算机网络应急技术处理协调中心）
The small receptive field and capacity of minimal neural networks limit their performance when using them to be the backbone of detectors. In this work, we find that the appearance feature of a generic face is discriminative enough for a tiny and shallow neural network to verify from the background. And the essential barriers behind us are 1) the vague definition of the face bounding box and 2) tricky design of anchor-boxes or receptive field. Unlike most top-down methods for joint face detection and alignment, the proposed KPNet detects small facial keypoints instead of the whole face by in a…
Tsinghua University（清华大学）；University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）
Masked image modeling (MIM) as pre-training is shown to be effective for numerous vision downstream tasks, but how and where MIM works remain unclear. In this paper, we compare MIM with the long-dominant supervised pre-trained models from two perspectives, the visualizations and the experiments, to uncover their key representational differences. From the visualizations, we find that MIM brings locality inductive bias to all layers of the trained models, but supervised models tend to focus locally at lower layers but more globally at higher layers. That may be the reason why MIM helps Vision Tr…
Image inpainting techniques have shown promising improvement with the assistance of generative adversarial networks (GANs) recently. However, most of them often suffered from completed results with unreasonable structure or blurriness. To mitigate this problem, in this paper, we present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. Benefited from the property of this network, we can more easily recover large regions in an incomplete image. To better train this efficient generator, except for frequently-used VGG …
National University of Singapore（新加坡国立大学）；South China University of Technology（华南理工大学）；ByteDance Inc.（字节跳动公司）
Recent Vision Transformer~(ViT) models have demonstrated encouraging results across various computer vision tasks, thanks to their competence in modeling long-range dependencies of image patches or tokens via self-attention. These models, however, usually designate the similar receptive fields of each token feature within each layer. Such a constraint inevitably limits the ability of each self-attention layer in capturing multi-scale features, thereby leading to performance degradation in handling images with multiple objects of different scales. To address this issue, we propose a novel and g…
German Research Center for Artificial Intelligence (DFKI) Saarbrücken（德国人工智能研究中心（DFKI）萨尔布吕肯）；Saarland University（萨尔大学）；Max-Planck Institute for Informatics（马克斯-普朗克计算机科学研究所）
Human motion capture data has been widely used in data-driven character animation. In order to generate realistic, natural-looking motions, most data-driven approaches require considerable efforts of pre-processing, including motion segmentation and annotation. Existing (semi-) automatic solutions either require hand-crafted features for motion segmentation or do not produce the semantic annotations required for motion synthesis and building large-scale motion databases. In addition, human labeled annotation data suffers from inter- and intra-labeler inconsistencies by design. We propose a sem…
Inception Institute of Artificial Intelligence（因赛普申人工智能研究院）；Beijing BeYes Technology Co. Ltd.（北京贝叶斯科技有限公司）
Diabetic retinopathy (DR) is a complication of diabetes that severely affects eyes. It can be graded into five levels of severity according to international protocol. However, optimizing a grading model to have strong generalizability requires a large amount of balanced training data, which is difficult to collect particularly for the high severity levels. Typical data augmentation methods, including random flipping and rotation, cannot generate data with high diversity. In this paper, we propose a diabetic retinopathy generative adversarial network (DR-GAN) to synthesize high-resolution fundu…
Beijing University of Posts and Telecommunications（北京邮电大学）
Transformer models have achieved promising performances in point cloud segmentation. However, most existing attention schemes provide the same feature learning paradigm for all points equally and overlook the enormous difference in size among scene objects. In this paper, we propose the Size-Aware Transformer (SAT) that can tailor effective receptive fields for objects of different sizes. Our SAT achieves size-aware learning via two steps: introduce multi-scale features to each attention layer and allow each point to choose its attentive fields adaptively. It contains two key designs: the Mult…
Beihang University（北京航空航天大学）；Xiaomi AI Lab（小米人工智能实验室）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
Continuous sign language recognition (cSLR) is a public significant task that transcribes a sign language video into an ordered gloss sequence. It is important to capture the fine-grained gloss-level details, since there is no explicit alignment between sign video frames and the corresponding glosses. Among the past works, one promising way is to adopt a one-dimensional convolutional network (1D-CNN) to temporally fuse the sequential frames. However, CNNs are agnostic to similarity or dissimilarity, and thus are unable to capture local consistent semantics within temporally neighboring frames.…
PMR-Net: Parallel Multi-Resolution Encoder-Decoder Network Framework for Medical Image Segmentation
Shaanxi University of Science and Technology（陕西科技大学）；School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology（陕西科技大学电子信息与人工智能学院）
MSVS-VAE: Multi-Scale Anchored VecSet for High-Fidelity 3D Reconstruction
The Hong Kong University of Science and Technology（香港科技大学）；Peking University（北京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Lightspeed Studios（光速工作室）
Frequency-enhanced Multi-granularity Context Network for Efficient Vertebrae Segmentation
School of Software Technology \& DUT-RU International School of Information Science；Engineering, Dalian University of Technology, Dalian, China School of Future Technology, School of Artificial Intelligence, Dalian University of Technology, Dalian, China The Fifth Affiliated Hospital of Zhengzhou University, Zhengzhou, China College of Computer Science；Engineering, Shandong University of Science
