少样本图像分类已成为计算机视觉领域的一项关键挑战，它凸显了利用极少标注数据快速适应新任务的能力。现有方法主要依赖图像级特征或局部描述子，却往往忽视这些描述子周围的整体上下文。在本研究中，我们提出了一种名为“带上下文增强的局部描述子（Local Descriptor with Contextual Augmentation，LDCA）”的新方法。具体而言，该方法通过利用自适应全局上下文增强模块，以独特方式弥合局部理解与全局理解之间的差距。该模块集成了视觉 Transformer，使局部描述子具备从广泛全局视角到复杂周围细微差别的上下文感知能力。通过这种方式，LDCA 超越了传统的基于描述子的方法，确保每个局部特征都在其更宏大的视觉叙事中得到解释。大量实验证明了本方法的有效性：在细粒度分类数据集上，相比次优方法最高取得20%的绝对提升，从而表明其在少样本分类任务中的显著进展。
英文摘要：
Few-shot image classification has emerged as a key challenge in the field of computer vision, highlighting the capability to rapidly adapt to new tasks with minimal labeled data. Existing methods predominantly rely on image-level features or local descriptors, often overlooking the holistic context surrounding these descriptors. In this work, we introduce a novel approach termed "Local Descriptor with Contextual Augmentation (LDCA)". Specifically, this method bridges the gap between local and global understanding uniquely by leveraging an adaptive global contextual enhancement module. This module incorporates a visual transformer, endowing local descriptors with contextual awareness capabilities, ranging from broad global perspectives to intricate surrounding nuances. By doing so, LDCA transcends traditional descriptor-based approaches, ensuring each local feature is interpreted within its larger visual narrative. Extensive experiments underscore the efficacy of our method, showing a maximal absolute improvement of 20\% over the next-best on fine-grained classification datasets, thus demonstrating significant advancements in few-shot classification tasks.
University of Southern Denmark（南丹麦大学）；Tampere University of Technology（坦佩雷理工大学）
We address the problem of estimating the alignment pose between two models using structure-specific local descriptors. Our descriptors are generated using a combination of 2D image data and 3D contextual shape data, resulting in a set of semi-local descriptors containing rich appearance and shape information for both edge and texture structures. This is achieved by defining feature space relations which describe the neighborhood of a descriptor. By quantitative evaluations, we show that our descriptors provide high discriminative power compared to state of the art approaches. In addition, we s…
School of Computer Science and Electronic Engineering, University of Essex（埃塞克斯大学计算机科学与电子工程学院）；School of Electrical Engineering and Computer Science, Queensland University of Technology（昆士兰科技大学电气工程与计算机科学学院）
Visual navigation localizes a query place image against a reference database of place images, also known as a `visual map'. Localization accuracy requirements for specific areas of the visual map, `scene classes', vary according to the context of the environment and task. State-of-the-art visual mapping is unable to reflect these requirements by explicitly targetting scene classes for inclusion in the map. Four different scene classes, including pedestrian crossings and stations, are identified in each of the Nordland and St. Lucia datasets. Instead of re-training separate scene classifiers wh…
Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Shenzhen Zhuke Innovation Technology (Altizure)（深圳珠科创新技术（Altizure））
Most existing studies on learning local features focus on the patch-based descriptions of individual keypoints, whereas neglecting the spatial relations established from their keypoint locations. In this paper, we go beyond the local detail representation by introducing context awareness to augment off-the-shelf local feature descriptors. Specifically, we propose a unified learning framework that leverages and aggregates the cross-modality contextual information, including (i) visual context from high-level image representation, and (ii) geometric context from 2D keypoint distribution. Moreove…
Point cloud based retrieval for place recognition is an emerging problem in vision field. The main challenge is how to find an efficient way to encode the local features into a discriminative global descriptor. In this paper, we propose a Point Contextual Attention Network (PCAN), which can predict the significance of each local point feature based on point context. Our network makes it possible to pay more attention to the task-relevent features when aggregating local features. Experiments on various benchmark datasets show that the proposed network can provide outperformance than current sta…
The paper presents a learning-based method for computing a discriminative 3D point cloud descriptor for place recognition purposes. Existing methods, such as PointNetVLAD, are based on unordered point cloud representation. They use PointNet as the first processing step to extract local features, which are later aggregated into a global descriptor. The PointNet architecture is not well suited to capture local geometric structures. Thus, state-of-the-art methods enhance vanilla PointNet architecture by adding different mechanism to capture local contextual information, such as graph convolutiona…
University of California San Diego（加州大学圣地亚哥分校）；Carnegie Mellon University（卡内基梅隆大学）
LiDAR-based localization approach is a fundamental module for large-scale navigation tasks, such as last-mile delivery and autonomous driving, and localization robustness highly relies on viewpoints and 3D feature extraction. Our previous work provides a viewpoint-invariant descriptor to deal with viewpoint differences; however, the global descriptor suffers from a low signal-noise ratio in unsupervised clustering, reducing the distinguishable feature extraction ability. We develop SphereVLAD++, an attention-enhanced viewpoint invariant place recognition method in this work. SphereVLAD++ proje…
Lidar datasets are becoming more and more common. They are appreciated for their precise 3D nature, and have a wide range of applications, such as surface reconstruction, object detection, visualisation, etc. For all this applications, having additional semantic information per point has potential of increasing the quality and the efficiency of the application. In the last decade the use of Machine Learning and more specifically classification methods have proved to be successful to create this semantic information. In this paradigm, the goal is to classify points into a set of given classes (…
University of California, Berkeley（加州大学伯克利分校）；Tsinghua University（清华大学）；ICSI, Berkeley（伯克利国际计算机科学研究所）；National University of Singapore（新加坡国立大学）；University of Massachusetts, Lowell（马萨诸塞大学洛厄尔分校）
In this paper, we address the task of natural language object retrieval, to localize a target object within a given image based on a natural language query of the object. Natural language object retrieval differs from text-based image retrieval task as it involves spatial information about objects within the scene and global scene context. To address this issue, we propose a novel Spatial Context Recurrent ConvNet (SCRC) model as scoring function on candidate boxes for object retrieval, integrating spatial configurations and global scene-level contextual information into the network. Our model…
Nanjing University of Science and Technology（南京理工大学）；University of Macau（澳门大学）
Place recognition or loop closure detection is one of the core components in a full SLAM system. In this paper, aiming at strengthening the relevancy of local neighboring points and the contextual dependency among global points simultaneously, we investigate the exploitation of transformer-based network for feature extraction, and propose a Hierarchical Transformer for Place Recognition (HiTPR). The HiTPR consists of four major parts: point cell generation, short-range transformer (SRT), long-range transformer (LRT) and global descriptor aggregation. Specifically, the point cloud is initially …
Robust Saliency-Aware Distillation for Few-shot Fine-grained Visual Recognition
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Brain and Affective Cognitive Research Center, Pazhou Lab（琶洲实验室脑与情感认知研究中心）；Engineering Research Center of the Ministry of Education on Health Intelligent Perception and Paralleled Digital-Human（教育部健康智能感知与平行数字人工程研究中心）
