Existing long-tailed recognition methods, aiming to train class-balanced models from long-tailed data, generally assume the models would be evaluated on the uniform test class distribution. However, practical test class distributions often violate this assumption (e.g., being either long-tailed or even inversely long-tailed), which may lead existing methods to fail in real applications. In this paper, we study a more practical yet challenging task, called test-agnostic long-tailed recognition, where the training class distribution is long-tailed while the test class distribution is agnostic and not necessarily uniform. In addition to the issue of class imbalance, this task poses another challenge: the class distribution shift between the training and test data is unknown. To tackle this task, we propose a novel approach, called Self-supervised Aggregation of Diverse Experts, which consists of two strategies: (i) a new skill-diverse expert learning strategy that trains multiple experts from a single and stationary long-tailed dataset to separately handle different class distributions; (ii) a novel test-time expert aggregation strategy that leverages self-supervision to aggregate the learned multiple experts for handling unknown test class distributions. We theoretically show that our self-supervised strategy has a provable ability to simulate test-agnostic class distributions. Promising empirical results demonstrate the effectiveness of our method on both vanilla and test-agnostic long-tailed recognition. Code is available at \url{https://github.com/Vanint/SADE-AgnosticLT}.
Disentangling Hardness from Noise: An Uncertainty-Driven Model-Agnostic Framework for Long-Tailed Remote Sensing Classification
Research Center for Space Computing System, Zhejiang Lab（空间计算系统研究中心，浙江实验室）；School of Cyber Science and Engineering, Zhengzhou University（网络安全工程学院，郑州大学）；School of Computer Science and Technology, Xi'an Jiaotong University（计算机科学与技术学院，西安交通大学）
A dramatic increase in real-world video volume with extremely diverse and emerging topics naturally forms a long-tailed video distribution in terms of their categories, and it spotlights the need for Video Long-Tailed Recognition (VLTR). In this work, we summarize the challenges in VLTR and explore how to overcome them. The challenges are: (1) it is impractical to re-train the whole model for high-quality features, (2) acquiring frame-wise labels requires extensive cost, and (3) long-tailed data triggers biased training. Yet, most existing works for VLTR unavoidably utilize image-level feature…
University of Science and Technology of China（科学技术大学）；Hefei University of Technology（合肥工业大学）；IAI, Hefei Comprehensive National Science Center（IAI合肥国家科学中心）
In this work, we address the challenging task of long-tailed image recognition. Previous long-tailed recognition methods commonly focus on the data augmentation or re-balancing strategy of the tail classes to give more attention to tail classes during the model training. However, due to the limited training images for tail classes, the diversity of tail class images is still restricted, which results in poor feature representations. In this work, we hypothesize that common latent features among the head and tail classes can be used to give better feature representation. Motivated by this, we i…
As few-shot object detectors are often trained with abundant base samples and fine-tuned on few-shot novel examples,the learned models are usually biased to base classes and sensitive to the variance of novel examples. To address this issue, we propose a meta-learning framework with two novel feature aggregation schemes. More precisely, we first present a Class-Agnostic Aggregation (CAA) method, where the query and support features can be aggregated regardless of their categories. The interactions between different classes encourage class-agnostic representations and reduce confusion between b…
Rectify the Regression Bias in Long-Tailed Object Detection
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）
Object detection and motion parameters estimation are crucial tasks for self-driving vehicle safe navigation in a complex urban environment. In this work we propose a novel real-time approach of temporal context aggregation for motion detection and motion parameters estimation based on 3D point cloud sequence. We introduce an ego-motion compensation layer to achieve real-time inference with performance comparable to a naive odometric transform of the original point cloud sequence. Not only is the proposed architecture capable of estimating the motion of common road participants like vehicles o…
Class imbalance and noisy labels are the norm rather than the exception in many large-scale classification datasets. Nevertheless, most works in machine learning typically assume balanced and clean data. There have been some recent attempts to tackle, on one side, the problem of learning from noisy labels and, on the other side, learning from long-tailed data. Each group of methods make simplifying assumptions about the other. Due to this separation, the proposed solutions often underperform when both assumptions are violated. In this work, we present a simple two-stage approach based on recen…
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；DiDi Chuxing（滴滴出行）；Shanghai AI Laboratory（上海人工智能实验室）
Predicting the future trajectories of the traffic agents is a gordian technique in autonomous driving. However, trajectory prediction suffers from data imbalance in the prevalent datasets, and the tailed data is often more complicated and safety-critical. In this paper, we focus on dealing with the long-tail phenomenon in trajectory prediction. Previous methods dealing with long-tail data did not take into account the variety of motion patterns in the tailed data. In this paper, we put forward a future enhanced contrastive learning framework to recognize tail trajectory patterns and form a fea…
Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling
Graduate Institute of Electrical Engineering, National Taiwan University, Taiwan（台湾大学电子工程研究所）；Institute of Information Science, Academia Sinica, Taiwan（中科院资讯研究所）；Graduate Institute of Communication Engineering, National Taiwan University, Taiwan（台湾大学通讯工程研究所）；Institute of Information Science and the Research Center for Information Technology Innovation, Academia Sinica, Taiwan（中科院资讯研究所及资讯科技创新研究中心）
Heterogeneous face recognition is a challenging task due to the large modality discrepancy and insufficient cross-modal samples. Most existing works focus on discriminative feature transformation, metric learning and cross-modal face synthesis. However, the fact that cross-modal faces are always coupled by domain (modality) and identity information has received little attention. Therefore, how to learn and utilize the domain-private feature and domain-agnostic feature for modality adaptive face recognition is the focus of this work. Specifically, this paper proposes a Feature Aggregation Netwo…
Towards Distribution-Agnostic Generalized Category Discovery
Zhejiang University（浙江大学）；The Hong Kong University of Science and Technology（香港科技大学）；Centre for Frontier AI Research, A*STAR（新加坡科技研究局前沿人工智能研究中心）；Angelalign Technology Inc.（时代天使科技有限公司）
Most of the current action recognition algorithms are based on deep networks which stack multiple convolutional, pooling and fully connected layers. While convolutional and fully connected operations have been widely studied in the literature, the design of pooling operations that handle action recognition, with different sources of temporal granularity in action categories, has comparatively received less attention, and existing solutions rely mainly on max or averaging operations. The latter are clearly powerless to fully exhibit the actual temporal granularity of action categories and there…
BroadCAM: Outcome-agnostic Class Activation Mapping for Small-scale Weakly Supervised Applications
South China University of Technology（华南理工大学）；Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences)（广东省人民医院（广东省医学科学院））；Southern Medical University（南方医科大学）；The Chinese University of Hong Kong（香港中文大学）
A large labeled dataset is a key to the success of supervised deep learning, but for medical image segmentation, it is highly challenging to obtain sufficient annotated images for model training. In many scenarios, unannotated images are abundant and easy to acquire. Self-supervised learning (SSL) has shown great potentials in exploiting raw data information and representation learning. In this paper, we propose Hierarchical Self-Supervised Learning (HSSL), a new self-supervised framework that boosts medical image segmentation by making good use of unannotated data. Unlike the current literatu…
A cross-domain visual place recognition (VPR) task is proposed in this work, i.e., matching images of the same architectures depicted in different domains. VPR is commonly treated as an image retrieval task, where a query image from an unknown location is matched with relevant instances from geo-tagged gallery database. Different from conventional VPR settings where the query images and gallery images come from the same domain, we propose a more common but challenging setup where the query images are collected under a new unseen condition. The two domains involved in this work are contemporary…
Stanford University（斯坦福大学）；University of Pennsylvania（宾夕法尼亚大学）
To generalize well, classifiers must learn to be invariant to nuisance transformations that do not alter an input's class. Many problems have "class-agnostic" nuisance transformations that apply similarly to all classes, such as lighting and background changes for image classification. Neural networks can learn these invariances given sufficient data, but many real-world datasets are heavily class imbalanced and contain only a few examples for most of the classes. We therefore pose the question: how well do neural networks transfer class-agnostic invariances learned from the large classes to t…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We introduce REDO, a class-agnostic framework to REconstruct the Dynamic Objects from RGBD or calibrated videos. Compared to prior work, our problem setting is more realistic yet more challenging for three reasons: 1) due to occlusion or camera settings an object of interest may never be entirely visible, but we aim to reconstruct the complete shape; 2) we aim to handle different object dynamics including rigid motion, non-rigid motion, and articulation; 3) we aim to reconstruct different categories of objects with one unified framework. To address these challenges, we develop two novel module…
Exploring "Many in Few" and "Few in Many" Properties in Long-Tailed, Highly-Imbalanced IC Defect Classification
Institute of Data Science and Information Computing, National Chung Hsing University（数据科学与信息计算研究所，中国台湾 Chung Hsing 大学）；Department of Electrical Engineering, National Tsing Hua University（电子工程系，中国台湾 Tsing Hua 大学）；Department of Electrical Engineering, National Taiwan University of Science and Technology（电子工程系，中国台湾 Science and Technology 大学）；Development of Smart Manufacturing, United Microelectronics Corporation（智能制造发展，联合微电子公司）
H3Former: Hypergraph-based Semantic-Aware Aggregation via Hyperbolic Hierarchical Contrastive Loss for Fine-Grained Visual Classification
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；College of Software, Jilin University（吉林大学软件学院）；School of Software, Tsinghua University（清华大学软件学院）
FedReLa: Imbalanced Federated Learning via Re-Labeling
School of Computing and Information Systems, University of Melbourne, Victoria, Australia（墨尔本大学计算机与信息系统学院）；School of Statistics and Data Science, LPMC, KLMDASR, and LEBPS, Nankai University（南开大学统计与数据科学学院）；School of Mathematics and Statistics, University of Melbourne, Victoria, Australia（墨尔本大学数学与统计学学院）
