Object detection is an important task in computer vision which serves a lot of real-world applications such as autonomous driving, surveillance and robotics. Along with the rapid thrive of large-scale data, numerous state-of-the-art generalized object detectors (e.g. Faster R-CNN, YOLO, SSD) were developed in the past decade. Despite continual efforts in model modification and improvement in training strategies to boost detection accuracy, there are still limitations in performance of detectors when it comes to specialized datasets with uneven object class distributions. This originates from the common usage of Cross Entropy loss function for object classification sub-task that simply ignores the frequency of appearance of object class during training, and thus results in lower accuracies for object classes with fewer number of samples. Class-imbalance in general machine learning has been widely studied, however, little attention has been paid on the subject of object detection. In this paper, we propose to explore and overcome such problem by application of several weighted variants of Cross Entropy loss, for examples Balanced Cross Entropy, Focal Loss and Class-Balanced Loss Based on Effective Number of Samples to our object detector. Experiments with BDD100K (a highly class-imbalanced driving database acquired from on-vehicle cameras capturing mostly Car-class objects and other minority object classes such as Bus, Person and Motor) have proven better class-wise performances of detector trained with the afore-mentioned loss functions.
Korea University（高丽大学）；Hyundai Motor Company（现代汽车公司）
An autonomous driving system requires a 3D object detector, which must perceive all present road agents reliably to navigate an environment safely. However, real-world driving datasets often suffer from the problem of data imbalance, which causes difficulties in training a model that works well across all classes, resulting in an undesired imbalanced sub-optimal performance. In this work, we propose a method to address this data imbalance problem. Our method consists of two main components: (i) a LiDAR-based 3D object detector with per-class multiple detection heads where losses from each head…
Size-invariance Matters: Rethinking Metrics and Losses for Imbalanced Multi-object Salient Object Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Cyber Science and Tech., Shenzhen Campus, Sun Yat-sen University（中山大学深圳校区网络空间科学与技术学院）；Key Laboratory of Big Data Mining and Knowledge Management, Chinese Academy of Sciences（中国科学院大数据挖掘与知识管理重点实验室）
National Tsing Hua University（台湾清华大学）；Virginia Tech（弗吉尼亚理工大学）；Aeolus Robotics
Long-tailed class distributions are prevalent among the practical applications of object detection and instance segmentation. Prior work in long-tail instance segmentation addresses the imbalance of losses between rare and frequent categories by reducing the penalty for a model incorrectly predicting a rare class label. We demonstrate that the rare categories are heavily suppressed by correct background predictions, which reduces the probability for all foreground categories with equal weight. Due to the relative infrequency of rare categories, this leads to an imbalance that biases towards pr…
Tongji University（同济大学）；Sensetime Research（商汤科技研究院）；University of Sydney（悉尼大学）；Shanghai AI Laboratory（上海人工智能实验室）
Long-tail distribution is widely spread in real-world applications. Due to the extremely small ratio of instances, tail categories often show inferior accuracy. In this paper, we find such performance bottleneck is mainly caused by the imbalanced gradients, which can be categorized into two parts: (1) positive part, deriving from the samples of the same category, and (2) negative part, contributed by other categories. Based on comprehensive experiments, it is also observed that the gradient ratio of accumulated positives to negatives is a good indicator to measure how balanced a category is tr…
Universitat Pompeu Fabra（庞培法布拉大学）；University of Adelaide（阿德莱德大学）；Catalan Institution for Research and Advanced Studies (ICREA)（加泰罗尼亚研究与高级研究所（ICREA））
We study the impact of different loss functions on lesion segmentation from medical images. Although the Cross-Entropy (CE) loss is the most popular option when dealing with natural images, for biomedical image segmentation the soft Dice loss is often preferred due to its ability to handle imbalanced scenarios. On the other hand, the combination of both functions has also been successfully applied in this kind of tasks. A much less studied problem is the generalization ability of all these losses in the presence of Out-of-Distribution (OoD) data. This refers to samples appearing in test time t…
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）
Existing methods for object detection in UAV images ignored an important challenge - imbalanced class distribution in UAV images - which leads to poor performance on tail classes. We systematically investigate existing solutions to long-tail problems and unveil that re-balancing methods that are effective on natural image datasets cannot be trivially applied to UAV datasets. To this end, we rethink long-tailed object detection in UAV images and propose the Dual Sampler and Head detection Network (DSHNet), which is the first work that aims to resolve long-tail distribution in UAV images. The ke…
CASIA（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Horizon Robotics（地平线机器人）
In object detection, bounding box regression (BBR) is a crucial step that determines the object localization performance. However, we find that most previous loss functions for BBR have two main drawbacks: (i) Both $\ell_n$-norm and IOU-based loss functions are inefficient to depict the objective of BBR, which leads to slow convergence and inaccurate regression results. (ii) Most of the loss functions ignore the imbalance problem in BBR that the large number of anchor boxes which have small overlaps with the target boxes contribute most to the optimization of BBR. To mitigate the adverse effec…
University of Cambridge（剑桥大学）；University of Salerno（萨莱诺大学）
Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentati…
Object detection is a typical multi-task learning application, which optimizes classification and regression simultaneously. However, classification loss always dominates the multi-task loss in anchor-based methods, hampering the consistent and balanced optimization of the tasks. In this paper, we find that shifting the bounding boxes can change the division of positive and negative samples in classification, meaning classification depends on regression. Moreover, we summarize three important conclusions about fine-tuning loss weights, considering different datasets, optimizers and regression …
Medical image classification is often challenging for two reasons: a lack of labelled examples due to expensive and time-consuming annotation protocols, and imbalanced class labels due to the relative scarcity of disease-positive individuals in the wider population. Semi-supervised learning (SSL) methods exist for dealing with a lack of labels, but they generally do not address the problem of class imbalance. In this study we propose Adaptive Blended Consistency Loss (ABCL), a drop-in replacement for consistency loss in perturbation-based SSL methods. ABCL counteracts data skew by adaptively m…
Xi'an Jiaotong University（西安交通大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Fog and haze are weathers with low visibility which are adversarial to the driving safety of intelligent vehicles equipped with optical sensors like cameras and LiDARs. Therefore image dehazing for perception enhancement and haze image synthesis for testing perception abilities are equivalently important in the development of such autonomous driving systems. From the view of image translation, these two problems are essentially dual with each other, which have the potentiality to be solved jointly. In this paper, we propose an unsupervised Image-to-Image Translation framework based on Variatio…
University of Toronto（多伦多大学）；Mila（米拉研究所）；Université de Montréal（蒙特利尔大学）
An effective framework for learning 3D representations for perception tasks is distilling rich self-supervised image features via contrastive learning. However, image-to point representation learning for autonomous driving datasets faces two main challenges: 1) the abundance of self-similarity, which results in the contrastive losses pushing away semantically similar point and image regions and thus disturbing the local semantic structure of the learned representations, and 2) severe class imbalance as pretraining gets dominated by over-represented classes. We propose to alleviate the self-sim…
Harbin Institute of Technology（哈尔滨工业大学）；Peng Cheng Laboratory（鹏城实验室）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Gaoling School of Artificial Intelligence, Renmin University of China（中国人民大学高瓴人工智能学院）；Tsinghua University（清华大学）
One of the main challenges for feature representation in deep learning-based classification is the design of appropriate loss functions that exhibit strong discriminative power. The classical softmax loss does not explicitly encourage discriminative learning of features. A popular direction of research is to incorporate margins in well-established losses in order to enforce extra intra-class compactness and inter-class separability, which, however, were developed through heuristic means, as opposed to rigorous mathematical principles. In this work, we attempt to address this limitation by form…
VICO Group, University of Edinburgh（爱丁堡大学VICO研究组）；University of Edinburgh（爱丁堡大学）；United Kingdom（英国）
Multi-task learning (MTL) is to learn one single model that performs multiple tasks for achieving good performance on all tasks and lower cost on computation. Learning such a model requires to jointly optimize losses of a set of tasks with different difficulty levels, magnitudes, and characteristics (e.g. cross-entropy, Euclidean loss), leading to the imbalance problem in multi-task learning. To address the imbalance problem, we propose a knowledge distillation based method in this work. We first learn a task-specific model for each task. We then learn the multi-task model for minimizing task-…
Dynamic Residual Classifier for Class Incremental Learning
Sun Yat-sen University（中山大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）；Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education（教育部机器智能与先进计算重点实验室）
Beyond scalar losses: calibrating segmentation models via gradient vector field surgery
School of Computation, Information and Technology, TUM（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Department of Radiology, Weill Cornell Medicine（威尔康乃尔医学院放射科）；School of Medicine and Health, TUM University Hospital（慕尼黑工业大学医院医学与健康学院）；Cornell Tech（康奈尔科技学院）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）
Benchmarking Foundation Models and Parameter-Efficient Fine-Tuning for Prognosis Prediction in Medical Imaging
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma（人工智能与计算机系统单位，工程系，罗马生物医学大学）；College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University（诊断与介入系，辐射物理，生物医学工程，乌梅大学）
A Unified Perspective for Loss-Oriented Imbalanced Learning via Localization
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Artificial Intelligence Institute of China Electronics Technology Group Corporation（中国电子科技集团有限公司人工智能研究院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络科学与技术学院）；Key Laboratory of Big Data Mining and Knowledge Management (BDKM), University of Chinese Academy of Sciences（大数据挖掘与知识管理关键实验室（BDKM），中国科学院大学）
Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学伊萨尔河畔医院）
Osteoporotic vertebral fractures have a severe impact on patients' overall well-being but are severely under-diagnosed. These fractures present themselves at various levels of severity measured using the Genant's grading scale. Insufficient annotated datasets, severe data-imbalance, and minor difference in appearances between fractured and healthy vertebrae make naive classification approaches result in poor discriminatory performance. Addressing this, we propose a representation learning-inspired approach for automated vertebral fracture detection, aimed at learning latent representations eff…
