Learning from imbalanced data is one of the most significant challenges in real-world classification tasks. In such cases, neural networks performance is substantially impaired due to preference towards the majority class. Existing approaches attempt to eliminate the bias through data re-sampling or re-weighting the loss in the learning process. Still, these methods tend to overfit the minority samples and perform poorly when the structure of the minority class is highly irregular. Here, we propose a novel deep meta-learning technique to augment a given imbalanced dataset with new minority instances. These additional data are incorporated in the classifier's deep-learning process, and their contributions are learned explicitly. The advantage of the proposed method is demonstrated on synthetic and real-world datasets with various imbalance ratios.
University at Buffalo, SUNY（纽约州立大学布法罗分校）；Department of Computer Science and Engineering, University at Buffalo, SUNY（纽约州立大学布法罗分校计算机科学与工程系）
Recognizing new objects by learning from a few labeled examples in an evolving environment is crucial to obtain excellent generalization ability for real-world machine learning systems. A typical setting across current meta learning algorithms assumes a stationary task distribution during meta training. In this paper, we explore a more practical and challenging setting where task distribution changes over time with domain shift. Particularly, we consider realistic scenarios where task distribution is highly imbalanced with domain labels unavailable in nature. We propose a kernel-based method f…
Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning
Peng Cheng Laboratory（鹏城实验室）；School of Automation, Chongqing University（重庆大学自动化学院）；School of Information Science and Engineering, East China University of Science and Technology（东华大学信息科学与工程学院）
Delft University of Technology（代尔夫特理工大学）；California Institute of Technology（加州理工学院）
To alleviate lower classification performance on rare classes in imbalanced datasets, a possible solution is to augment the underrepresented classes with synthetic samples. Domain adaptation can be incorporated in a classifier to decrease the domain discrepancy between real and synthetic samples. While domain adaptation is generally applied on completely synthetic source domains and real target domains, we explore how domain adaptation can be applied when only a single rare class is augmented with simulated samples. As a testbed, we use a camera trap animal dataset with a rare deer class, whic…
University of New South Wales（新南威尔士大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61部门）
We regularly consider answering counterfactual questions in practice, such as "Would people with diabetes take a turn for the better had they choose another medication?". Observational studies are growing in significance in answering such questions due to their widespread accumulation and comparatively easier acquisition than Randomized Control Trials (RCTs). Recently, some works have introduced representation learning and domain adaptation into counterfactual inference. However, most current works focus on the setting of binary treatments. None of them considers that different treatments' sam…
Introduction Data imbalance is one of the crucial issues in big data analysis with fewer labels. For example, in real-world healthcare data, spam detection labels, and financial fraud detection datasets. Many data balance methods were introduced to improve machine learning algorithms' performance. Research claims SMOTE and SMOTE-based data-augmentation (generate new data points) methods could improve algorithm performance. However, we found in many online tutorials, the valuation methods were applied based on synthesized datasets that introduced bias into the evaluation, and the performance go…
Fudan University（复旦大学）；The University of Melbourne（墨尔本大学）
Evaluating the robustness of a defense model is a challenging task in adversarial robustness research. Obfuscated gradients have previously been found to exist in many defense methods and cause a false signal of robustness. In this paper, we identify a more subtle situation called Imbalanced Gradients that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a Margin Decomposition (MD) att…
University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）；Clemson University（克莱姆森大学）
ECG databases are usually highly imbalanced due to the abundance of Normal ECG and scarcity of abnormal cases. As such, deep learning classifiers trained on imbalanced datasets usually perform poorly, especially on minor classes. One solution is to generate realistic synthetic ECG signals using Generative Adversarial Networks (GAN) to augment imbalanced datasets. In this study, we combined conditional GAN with WGAN-GP and developed AC-WGAN-GP in 1D form for the first time to be applied on MIT-BIH Arrhythmia dataset. We investigated the impact of data augmentation on arrhythmia classification. …
Class imbalance distribution widely exists in real-world engineering. However, the mainstream optimization algorithms that seek to minimize error will trap the deep learning model in sub-optimums when facing extreme class imbalance. It seriously harms the classification precision, especially on the minor classes. The essential reason is that the gradients of the classifier weights are imbalanced among the components from different classes. In this paper, we propose Attraction-Repulsion-Balanced Loss (ARB-Loss) to balance the different components of the gradients. We perform experiments on the …
Kangwon National University（江原大学）；Bionsight, Inc.（Bionsight公司）
The $F_β$ score is a commonly used measure of classification performance, which plays crucial roles in classification tasks with imbalanced data sets. However, the $F_β$ score cannot be used as a loss function by gradient-based learning algorithms for optimizing neural network parameters due to its non-differentiability. On the other hand, commonly used loss functions such as the binary cross-entropy (BCE) loss are not directly related to performance measures such as the $F_β$ score, so that neural networks optimized by using the loss functions may not yield optimal performance measures. In th…
Cyber Security Data Science: Machine Learning Methods and their Performance on Imbalanced Datasets
Computational Systems Engineering, Universidad Privada Boliviana（计算系统工程，私人 Boliviana 大学）；Extended Artificial Intelligence, IU International University of Applied Sciences（扩展人工智能，IU 国际应用科学大学）
From the ad network standpoint, a user's activity is a multi-type sequence of temporal events consisting of event types and time intervals. Understanding user patterns in ad networks has received increasing attention from the machine learning community. Particularly, the problems of fraud detection, Conversion Rate (CVR), and Click-Through Rate (CTR) prediction are of interest. However, the class imbalance between major and minor classes in these tasks can bias a machine learning model leading to poor performance. This study proposes using two multi-type (continuous and discrete) training appr…
Virtual Big Data (VBD) proved to be effective to alleviate mode collapse and vanishing generator gradient as two major problems of Generative Adversarial Neural Networks (GANs) very recently. In this paper, we investigate the capability of VBD to address two other major challenges in Machine Learning including deep autoencoder training and imbalanced data classification. First, we prove that, VBD can significantly decrease the validation loss of autoencoders via providing them a huge diversified training data which is the key to reach better generalization to minimize the over-fitting problem.…
Brandeis University（布兰迪斯大学）；University of Hong Kong（香港大学）；University of Notre Dame（圣母大学）；Hunan University（湖南大学）
Even pruned by the state-of-the-art network compression methods, Graph Neural Networks (GNNs) training upon non-Euclidean graph data often encounters relatively higher time costs, due to its irregular and nasty density properties, compared with data in the regular Euclidean space. Another natural property concomitantly with graph is class-imbalance which cannot be alleviated by the massive graph data while hindering GNNs' generalization. To fully tackle these unpleasant properties, (i) theoretically, we introduce a hypothesis about what extent a subset of the training data can approximate the …
Nanjing University（南京大学）；The Ohio State University（俄亥俄州立大学）
Neural networks trained with class-imbalanced data are known to perform poorly on minor classes of scarce training data. Several recent works attribute this to over-fitting to minor classes. In this paper, we provide a novel explanation of this issue. We found that a neural network tends to first under-fit the minor classes by classifying most of their data into the major classes in early training epochs. To correct these wrong predictions, the neural network then must focus on pushing features of minor class data across the decision boundaries between major and minor classes, leading to much …
Quantum-Informed Contrastive Learning with Dynamic Mixup Augmentation for Class-Imbalanced Expert Systems
organization= Thomas Lord Department of Computer Science, Viterbi School of Engineering, University of Southern California , city= Los Angeles , state= CA , postcode= 90089 , country= USA；organization= Department of Industrial Engineering；Management, Khulna University of Engineering \& Technology (KUET) , city= Khulna , postcode= 9203 , country= Bangladesh；organization= Department of Computer Science, American International University-Bangladesh (AIUB) , city= Dhaka , postcode= 1229 , country= Bangladesh
Indian Institute of Technology Delhi（印度理工学院德里分校）；Max Hospital（马克斯医院）；All India Institute of Medical Sciences（全印度医学科学研究所）
The gold standard for COVID-19 is RT-PCR, testing facilities for which are limited and not always optimally distributed. Test results are delayed, which impacts treatment. Expert radiologists, one of whom is a co-author, are able to diagnose COVID-19 positivity from Chest X-Rays (CXR) and CT scans, that can facilitate timely treatment. Such diagnosis is particularly valuable in locations lacking radiologists with sufficient expertise and familiarity with COVID-19 patients. This paper has two contributions. One, we analyse literature on CXR based COVID-19 diagnosis. We show that popular choices…
Synthetic Augmentation in Imbalanced Learning: When It Helps, When It Hurts, and How Much to Add
Department of Electrical & Computer Engineering, Duke University（电气与计算机工程系，杜克大学）；Department of Biostatistics & Bioinformatics and Department of Computer Science, Duke University（生物统计学与生物信息学系和计算机科学系，杜克大学）
PROTOCOL: Partial Optimal Transport-enhanced Contrastive Learning for Imbalanced Multi-view Clustering
Shanghai Key Laboratory of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University, China（上海智能信息处理实验室，计算机科学与人工智能学院，复旦大学）；College of Computer Science and Technology, Qingdao University, China（计算机科学与技术学院，青岛大学）；Shanghai Key Laboratory of Navigation and Location-based Services, School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, China（上海导航与位置服务实验室，电子信息与电气工程学院，上海交通大学）；Institute of Science and Technology for Brain-Inspired Intelligence and Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University, China（脑启发智能科学技术研究院和计算神经科学与脑启发智能重点实验室，复旦大学）
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；McGill University（麦吉尔大学）；École Polytechnique de Montréal（蒙特利尔综合理工学院）
Popular approaches for minimizing loss in data-driven learning often involve an abstraction or an explicit retention of the history of gradients for efficient parameter updates. The aggregated history of gradients nudges the parameter updates in the right direction even when the gradients at any given step are not informative. Although the history of gradients summarized in meta-parameters or explicitly stored in memory has been shown effective in theory and practice, the question of whether $all$ or only a subset of the gradients in the history are sufficient in deciding the parameter updates…
University of Würzburg（维尔茨堡大学）；Univ. Lille（里尔大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Centrale Lille（里尔中央理工学院）；TU Darmstadt（达姆施塔特工业大学）；Hessian Center for Artificial Intelligence (Hessian.ai)（黑森人工智能中心）
TCG Crest Kolkata, India（印度加尔各答TCG Crest）；School of Computer Science, University College Dublin, Ireland（爱尔兰都柏林大学学院计算机科学学院）；Indian Statistical Institute Kolkata, India（印度统计研究所加尔各答分所）
There is often a mixture of very frequent labels and very infrequent labels in multi-label datatsets. This variation in label frequency, a type class imbalance, creates a significant challenge for building efficient multi-label classification algorithms. In this paper, we tackle this problem by proposing a minority class oversampling scheme, UCLSO, which integrates Unsupervised Clustering and Label-Specific data Oversampling. Clustering is performed to find out the key distinct and locally connected regions of a multi-label dataset (irrespective of the label information). Next, for each label,…
Sichuan University（四川大学）；Rutgers University – New Brunswick（罗格斯大学新不伦瑞克分校）；Zhejiang University（浙江大学）；Tennessee State University（田纳西州立大学）
The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on binary imbalanced data classification, a novel dynamic ensemble method, namely adaptive ensemble of classifiers with regularization (AER), is proposed, to overcome the stated limitations. The method solves the overfitting problem through implicit regularization. Specifically, it lev…
Enhancing Glass Defect Detection with Diffusion Models: Addressing Imbalanced Datasets in Manufacturing Quality Control
Department of Applied Statistics & Operations Research (ASOR), Bowling Green State University, Bowling Green, OH, USA（应用统计与运筹学系，布罗金格林州立大学）；Data Science Department, Actual Reality Technologies, OH, USA（数据科学系，Actual Reality Technologies）
University of Verona（维罗纳大学）；University of Nottingham（诺丁汉大学）
A supervised learning approach for the solution of large-scale nonlinear stabilization problems is presented. A stabilizing feedback law is trained from a dataset generated from State-dependent Riccati Equation solves. The training phase is enriched by the use gradient information in the loss function, which is weighted through the use of hyperparameters. High-dimensional nonlinear stabilization tests demonstrate that real-time sequential large-scale Algebraic Riccati Equation solves can be substituted by a suitably trained feedforward neural network.
Beyond Homoscedasticity: Decoupled Uncertainty Optimization for Deep Imbalanced Regression
School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；School of Mathematics and Statistics, Wuhan University（武汉大学数学与统计学院）；School of Synthetic Biology and Biomanufacturing, Tianjin University（天津大学合成生物学与生物制造学院）
School of Information Science and Technology（信息科学与技术学院）；College of Mathematics and Statistics（数学与统计学院）；ShanghaiTech University（上海科技大学）；Chongqing University（重庆大学）
For a class of nonsmooth composite optimization problems with linear equality constraints, we utilize a Lyapunov-based approach to establish the global exponential stability of the primal-dual gradient flow dynamics based on the proximal augmented Lagrangian. The result holds when the differentiable part of the objective function is strongly convex with a Lipschitz continuous gradient; the non-differentiable part is proper, lower semi-continuous, and convex; and the matrix in the linear constraint is full row rank. Our quadratic Lyapunov function generalizes recent result from strongly convex …
ASIL: Augmented Structural Information Learning for Deep Graph Clustering in Hyperbolic Space
Beijing University of Posts and Telecommunications（北京邮电大学）；North China Electric Power University（华北电力大学）；DiDi Chuxing（滴滴出行）；Beihang University（北航）；University of Illinois at Chicago（伊利诺伊大学香槟分校）
HyperSMOTE: A Hypergraph-based Oversampling Approach for Imbalanced Node Classifications
School of Information University of Michigan, USA；College of Electronics；Information Engineering Tongji University, China；School of Computer Science；AI Wuhan University of Technology, China , z\
