From the ad network standpoint, a user's activity is a multi-type sequence of temporal events consisting of event types and time intervals. Understanding user patterns in ad networks has received increasing attention from the machine learning community. Particularly, the problems of fraud detection, Conversion Rate (CVR), and Click-Through Rate (CTR) prediction are of interest. However, the class imbalance between major and minor classes in these tasks can bias a machine learning model leading to poor performance. This study proposes using two multi-type (continuous and discrete) training approaches for GANs to deal with the limitations of traditional GANs in passing the gradient updates for discrete tokens. First, we used the Reinforcement Learning (RL)-based training approach and then, an approximation of the multinomial distribution parameterized in terms of the softmax function (Gumble-Softmax). Our extensive experiments based on synthetic data have shown the trained generator can generate sequences with desired properties measured by multiple criteria.
Imbalanced learning is a fundamental challenge in data mining, where there is a disproportionate ratio of training samples in each class. Over-sampling is an effective technique to tackle imbalanced learning through generating synthetic samples for the minority class. While numerous over-sampling algorithms have been proposed, they heavily rely on heuristics, which could be sub-optimal since we may need different sampling strategies for different datasets and base classifiers, and they cannot directly optimize the performance metric. Motivated by this, we investigate developing a learning-base…
the University of New South Wales（新南威尔士大学）；Hospital Universitario San Cecilio（圣塞西利奥大学医院）；Karolinska Institutet（卡罗林斯卡学院）；Università degli Studi di Siena（锡耶纳大学）；Universität zu Köln（科隆大学）；Australian National University（澳大利亚国立大学）
Clinical data usually cannot be freely distributed due to their highly confidential nature and this hampers the development of machine learning in the healthcare domain. One way to mitigate this problem is by generating realistic synthetic datasets using generative adversarial networks (GANs). However, GANs are known to suffer from mode collapse thus creating outputs of low diversity. This lowers the quality of the synthetic healthcare data, and may cause it to omit patients of minority demographics or neglect less common clinical practices. In this paper, we extend the classic GAN setup with …
Class imbalance is a common problem in supervised learning and impedes the predictive performance of classification models. Popular countermeasures include oversampling the minority class. Standard methods like SMOTE rely on finding nearest neighbours and linear interpolations which are problematic in case of high-dimensional, complex data distributions. Generative Adversarial Networks (GANs) have been proposed as an alternative method for generating artificial minority examples as they can model complex distributions. However, prior research on GAN-based oversampling does not incorporate rece…
Maastricht University（马斯特里赫特大学）；Institute of Data Science（数据科学研究所）；Brightlands Institute of Smart Society（布赖特兰兹智能社会研究所）
Despite the remarkable success of Generative Adversarial Networks (GANs) on text, images, and videos, generating high-quality tabular data is still under development owing to some unique challenges such as capturing dependencies in imbalanced data, optimizing the quality of synthetic patient data while preserving privacy. In this paper, we propose DP-CGANS, a differentially private conditional GAN framework consisting of data transformation, sampling, conditioning, and networks training to generate realistic and privacy-preserving tabular data. DP-CGANS distinguishes categorical and continuous…
Harmonized Gradient Descent for Class Imbalanced Data Stream Online Learning
Peng Cheng Laboratory（鹏城实验室）；School of Automation, Chongqing University（重庆大学自动化学院）；School of Information Science and Engineering, East China University of Science and Technology（东华大学信息科学与工程学院）
Texas A&M University（德州农工大学）；University of Washington（华盛顿大学）
Medical events of interest, such as mortality, often happen at a low rate in electronic medical records, as most admitted patients survive. Training models with this imbalance rate (class density discrepancy) may lead to suboptimal prediction. Traditionally this problem is addressed through ad-hoc methods such as resampling or reweighting but performance in many cases is still limited. We propose a framework for training models for this imbalance issue: 1) we first decouple the feature extraction and classification process, adjusting training batches separately for each component to mitigate b…
L3S Research Center（L3S研究中心）；Leibniz University of Hanover（汉诺威莱布尼茨大学）
Online class imbalance learning deals with data streams that are affected by both concept drift and class imbalance. Online learning tries to find a trade-off between exploiting previously learned information and incorporating new information into the model. This requires both the incremental update of the model and the ability to unlearn outdated information. The improper use of unlearning, however, can lead to the retroactive interference problem, a phenomenon that occurs when newly learned information interferes with the old information and impedes the recall of previously learned informati…
NOVA Information Management School, Universidade Nova de Lisboa（里斯本新大学信息管理学院）
Classification of imbalanced datasets is a challenging task for standard algorithms. Although many methods exist to address this problem in different ways, generating artificial data for the minority class is a more general approach compared to algorithmic modifications. SMOTE algorithm and its variations generate synthetic samples along a line segment that joins minority class instances. In this paper we propose Geometric SMOTE (G-SMOTE) as a generalization of the SMOTE data generation mechanism. G-SMOTE generates synthetic samples in a geometric region of the input space, around each selecte…
National University of Singapore（新加坡国立大学）；University of Chicago（芝加哥大学）
Supervised learning from training data with imbalanced class sizes, a commonly encountered scenario in real applications such as anomaly/fraud detection, has long been considered a significant challenge in machine learning. Motivated by recent progress in curriculum and self-paced learning, we propose to adopt a semi-supervised learning paradigm by training a deep neural network, referred to as SelectNet, to selectively add unlabelled data together with their predicted labels to the training dataset. Unlike existing techniques designed to tackle the difficulty in dealing with class imbalanced …
By leveraging deep learning to automatically classify camera trap images, ecologists can monitor biodiversity conservation efforts and the effects of climate change on ecosystems more efficiently. Due to the imbalanced class-distribution of camera trap datasets, current models are biased towards the majority classes. As a result, they obtain good performance for a few majority classes but poor performance for many minority classes. We used two-phase training to increase the performance for these minority classes. We trained, next to a baseline model, four models that implemented a different ve…
Southern University of Science and Technology（南方科技大学）；The University of Queensland（昆士兰大学）
Imbalanced classification on graphs is ubiquitous yet challenging in many real-world applications, such as fraudulent node detection. Recently, graph neural networks (GNNs) have shown promising performance on many network analysis tasks. However, most existing GNNs have almost exclusively focused on the balanced networks, and would get unappealing performance on the imbalanced networks. To bridge this gap, in this paper, we present a generative adversarial graph network model, called ImGAGN to address the imbalanced classification problem on graphs. It introduces a novel generator for graph st…
Imbalanced Semi-Supervised Learning via Label Refinement and Threshold Adjustment
College of Biomedical Engineering, Fudan University, Shanghai, China（复旦大学生物医学工程学院，上海，中国）；FMRIB Centre, Oxford Centre for Integrative Neuroimaging (OxCIN), University of Oxford, Oxford, UK（牛津大学FMRIB研究中心，牛津大学整合神经影像中心（OxCIN），牛津，英国）；School of Computer Science, University of Sheffield, Sheffield, UK（谢菲尔德大学计算机科学学院，谢菲尔德，英国）；Department of Engineering Science, University of Oxford, Oxford, UK（牛津大学工程科学系，牛津，英国）
L3S Research Center（L3S研究中心）；Leibniz University of Hannover（汉诺威莱布尼茨大学）；Carnegie Mellon University（卡内基梅隆大学）
Data-driven learning algorithms are employed in many online applications, in which data become available over time, like network monitoring, stock price prediction, job applications, etc. The underlying data distribution might evolve over time calling for model adaptation as new instances arrive and old instances become obsolete. In such dynamic environments, the so-called data streams, fairness-aware learning cannot be considered as a one-off requirement, but rather it should comprise a continual requirement over the stream. Recent fairness-aware stream classifiers ignore the problem of class…
Imbalanced class distribution is a common problem in a number of fields including medical diagnostics, fraud detection, and others. It causes bias in classification algorithms leading to poor performance on the minority class data. In this paper, we propose a novel method for balancing the class distribution in data through intelligent resampling of the minority class instances. The proposed method is based on generating new minority instances in the neighborhood of the existing minority points via a gamma distribution. Our method offers a natural and coherent approach to balancing the data. W…
The collected data from industrial machines are often imbalanced, which poses a negative effect on learning algorithms. However, this problem becomes more challenging for a mixed type of data or while there is overlapping between classes. Class-imbalance problem requires a robust learning system which can timely predict and classify the data. We propose a new adversarial network for simultaneous classification and fault detection. In particular, we restore the balance in the imbalanced dataset by generating faulty samples from the proposed mixture of data distribution. We designed the discrimi…
University of Birjand（比尔詹德大学）；Islamic Azad University（伊斯兰阿扎德大学）；Fasa University（法萨大学）；Luleå University of Technology（吕勒奥理工大学）；Dibrugarh University（迪布鲁格尔大学）
Imbalanced Data (ID) is a problem that deters Machine Learning (ML) models for achieving satisfactory results. ID is the occurrence of a situation where the quantity of the samples belonging to one class outnumbers that of the other by a wide margin, making such models learning process biased towards the majority class. In recent years, to address this issue, several solutions have been put forward, which opt for either synthetically generating new data for the minority class or reducing the number of majority classes for balancing the data. Hence, in this paper, we investigate the effectivene…
University College London（伦敦大学学院）；University of Birmingham（伯明翰大学）；University of Cambridge（剑桥大学）；University of Córdoba（科尔多瓦大学）
Data augmentation is rapidly gaining attention in machine learning. Synthetic data can be generated by simple transformations or through the data distribution. In the latter case, the main challenge is to estimate the label associated to new synthetic patterns. This paper studies the effect of generating synthetic data by convex combination of patterns and the use of these as unsupervised information in a semi-supervised learning framework with support vector machines, avoiding thus the need to label synthetic examples. We perform experiments on a total of 53 binary classification datasets. Ou…
Imbalanced response variable distribution is a common occurrence in data science. In fields such as fraud detection, medical diagnostics, system intrusion detection and many others where abnormal behavior is rarely observed the data under study often features disproportionate target class distribution. One common way to combat class imbalance is through resampling the minority class to achieve a more balanced distribution. In this paper, we investigate the performance of the sampling method based on kernel density estimation (KDE). We believe that KDE offers a more natural way of generating ne…
School of Management Engineering, Ulsan National Institute of Science and Technology (UNIST)（蔚山科学技术院管理工程学院）
In clinical research, the lack of events of interest often necessitates imbalanced learning. One approach to resolve this obstacle is data integration or sharing, but due to privacy concerns neither is practical. Therefore, there is an increasing demand for a platform on which an analysis can be performed in a federated environment while maintaining privacy. However, it is quite challenging to develop a federated learning algorithm that can address both privacy-preserving and class imbalanced issues. In this study, we introduce a federated generative model learning platform for generating samp…
Data-driven fault diagnostics and prognostics suffers from class-imbalance problem in industrial systems and it raises challenges to common machine learning algorithms as it becomes difficult to learn the features of the minority class samples. Synthetic oversampling methods are commonly used to tackle these problems by generating the minority class samples to balance the distributions between majority and minority classes. However, many of oversampling methods are inappropriate that they cannot generate effective and useful minority class samples according to different distributions of data, …
The University of the South Pacific（南太平洋大学）；IIT (BHU) Varanasi（印度理工学院（瓦拉纳西分校））；University of New South Wales（新南威尔士大学）
Class imbalance in a dataset is a major problem for classifiers that results in poor prediction with a high true positive rate (TPR) but a low true negative rate (TNR) for a majority positive training dataset. Generally, the pre-processing technique of oversampling of minority class(es) are used to overcome this deficiency. Our focus is on using the hybridization of Generative Adversarial Network (GAN) and Synthetic Minority Over-Sampling Technique (SMOTE) to address class imbalanced problems. We propose a novel two-phase oversampling approach involving knowledge transfer that has the synergy …
Peking University（北京大学）；University of Pennsylvania（宾夕法尼亚大学）
In this paper, we introduce the \textit{Layer-Peeled Model}, a nonconvex yet analytically tractable optimization program, in a quest to better understand deep neural networks that are trained for a sufficiently long time. As the name suggests, this new model is derived by isolating the topmost layer from the remainder of the neural network, followed by imposing certain constraints separately on the two parts of the network. We demonstrate that the Layer-Peeled Model, albeit simple, inherits many characteristics of well-trained neural networks, thereby offering an effective tool for explaining …
When Does Synthetic Data Augmentation Improve Score-Based Imbalanced Classification?
Department of Electrical & Computer Engineering, Duke University（电气与计算机工程系，杜克大学）；Department of Biostatistics & Bioinformatics, Duke University（生物统计学与生物信息学系，杜克大学）；Department of Biostatistics & Bioinformatics and Department of Computer Science, Duke University（生物统计学与生物信息学系和计算机科学系，杜克大学）
SamGoG: A Sampling-Based Graph-of-Graphs Framework for Imbalanced Graph Classification
School of Artificial Intelligence and Data Science, University of Science and Technology of China (USTC)（人工智能与数据科学学院，中国科学技术大学）；School of Biomedical Engineering, USTC（生物医学工程学院，中国科学技术大学）；Data Darkness Lab, Suzhou Institute for Advanced Research, USTC（数据黑暗实验室，苏州先进研究院）
In recent years, graph neural networks (GNNs) have achieved state-of-the-art performance for node classification. However, most existing GNNs would suffer from the graph imbalance problem. In many real-world scenarios, node classes are imbalanced, with some majority classes making up most parts of the graph. The message propagation mechanism in GNNs would further amplify the dominance of those majority classes, resulting in sub-optimal classification performance. In this work, we seek to address this problem by generating pseudo instances of minority classes to balance the training data, exten…
Time Series Data Augmentation as an Imbalanced Learning Problem
Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；Laboratory for Artificial Intelligence and Computer Science (LIACC)（人工智能与计算机科学实验室（LIACC））；Notre Dame University（圣母大学）；Fraunhofer Portugal AICOS（弗劳恩霍夫葡萄牙AICOS机构）
