Classifiers trained with class-imbalanced data are known to perform poorly on test data of the "minor" classes, of which we have insufficient training data. In this paper, we investigate learning a ConvNet classifier under such a scenario. We found that a ConvNet significantly over-fits the minor classes, which is quite opposite to traditional machine learning algorithms that often under-fit minor classes. We conducted a series of analysis and discovered the feature deviation phenomenon -- the learned ConvNet generates deviated features between the training and test data of minor classes -- which explains how over-fitting happens. To compensate for the effect of feature deviation which pushes test data toward low decision value regions, we propose to incorporate class-dependent temperatures (CDT) in training a ConvNet. CDT simulates feature deviation in the training phase, forcing the ConvNet to enlarge the decision values for minor-class data so that it can overcome real feature deviation in the test phase. We validate our approach on benchmark datasets and achieve promising performance. We hope that our insights can inspire new ways of thinking in resolving class-imbalanced deep learning.
Class imbalance problem has been a challenging research problem in the fields of machine learning and data mining as most real life datasets are imbalanced. Several existing machine learning algorithms try to maximize the accuracy classification by correctly identifying majority class samples while ignoring the minority class. However, the concept of the minority class instances usually represents a higher interest than the majority class. Recently, several cost sensitive methods, ensemble models and sampling techniques have been used in literature in order to classify imbalance datasets. In t…
Politecnico di Milano（米兰理工大学）；Human Technopole（人类技术学院）；Bicocca University（比可卡大学）；University of Cambridge（剑桥大学）
Class imbalance is a common issue in many domain applications of learning algorithms. Oftentimes, in the same domains it is much more relevant to correctly classify and profile minority class observations. This need can be addressed by Feature Selection (FS), that offers several further advantages, s.a. decreasing computational costs, aiding inference and interpretability. However, traditional FS techniques may become sub-optimal in the presence of strongly imbalanced data. To achieve FS advantages in this setting, we propose a filtering FS algorithm ranking feature importance on the basis of …
Graph Neural Networks (GNNs) have achieved unprecedented success in identifying categorical labels of graphs. However, most existing graph classification problems with GNNs follow the protocol of balanced data splitting, which misaligns with many real-world scenarios in which some classes have much fewer labels than others. Directly training GNNs under this imbalanced scenario may lead to uninformative representations of graphs in minority classes, and compromise the overall classification performance, which signifies the importance of developing effective GNNs towards handling imbalanced grap…
State-of-the-art machine learning models require access to significant amount of annotated data in order to achieve the desired level of performance. While unlabelled data can be largely available and even abundant, annotation process can be quite expensive and limiting. Under the assumption that some samples are more important for a given task than others, active learning targets the problem of identifying the most informative samples that one should acquire annotations for. Instead of the conventional reliance on model uncertainty as a proxy to leverage new unknown labels, in this work we pr…
University of L’Aquila（拉奎拉大学）；School Of AI（人工智能学院）
Despite the enormous amount of data, particular events of interest can still be quite rare. Classification of rare events is a common problem in many domains, such as fraudulent transactions, malware traffic analysis and network intrusion detection. Many studies have been developed for malware detection using machine learning approaches on various datasets, but as far as we know only the MTA-KDD'19 dataset has the peculiarity of updating the representative set of malicious traffic on a daily basis. This daily updating is the added value of the dataset, but it translates into a potential due to…
With growing credit card transaction volumes, the fraud percentages are also rising, including overhead costs for institutions to combat and compensate victims. The use of machine learning into the financial sector permits more effective protection against fraud and other economic crime. Suitably trained machine learning classifiers help proactive fraud detection, improving stakeholder trust and robustness against illicit transactions. However, the design of machine learning based fraud detection algorithms has been challenging and slow due the massively unbalanced nature of fraud data and the…
Swinburne University of Technology（斯威本科技大学）；Deakin University（迪肯大学）；Harvard Medical School（哈佛医学院）
Biomedical data are widely accepted in developing prediction models for identifying a specific tumor, drug discovery and classification of human cancers. However, previous studies usually focused on different classifiers, and overlook the class imbalance problem in real-world biomedical datasets. There are a lack of studies on evaluation of data pre-processing techniques, such as resampling and feature selection, on imbalanced biomedical data learning. The relationship between data pre-processing techniques and the data distributions has never been analysed in previous studies. This article ma…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；ByteDance（字节跳动）；University of Virginia（弗吉尼亚大学）；Jilin University（吉林大学）；University of Technology Sydney（悉尼科技大学）；Microsoft Research（微软研究院）
Class-imbalance is a common problem in machine learning practice. Typical Imbalanced Learning (IL) methods balance the data via intuitive class-wise resampling or reweighting. However, previous studies suggest that beyond class-imbalance, intrinsic data difficulty factors like overlapping, noise, and small disjuncts also play critical roles. To handle them, many solutions have been proposed (e.g., noise removal, borderline sampling, hard example mining) but are still confined to a specific factor and cannot generalize to broader scenarios, which raises an interesting question: how to handle bo…
Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction
International University of Business Agriculture and Technology（国际商业农业技术大学）；Jagannath University（贾甘纳特大学）；Deakin University（迪肯大学）；University of New South Wales（新南威尔士大学）；Imam Mohammad Ibn Saud Islamic University（伊玛目穆罕默德·伊本·沙特伊斯兰大学）；The University of Queensland（昆士兰大学）
Sungsin Women’s University（诚信女子大学）；Seoul National University（首尔大学）
In many classification problems, collecting massive clean-annotated data is not easy, and thus a lot of researches have been done to handle data with noisy labels. Most recent state-of-art solutions for noisy label problems are built on the small-loss strategy which exploits the memorization effect. While it is a powerful tool, the memorization effect has several drawbacks. The performances are sensitive to the choice of a training epoch required for utilizing the memorization effect. In addition, when the labels are heavily contaminated or imbalanced, the memorization effect may not occur in …
A Comprehensive Study of Supervised Machine Learning Models for Zero-Day Attack Detection: Analyzing Performance on Imbalanced Data
School of Electrical Engineering and Computer Science University of Ottawa（电气工程与计算机科学学院，渥太华大学）；Department of Computer Engineering K.N.Toosi University（计算机工程系，K.N.Toosi大学）
Sharif University of Technology（谢里夫理工大学）；Technical University of Munich（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Disease prediction is a well-known classification problem in medical applications. GCNs provide a powerful tool for analyzing the patients' features relative to each other. This can be achieved by modeling the problem as a graph node classification task, where each node is a patient. Due to the nature of such medical datasets, class imbalance is a prevalent issue in the field of disease prediction, where the distribution of classes is skewed. When the class imbalance is present in the data, the existing graph-based classifiers tend to be biased towards the major class(es) and neglect the sampl…
An Imbalanced Dataset with Multiple Feature Representations for Studying Quality Control of Next-Generation Sequencing
Johannes Gutenberg University Mainz（明斯特-耶拿大学）；University of Southern Denmark（南丹麦大学）；Department of Dermatology, University Medical Center of the Johannes Gutenberg University（耶拿大学皮肤科部门）；University of Auckland（奥克兰大学）；Institute of Quantitative and Computational Biology, Johannes Gutenberg University Mainz（明斯特-耶拿大学定量与计算生物学研究所）
A Machine Learning Approach for Identifying Anatomical Biomarkers of Early Mild Cognitive Impairment
Universiti Teknologi PETRONAS（国油科技大学）；Hospital Universiti Sains Malaysia（马来西亚理科大学医院）；Universiti Sains Malaysia（马来西亚理科大学）；Ulster University（阿尔斯特大学）；University of Calgary（卡尔加里大学）；University of Bath（巴斯大学）
NGA Research（NGA研究院）；Etegent Technologies Ltd（Etegent科技有限公司）；CACI；NVIDIA（英伟达）
The need for labeled data is among the most common and well-known practical obstacles to deploying deep learning algorithms to solve real-world problems. The current generation of learning algorithms requires a large volume of data labeled according to a static and pre-defined schema. Conversely, humans can quickly learn generalizations based on large quantities of unlabeled data, and turn these generalizations into classifications using spontaneous labels, often including labels not seen before. We apply a state-of-the-art unsupervised learning algorithm to the noisy and extremely imbalanced …
Paris-Saclay University（巴黎-萨克雷大学）；UVSQ（凡尔赛大学）；Inserm（法国国家健康与医学研究院）；Gustave Roussy（古斯塔夫·鲁西研究所）；Epiconcept Company（Epiconcept公司）；Accenture Labs（埃森哲实验室）；Sorbonne University（索邦大学）；University Sorbonne Paris Nord（索邦巴黎北大学）；Indian Institute of Science（印度科学学院）；Luxembourg Institute of Health（卢森堡健康研究院）
Objective: Leveraging machine learning methods, we aim to extract both explicit and implicit cause-effect associations in patient-reported, diabetes-related tweets and provide a tool to better understand opinion, feelings and observations shared within the diabetes online community from a causality perspective. Materials and Methods: More than 30 million diabetes-related tweets in English were collected between April 2017 and January 2021. Deep learning and natural language processing methods were applied to focus on tweets with personal and emotional content. A cause-effect-tweet dataset was …
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Nvidia Inc.（英伟达公司）；Xilinx Inc.（赛灵思公司）
Blindly decoding a signal requires estimating its unknown transmit parameters, compensating for the wireless channel impairments, and identifying the modulation type. While deep learning can solve complex problems, digital signal processing (DSP) is interpretable and can be more computationally efficient. To combine both, we propose the dual path network (DPN). It consists of a signal path of DSP operations that recover the signal, and a feature path of neural networks that estimate the unknown transmit parameters. By interconnecting the paths over several recovery stages, later stages benefit…
L3S Research Center, Leibniz University of Hannover（莱布尼茨汉诺威大学L3S研究中心）；Institute of Computer Science, FORTH-ICS（希腊研究与技术基金会计算机科学研究所）；Dept. of CSE, IIT Patna（印度理工学院巴特那分校计算机科学与工程系）；Chongqing University of Technology（重庆理工大学）；GESIS - Leibniz Institute for the Social Sciences, Cologne & Heinrich-Heine-University Düsseldorf（科隆莱布尼茨社会科学研究所暨杜塞尔多夫海因里希·海涅大学）
Fact checking is an essential challenge when combating fake news. Identifying documents that agree or disagree with a particular statement (claim) is a core task in this process. In this context, stance detection aims at identifying the position (stance) of a document towards a claim. Most approaches address this task through a 4-class classification model where the class distribution is highly imbalanced. Therefore, they are particularly ineffective in detecting the minority classes (for instance, 'disagree'), even though such instances are crucial for tasks such as fact-checking by providing…
Detecting offensive language on social media is an important task. The ICWSM-2020 Data Challenge Task 2 is aimed at identifying offensive content using a crowd-sourced dataset containing 100k labelled tweets. The dataset, however, suffers from class imbalance, where certain labels are extremely rare compared with other classes (e.g, the hateful class is only 5% of the data). In this work, we present Dager (Data Augmenter), a generation-based data augmentation method, that improves the performance of classification on imbalanced and low-resource data such as the offensive language dataset. Dage…
University of Exeter（埃克塞特大学）；China University of Petroleum (East China)（中国石油大学（华东））
Intrusion detection systems (IDSs) play an important role in identifying malicious attacks and threats in networking systems. As fundamental tools of IDSs, learning based classification methods have been widely employed. When it comes to detecting network intrusions in small sample sizes (e.g., emerging intrusions), the limited number and imbalanced proportion of training samples usually cause significant challenges in training supervised and semi-supervised classifiers. In this paper, we propose a general network intrusion detection framework to address the challenges of both \emph{data scarc…
Purpose: Malicious web domain identification is of significant importance to the security protection of Internet users. With online credibility and performance data, this paper aims to investigate the use of machine learning tech-niques for malicious web domain identification by considering the class imbalance issue (i.e., there are more benign web domains than malicious ones). Design/methodology/approach: We propose an integrated resampling approach to handle class imbalance by combining the Synthetic Minority Over-sampling TEchnique (SMOTE) and Particle Swarm Optimisation (PSO), a population…
