Graph Neural Network with Curriculum Learning for Imbalanced Node Classification
Aerospace Information Research Institute, Chinese Academy Of Sciences（中国科学院空天信息研究院）；School of Software, Tsinghua University（清华大学软件学院）；National University of Singapore（新加坡国立大学）
Xiaohe Li, Lijie Wen, Yawen Deng, Fuli Feng, Xuming Hu, Lei Wang, Zide Fan
Graph Neural Network (GNN) is an emerging technique for graph-based learning tasks such as node classification. In this work, we reveal the vulnerability of GNN to the imbalance of node labels. Traditional solutions for imbalanced classification (e.g. resampling) are ineffective in node classification without considering the graph structure. Worse still, they may even bring overfitting or underfitting results due to lack of sufficient prior knowledge. To solve these problems, we propose a novel graph neural network framework with curriculum learning (GNN-CL) consisting of two modules. For one thing, we hope to acquire certain reliable interpolation nodes and edges through the novel graph-based oversampling based on smoothness and homophily. For another, we combine graph classification loss and metric learning loss which adjust the distance between different nodes associated with minority class in feature space. Inspired by curriculum learning, we dynamically adjust the weights of different modules during training process to achieve better ability of generalization and discrimination. The proposed framework is evaluated via several widely used graph datasets, showing that our proposed model consistently outperforms the existing state-of-the-art methods.
Southern University of Science and Technology（南方科技大学）；The University of Queensland（昆士兰大学）
Imbalanced classification on graphs is ubiquitous yet challenging in many real-world applications, such as fraudulent node detection. Recently, graph neural networks (GNNs) have shown promising performance on many network analysis tasks. However, most existing GNNs have almost exclusively focused on the balanced networks, and would get unappealing performance on the imbalanced networks. To bridge this gap, in this paper, we present a generative adversarial graph network model, called ImGAGN to address the imbalanced classification problem on graphs. It introduces a novel generator for graph st…
ImGCL: Revisiting Graph Contrastive Learning on Imbalanced Node Classification
Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University（清华大学交叉信息研究院）；Tencent AI Lab（腾讯人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；AITRICS
Learning unbiased node representations under class-imbalanced graph data is challenging due to interactions between adjacent nodes. Existing studies have in common that they compensate the minor class nodes `as a group' according to their overall quantity (ignoring node connections in graph), which inevitably increase the false positive cases for major nodes. We hypothesize that the increase in these false positive cases is highly affected by the label distribution around each node and confirm it experimentally. In addition, in order to handle this issue, we propose Topology-Aware Margin (TAM)…
Westlake University（西湖大学）；Westlake Institute for Advanced Study（西湖高等研究院）；Zhejiang University（浙江大学）
Recent years have witnessed great success in handling node classification tasks with Graph Neural Networks (GNNs). However, most existing GNNs are based on the assumption that node samples for different classes are balanced, while for many real-world graphs, there exists the problem of class imbalance, i.e., some classes may have much fewer samples than others. In this case, directly training a GNN classifier with raw data would under-represent samples from those minority classes and result in sub-optimal performance. This paper presents GraphMixup, a novel mixup-based framework for improving …
Automated Loss function Search for Class-imbalanced Node Classification
School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；School of Cyber Engineering, Xidian University（西安电子科技大学网络与信息安全学院）；Guangzhou Institute of Technology, Xidian University（西安电子科技大学广州研究院）
College of Information Science and Technology, Penn State University（宾夕法尼亚州立大学信息科学与技术学院）
Node classification is an important research topic in graph learning. Graph neural networks (GNNs) have achieved state-of-the-art performance of node classification. However, existing GNNs address the problem where node samples for different classes are balanced; while for many real-world scenarios, some classes may have much fewer instances than others. Directly training a GNN classifier in this case would under-represent samples from those minority classes and result in sub-optimal performance. Therefore, it is very important to develop GNNs for imbalanced node classification. However, the w…
Pure Node Selection for Imbalanced Graph Node Classification
School of Intelligent Systems Science（智能系统科学学院）；Engineering, Jinan University, Zhuhai 519070, China（济南大学珠海校区）；College of Cyber Security, Jinan University, Guangzhou 510632, China（网络安全学院）；Department of Computer Science, University of Illinois Chicago, Chicago 60607, USA（伊利诺伊大学芝加哥分校计算机科学系）
Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNs
Fuzhou University（福州大学）；HKUST (Guangzhou)（香港科技大学（广州））；Shandong Provincial Key Laboratory of Computer Networks（山东省计算机网络重点实验室）；Shandong Fundamental Research Center for Computer Science（山东计算机科学基础研究中心）
PLA Strategy Support Force Information Engineering University（中国人民解放军战略支援部队信息工程大学）
The Graph Neural Network (GNN) has been widely used for graph data representation. However, the existing researches only consider the ideal balanced dataset, and the imbalanced dataset is rarely considered. Traditional methods such as resampling, reweighting, and synthetic samples that deal with imbalanced datasets are no longer applicable in GNN. This paper proposes an ensemble model called Boosting-GNN, which uses GNNs as the base classifiers during boosting. In Boosting-GNN, higher weights are set for the training samples that are not correctly classified by the previous classifier, thus ac…
Vanderbilt University（范德堡大学）；IBM T.J. Watson Research Center（IBM T.J. 华生研究中心）
Recent years have witnessed the significant success of applying graph neural networks (GNNs) in learning effective node representations for classification. However, current GNNs are mostly built under the balanced data-splitting, which is inconsistent with many real-world networks where the number of training nodes can be extremely imbalanced among the classes. Thus, directly utilizing current GNNs on imbalanced data would generate coarse representations of nodes in minority classes and ultimately compromise the classification performance. This therefore portends the importance of developing e…
In recent years, graph neural networks (GNNs) have achieved state-of-the-art performance for node classification. However, most existing GNNs would suffer from the graph imbalance problem. In many real-world scenarios, node classes are imbalanced, with some majority classes making up most parts of the graph. The message propagation mechanism in GNNs would further amplify the dominance of those majority classes, resulting in sub-optimal classification performance. In this work, we seek to address this problem by generating pseudo instances of minority classes to balance the training data, exten…
Wuhan University（武汉大学）；The University of Sydney（悉尼大学）；JD Explore Academy（京东探索研究院）
Node classification is a fundamental graph-based task that aims to predict the classes of unlabeled nodes, for which Graph Neural Networks (GNNs) are the state-of-the-art methods. Current GNNs assume that nodes in the training set contribute equally during training. However, the quality of training nodes varies greatly, and the performance of GNNs could be harmed by two types of low-quality training nodes: (1) inter-class nodes situated near class boundaries that lack the typical characteristics of their corresponding classes. Because GNNs are data-driven approaches, training on these nodes co…
In recent years, the node classification task in graph neural networks(GNNs) has developed rapidly, driving the development of research in various fields. However, there are a large number of class imbalances in the graph data, and there is a large gap between the number of different classes, resulting in suboptimal results in classification. Proposing a solution to the imbalance problem has become indispensable for the successful advancement of our downstream missions. Therefore, we start with the loss function and try to find a loss function that can effectively solve the imbalance of graph …
Adversarial approach has been widely used for data generation in the last few years. However, this approach has not been extensively utilized for classifier training. In this paper, we propose an adversarial framework for classifier training that can also handle imbalanced data. Indeed, a network is trained via an adversarial approach to give weights to samples of the majority class such that the obtained classification problem becomes more challenging for the discriminator and thus boosts its classification capability. In addition to the general imbalanced classification problems, the propose…
Imbalanced Graph-Level Anomaly Detection via Counterfactual Augmentation and Feature Learning
School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）；School of Computing, Macquarie University（麦考瑞大学计算学院）；Renmin Hospital of Wuhan University（武汉大学人民医院）
Aalborg University（奥尔堡大学）；Mila-Quebec AI Institute（米拉-魁北克人工智能研究所）；HEC Montreal（蒙特利尔高等商学院）
Path representations are critical in a variety of transportation applications, such as estimating path ranking in path recommendation systems and estimating path travel time in navigation systems. Existing studies often learn task-specific path representations in a supervised manner, which require a large amount of labeled training data and generalize poorly to other tasks. We propose an unsupervised learning framework Path InfoMax (PIM) to learn generic path representations that work for different downstream tasks. We first propose a curriculum negative sampling method, for each input path, t…
Graph Neural Networks (GNNs) have achieved unprecedented success in identifying categorical labels of graphs. However, most existing graph classification problems with GNNs follow the protocol of balanced data splitting, which misaligns with many real-world scenarios in which some classes have much fewer labels than others. Directly training GNNs under this imbalanced scenario may lead to uninformative representations of graphs in minority classes, and compromise the overall classification performance, which signifies the importance of developing effective GNNs towards handling imbalanced grap…
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Alibaba Group（阿里巴巴集团）；University of Chinese Academy of Sciences（中国科学院大学）
Capturing emotions within a conversation plays an essential role in modern dialogue systems. However, the weak correlation between emotions and semantics brings many challenges to emotion recognition in conversation (ERC). Even semantically similar utterances, the emotion may vary drastically depending on contexts or speakers. In this paper, we propose a Supervised Prototypical Contrastive Learning (SPCL) loss for the ERC task. Leveraging the Prototypical Network, the SPCL targets at solving the imbalanced classification problem through contrastive learning and does not require a large batch s…
Nanjing University of Posts and Telecommunications（南京邮电大学）
Graph machine learning (GML) has made great progress in node classification, link prediction, graph classification and so on. However, graphs in reality are often structurally imbalanced, that is, only a few hub nodes have a denser local structure and higher influence. The imbalance may compromise the robustness of existing GML models, especially in learning tail nodes. This paper proposes a selective graph augmentation method (SAug) to solve this problem. Firstly, a Pagerank-based sampling strategy is designed to identify hub nodes and tail nodes in the graph. Secondly, a selective augmentati…
Network Embedding with Completely-imbalanced Labels
Department of Computer Science and Technology, University of Science and Technology Beijing（计算机科学与技术系，北京科技大学）；School of Software, Tsinghua University（软件学院，清华大学）；Department of Computer Science, University of Illinois at Chicago（计算机科学系，伊利诺伊大学香槟分校）
Department of Economics, University of California, Berkeley（加州大学伯克利分校经济学系）；Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance, School of Computer and Computing Science, Hangzhou City University（浙江省实时智能科技城市治理工程研究中心，计算机与计算科学学院，杭州城市大学）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
Center for Research on Intelligent Perception and Computing, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所智能感知与计算研究中心）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Graph classification is a challenging research problem in many applications across a broad range of domains. In these applications, it is very common that class distribution is imbalanced. Recently, Graph Neural Network (GNN) models have achieved superior performance on various real-world datasets. Despite their success, most of current GNN models largely overlook the important setting of imbalanced class distribution, which typically results in prediction bias towards majority classes. To alleviate the prediction bias, we propose to leverage semantic structure of dataset based on the distribu…
SamGoG: A Sampling-Based Graph-of-Graphs Framework for Imbalanced Graph Classification
School of Artificial Intelligence and Data Science, University of Science and Technology of China (USTC)（人工智能与数据科学学院，中国科学技术大学）；School of Biomedical Engineering, USTC（生物医学工程学院，中国科学技术大学）；Data Darkness Lab, Suzhou Institute for Advanced Research, USTC（数据黑暗实验室，苏州先进研究院）
Graph neural networks can be effectively applied to find solutions for many real-world problems across widely diverse fields. The success of graph neural networks is linked to the message-passing mechanism on the graph, however, the message-aggregating behavior is still not entirely clear in most algorithms. To improve functionality, we propose a new transparent network called Graph Decipher to investigate the message-passing mechanism by prioritizing in two main components: the graph structure and node attributes, at the graph, feature, and global levels on a graph under the node classificati…
Virginia Tech（弗吉尼亚理工大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Graph pre-training strategies have been attracting a surge of attention in the graph mining community, due to their flexibility in parameterizing graph neural networks (GNNs) without any label information. The key idea lies in encoding valuable information into the backbone GNNs, by predicting the masked graph signals extracted from the input graphs. In order to balance the importance of diverse graph signals (e.g., nodes, edges, subgraphs), the existing approaches are mostly hand-engineered by introducing hyperparameters to re-weight the importance of graph signals. However, human interventio…
Mitigating Message Imbalance in Fraud Detection with Dual-View Graph Representation Learning
Key Lab of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University（教育区块链与智能技术重点实验室，教育部，广西师范大学）；Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University（广西多源信息挖掘与安全重点实验室，广西师范大学）；School of Software, Beihang University（北京航空航天大学软件学院）；SKLCCSE, School of Computer Science and Engineering, Beihang University（SKLCCSE，北京航空航天大学计算机科学与工程学院）；School of New Media and Communication, Tianjin University（天津大学新媒体与传播学院）
SPECTRA: Spectral Domain-Aware Graph Generation for Imbalanced Molecular Property Regression
University of Notre Dame, Dept. of Computer Science；University of Notre Dame, Dept. of Chemistry；University of Notre Dame, Lucy Family Institute for Data \& Society Notre Dame Indiana USA；University of Notre Dame, Lucy Family Institute for Data \& Society
Graph embedding, representing local and global neighborhood information by numerical vectors, is a crucial part of the mathematical modeling of a wide range of real-world systems. Among the embedding algorithms, random walk-based algorithms have proven to be very successful. These algorithms collect information by creating numerous random walks with a redefined number of steps. Creating random walks is the most demanding part of the embedding process. The computation demand increases with the size of the network. Moreover, for real-world networks, considering all nodes on the same footing, the…
Brandeis University（布兰迪斯大学）；University of Hong Kong（香港大学）；University of Notre Dame（圣母大学）；Hunan University（湖南大学）
Even pruned by the state-of-the-art network compression methods, Graph Neural Networks (GNNs) training upon non-Euclidean graph data often encounters relatively higher time costs, due to its irregular and nasty density properties, compared with data in the regular Euclidean space. Another natural property concomitantly with graph is class-imbalance which cannot be alleviated by the massive graph data while hindering GNNs' generalization. To fully tackle these unpleasant properties, (i) theoretically, we introduce a hypothesis about what extent a subset of the training data can approximate the …
