Scalability of graph neural networks remains one of the major challenges in graph machine learning. Since the representation of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes from previous layers, the receptive fields grow exponentially, which makes standard stochastic optimization techniques ineffective. Various approaches have been proposed to alleviate this issue, e.g., sampling-based methods and techniques based on pre-computation of graph filters.
In this paper, we take a different approach and propose to use graph coarsening for scalable training of GNNs, which is generic, extremely simple and has sublinear memory and time costs during training. We present extensive theoretical analysis on the effect of using coarsening operations and provides useful guidance on the choice of coarsening methods. Interestingly, our theoretical analysis shows that coarsening can also be considered as a type of regularization and may improve the generalization. Finally, empirical results on real world datasets show that, simply applying off-the-shelf coarsening methods, we can reduce the number of nodes by up to a factor of ten without causing a noticeable downgrade in classification accuracy.
Graph Coarsening via Supervised Granular-Ball for Scalable Graph Neural Network Training
Chongqing University of Posts and Telecommunications(重庆邮电大学);Chongqing Normal University(重庆师范大学);National Center for Applied Mathematics in Chongqing(重庆国家应用数学中心)
Remedying Coarsening-Based GNN Training under Heterophily via Adaptive Complementary Enhancement
Hong Kong Baptist University(香港浸会大学);Renmin University of China(中国人民大学);Chinese Academy of Sciences(中国科学院);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Sun Yat-sen University(中山大学)