Graph neural networks (GNNs) have achieved great success on various tasks and fields that require relational modeling. GNNs aggregate node features using the graph structure as inductive biases resulting in flexible and powerful models. However, GNNs remain hard to interpret as the interplay between node features and graph structure is only implicitly learned. In this paper, we propose a novel method called Kedge for explicitly sparsifying the underlying graph by removing unnecessary neighbors. Our key idea is based on a tractable method for sparsification using the Hard Kumaraswamy distribution that can be used in conjugation with any GNN model. Kedge learns edge masks in a modular fashion trained with any GNN allowing for gradient based optimization in an end-to-end fashion. We demonstrate through extensive experiments that our model Kedge can prune a large proportion of the edges with only a minor effect on the test accuracy. Specifically, in the PubMed dataset, Kedge learns to drop more than 80% of the edges with an accuracy drop of merely 2% showing that graph structure has only a small contribution in comparison to node features. Finally, we also show that Kedge effectively counters the over-smoothing phenomena in deep GNNs by maintaining good task performance with increasing GNN layers.
Interpretable Sparsification of Brain Graphs: Better Practices and Effective Designs for Graph Neural Networks
University of Michigan, Ann Arbor(密歇根大学安娜堡分校);Georgia State University(佐治亚州立大学);University of North Carolina, Charlotte(北卡罗来纳大学夏洛特分校);Dartmouth College(达特茅斯学院)
Improving the interpretability of GNN predictions through conformal-based graph sparsification
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);McGill University(麦吉尔大学);Saarland University(萨尔大学);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)
Restyling Unsupervised Concept Based Interpretable Networks with Generative Models
Sorbonne Université(索邦大学);Technical University of Munich(慕尼黑工业大学);Helmholtz Munich(亥姆霍兹慕尼黑中心);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);LTCI, Télécom Paris, Institut Polytechnique de Paris(巴黎理工学院 巴黎高等电信学院 LTCI)
Mathematics & Statistics, University of Vermont(数学与统计学系,佛蒙特大学);Vermont Complex Systems Center, University of Vermont(佛蒙特复杂系统中心,佛蒙特大学);Computer Science, University of Vermont(计算机科学系,佛蒙特大学)
Optimized Architectures for Kolmogorov-Arnold Networks
Mathematics & Statistics, University of Vermont(佛蒙特大学数学与统计学系);Vermont Complex Systems Center, University of Vermont(佛蒙特复杂系统中心);Computer Science, University of Vermont(佛蒙特大学计算机科学系)
Lesson Learnt: Modularization of Deep Networks Allow Cross-Modality Reuse
Friedrich-Alexander University(弗里德里希-亚历山大大学);International Max Planck Research School Physics of Light (IMPRS-PL)(国际马克斯·普朗克光物理研究学院(IMPRS-PL));MIT(麻省理工学院)
University of Cambridge(剑桥大学);Université Côte d’Azur(蔚蓝海岸大学);CNRS(法国国家科学研究中心);Inria(法国国家信息与自动化研究所);University of Siena(锡耶纳大学);INRA(法国国家农业食品与环境研究院);Université Paris-Saclay(巴黎萨克雷大学);KU Leuven(鲁汶大学)