Graph Neural Networks (GNNs) are widely adopted in advanced AI systems due to their capability of representation learning on graph data. Even though GNN explanation is crucial to increase user trust in the systems, it is challenging due to the complexity of GNN execution. Lately, many works have been proposed to address some of the issues in GNN explanation. However, they lack generalization capability or suffer from computational burden when the size of graphs is enormous. To address these challenges, we propose a multi-level GNN explanation framework based on an observation that GNN is a multimodal learning process of multiple components in graph data. The complexity of the original problem is relaxed by breaking into multiple sub-parts represented as a hierarchical structure. The top-level explanation aims at specifying the contribution of each component to the model execution and predictions, while fine-grained levels focus on feature attribution and graph structure attribution analysis based on knowledge distillation. Student models are trained in standalone modes and are responsible for capturing different teacher behaviors, later used for particular component interpretation. Besides, we also aim for personalized explanations as the framework can generate different results based on user preferences. Finally, extensive experiments demonstrate the effectiveness and fidelity of our proposed approach.
Graph-based Knowledge Distillation: A survey and experimental evaluation
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Zhejiang University(浙江大学);China University of Geosciences (Beijing)(中国地质大学(北京))
ProGMLP: A Progressive Framework for GNN-to-MLP Knowledge Distillation with Efficient Trade-offs
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology(香港科技大学土木与环境工程系);JD Explore Academy, Beijing, China(北京京东探索研究院);School of Information Science and Engineering, Yunnan University(云南大学信息科学与工程学院)
Preference-driven Knowledge Distillation for Few-shot Node Classification
College of Electronic and Information Engineering, Tongji University, China(同济大学电子与信息学院);Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University, China(上海智能自主系统研究院);National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, China(人机混合增强智能国家级重点实验室);School of Computer Science and Technology, Tongji University, China(同济大学计算机科学与技术学院);Department of Computer Science, University of Illinois Chicago, USA(伊利诺伊大学芝加哥分校计算机科学系)
Learning Accurate, Efficient, and Interpretable MLPs on Multiplex Graphs via Node-wise Multi-View Ensemble Distillation
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Laboratory for Big Data and Decision, National University of Defense Technology(国防科技大学大数据与决策实验室)
Efficient Graph Knowledge Distillation from GNNs to Kolmogorov--Arnold Networks via Self-Attention Dynamic Sampling
School of Railway Intelligent Engineering, Dalian Jiaotong University, Dalian 116028, Liaoning, China(大连交通大学铁路智能工程学院);College of Electronic Information and Automation, Civil Aviation University of China, Tianjin 300300, Tianjin, China(中国民航大学电子信息与自动化学院)
Deep Graph-level Anomaly Detection by Glocal Knowledge Distillation
University of Technology Sydney(悉尼科技大学);Australian Artificial Intelligence Institute(澳大利亚人工智能研究所);Singapore Management University(新加坡管理大学);The University of Adelaide(阿德莱德大学);Australian Institute for Machine Learning(澳大利亚机器学习研究所)
Train Your Own GNN Teacher: Graph-Aware Distillation on Textual Graphs
University of Minnesota(明尼苏达大学);Amazon Web Services(亚马逊云科技);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学)
提出 GRAD 图感知蒸馏框架,通过共享 LM 联合训练 GNN 教师与无图学生,实现快速无图推理并提升节点分类性能。
T2-GNN: Graph Neural Networks for Graphs with Incomplete Features and Structure via Teacher-Student Distillation
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Economics and Management, Beijing University of Posts and Telecommunications(北京邮电大学经济管理学院);Pinterest
Model-to-Data Distillation for Graph Neural Networks
Department of Computer Science, Rice University, Houston, TX 77005, USA(计算机科学系,里士大学,休斯顿,德克萨斯州,77005,美国);Department of Computer Science & Ken Kennedy Institute, Rice University, Houston, TX 77005, USA(计算机科学系及肯尼迪学院,里士大学,休斯顿,德克萨斯州,77005,美国)
Tencent AI Lab(腾讯人工智能实验室);University of Texas at Arlington(德克萨斯大学阿灵顿分校);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院)
Knowledge distillation as a pathway toward next-generation intelligent ecohydrological modeling systems
Puget Sound Institute, University of Washington(华盛顿大学普吉特海湾研究所);Department of Civil Engineering, The University of Hong Kong(香港大学土木工程系);School for the Environment, University of Massachusetts Boston(马萨诸塞大学波士顿分校环境学院);Thornwell Labs(索恩韦尔实验室);Department of Hydrology and Atmospheric Sciences, The University of Arizona(亚利桑那大学水文与大气科学系)
Enhancing Knowledge Graph Completion with GNN Distillation and Probabilistic Interaction Modeling
Shandong Key Laboratory of Industrial Network Security(山东工业网络安全重点实验室);School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院)
MuGSI: Distilling GNNs with Multi-Granularity Structural Information for Graph Classification
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Carnegie Mellon University(卡内基梅隆大学);University of Southern California(南加利福尼亚大学)
Teaching MLPs to Master Heterogeneous Graph-Structured Knowledge for Efficient and Accurate Inference
State Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家重点实验室,南京大学);School of Information Technology and Electrical Engineering, The University of Queensland(信息科技与电气工程学院,昆士兰大学)