arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.00219cs.LG
收藏

图的知识蒸馏研究综述

Knowledge Distillation on Graphs: A Survey

University of Notre Dame(圣母大学);Brandeis University(布兰迪斯大学)

Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, Nitesh V. Chawla

摘要:

图神经网络(GNN)通过展示其处理图数据的能力而引起了极大的关注。然而,由于模型大小和多跳数据依赖性所施加的可扩展性约束,它们难以部署在资源有限的设备中。此外,现实世界的图通常具有复杂的结构信息和特征。因此,为了提高GNN的适用性,充分编码复杂的拓扑信息,引入了图上的知识蒸馏(KDG),以构建更小而有效的模型,并从数据中挖掘更多的知识,从而实现模型压缩,提高性能。近年来,KDG取得了长足的进展,提出了许多研究。在本综述中,我们系统地回顾了这些作品。具体而言,首先介绍了KDG面临的挑战和基础,然后通过回答以下三个问题对KDG的现有工作进行分类和总结:1)蒸馏什么,2)蒸馏给谁,3)如何蒸馏。最后,我们分享了我们对未来研究方向的看法。

英文摘要:

Graph Neural Networks (GNNs) have attracted tremendous attention by demonstrating their capability to handle graph data. However, they are difficult to be deployed in resource-limited devices due to model sizes and scalability constraints imposed by the multi-hop data dependency. In addition, real-world graphs usually possess complex structural information and features. Therefore, to improve the applicability of GNNs and fully encode the complicated topological information, knowledge distillation on graphs (KDG) has been introduced to build a smaller yet effective model and exploit more knowledge from data, leading to model compression and performance improvement. Recently, KDG has achieved considerable progress with many studies proposed. In this survey, we systematically review these works. Specifically, we first introduce KDG challenges and bases, then categorize and summarize existing works of KDG by answering the following three questions: 1) what to distillate, 2) who to whom, and 3) how to distillate. Finally, we share our thoughts on future research directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑