arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09202cs.LG
收藏

基于特征基匹配的图蒸馏

Graph Distillation with Eigenbasis Matching

Beijing University of Posts and Telecommunication(北京邮电大学)

Yang Liu, Deyu Bo, Chuan Shi

AI总结:

针对现有图蒸馏方法存在谱偏差、跨架构泛化性差的问题,提出基于特征基匹配的图蒸馏方法GDEM,通过对齐特征基与节点特征、复制真实图谱并加入判别约束,实现了更优的蒸馏性能与跨架构泛化能力。

摘要:

图数据量的不断增长对图神经网络(GNN)的高效训练提出了要求。新兴的图蒸馏(GD)技术通过蒸馏出小型合成图来替代真实的大规模图,以解决这一挑战,确保在真实图和合成图上训练的GNN表现出相当的性能。然而,现有方法依赖与GNN相关的信息作为监督信号,包括梯度、表示和训练轨迹,这存在两方面局限。首先,GNN会影响真实图的谱(即特征值),导致合成图出现谱偏差。其次,GNN架构的多样性会生成不同的合成图,需要遍历架构才能获得最优性能。为解决这些问题,我们提出了基于特征基匹配的图蒸馏方法(GDEM),该方法对齐真实图与合成图的特征基和节点特征。同时,它直接复制真实图的谱,从而避免GNN的影响。此外,我们设计了一种判别约束来平衡GDEM的有效性和泛化性。理论上,GDEM蒸馏出的合成图是真实图的受限谱近似。大量实验表明,GDEM优于当前最优的图蒸馏方法,具备强大的跨架构泛化能力和显著的蒸馏效率。我们的代码可在https://github.com/liuyang-tian/GDEM 获取。

英文摘要:

The increasing amount of graph data places requirements on the efficient training of graph neural networks (GNNs). The emerging graph distillation (GD) tackles this challenge by distilling a small synthetic graph to replace the real large graph, ensuring GNNs trained on real and synthetic graphs exhibit comparable performance. However, existing methods rely on GNN-related information as supervision, including gradients, representations, and trajectories, which have two limitations. First, GNNs can affect the spectrum (i.e., eigenvalues) of the real graph, causing spectrum bias in the synthetic graph. Second, the variety of GNN architectures leads to the creation of different synthetic graphs, requiring traversal to obtain optimal performance. To tackle these issues, we propose Graph Distillation with Eigenbasis Matching (GDEM), which aligns the eigenbasis and node features of real and synthetic graphs. Meanwhile, it directly replicates the spectrum of the real graph and thus prevents the influence of GNNs. Moreover, we design a discrimination constraint to balance the effectiveness and generalization of GDEM. Theoretically, the synthetic graphs distilled by GDEM are restricted spectral approximations of the real graphs. Extensive experiments demonstrate that GDEM outperforms state-of-the-art GD methods with powerful cross-architecture generalization ability and significant distillation efficiency. Our code is available at https://github.com/liuyang-tian/GDEM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑