arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09192cs.LGcs.AI
收藏

图蒸馏是否像视觉数据集对应物一样?

Does Graph Distillation See Like Vision Dataset Counterpart?

Beihang University(北京航空航天大学);National University of Singapore(新加坡国立大学);Carnegie Mellon University(卡内基梅隆大学)

Beining Yang, Kai Wang, Qingyun Sun, Cheng Ji, Xingcheng Fu, Hao Tang, Yang You, Jianxin Li

AI总结:

针对图压缩忽视结构信息导致性能下降的问题,提出结构广播图数据集蒸馏(SGDD)方法,在9个数据集上取得最先进结果,显著降低LED偏移并大幅节省图规模。

摘要:

在大规模图上进行训练已在图表示学习中取得了显著成果,但其成本和存储问题日益引起关注。现有的图压缩方法主要侧重于优化压缩图的特征矩阵,而忽略了原始图结构信息的影响。为了探究结构信息的影响,我们从谱域进行分析,并在经验上识别出先前工作中存在显著的拉普拉斯能量分布(LED)偏移。这种偏移导致跨架构泛化性能不佳,以及在异常检测和链接预测等特定任务中的表现不佳。在本文中,我们提出了一种新颖的结构广播图数据集蒸馏(SGDD)方案,用于将原始结构信息广播到合成图的生成过程中,从而明确防止忽略原始结构信息。理论上,通过SGDD生成的合成图预计比先前工作具有更小的LED偏移,从而在跨架构设置和特定任务中均取得更优性能。我们在9个数据集上验证了所提出的SGDD,并在所有数据集上取得了最先进的结果:例如,在YelpChi数据集上,我们的方法保持了原始图数据集训练测试准确率的98.6%,同时图规模节省了1000倍。此外,我们经验评估了在9个数据集上LED偏移减少了17.6%至31.4%。大量实验和分析验证了所提出设计的有效性和必要性。代码可在GitHub仓库中获取:https://github.com/RingBDStack/SGDD。

英文摘要:

Training on large-scale graphs has achieved remarkable results in graph representation learning, but its cost and storage have attracted increasing concerns. Existing graph condensation methods primarily focus on optimizing the feature matrices of condensed graphs while overlooking the impact of the structure information from the original graphs. To investigate the impact of the structure information, we conduct analysis from the spectral domain and empirically identify substantial Laplacian Energy Distribution (LED) shifts in previous works. Such shifts lead to poor performance in cross-architecture generalization and specific tasks, including anomaly detection and link prediction. In this paper, we propose a novel Structure-broadcasting Graph Dataset Distillation (SGDD) scheme for broadcasting the original structure information to the generation of the synthetic one, which explicitly prevents overlooking the original structure information. Theoretically, the synthetic graphs by SGDD are expected to have smaller LED shifts than previous works, leading to superior performance in both cross-architecture settings and specific tasks. We validate the proposed SGDD across 9 datasets and achieve state-of-the-art results on all of them: for example, on the YelpChi dataset, our approach maintains 98.6% test accuracy of training on the original graph dataset with 1,000 times saving on the scale of the graph. Moreover, we empirically evaluate there exist 17.6% ~ 31.4% reductions in LED shift crossing 9 datasets. Extensive experiments and analysis verify the effectiveness and necessity of the proposed designs. The code is available in the GitHub repository: https://github.com/RingBDStack/SGDD.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑