arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.03584cs.LG
收藏

使用图拓扑采样训练图卷积网络的泛化保证

Generalization Guarantee of Training Graph Convolutional Networks with Graph Topology Sampling

Rensselaer Polytechnic Institute(伦斯勒理工学院);Michigan State University(密歇根州立大学);MIT-IBM Watson AI Lab, IBM Research(IBM研究院麻省理工学院-IBM沃森人工智能实验室);IBM Thomas J. Watson Research Center(IBM托马斯·J·沃森研究中心);University at Buffalo(布法罗大学)

Hongkang Li, Meng Wang, Sijia Liu, Pin-Yu Chen, Jinjun Xiong

AI总结:

本文针对图拓扑采样训练至多三层 GCN 的半监督节点分类任务,给出泛化误差递减的充分条件,揭示图结构、采样与样本复杂度的关系。

摘要:

图卷积网络(GCN)近来在学习图结构数据方面取得了巨大的经验成功。为了解决由于对邻近特征进行递归嵌入而导致的可扩展性问题,研究者提出了图拓扑采样,以降低训练 GCN 的内存和计算成本;在许多经验研究中,它取得了与不进行拓扑采样时相当的测试性能。据我们所知,本文首次为在半监督节点分类中训练(至多)三层 GCN 时使用图拓扑采样提供了理论论证。我们形式化刻画了图拓扑采样的若干充分条件,使 GCN 训练能够产生递减的泛化误差。此外,我们的方法处理了各层权重之间的非凸交互,而这一点在现有 GCN 理论分析中尚未得到充分研究。本文明确刻画了图结构和拓扑采样对泛化性能及样本复杂度的影响,并通过数值实验验证了这些理论发现。

英文摘要:

Graph convolutional networks (GCNs) have recently achieved great empirical success in learning graph-structured data. To address its scalability issue due to the recursive embedding of neighboring features, graph topology sampling has been proposed to reduce the memory and computational cost of training GCNs, and it has achieved comparable test performance to those without topology sampling in many empirical studies. To the best of our knowledge, this paper provides the first theoretical justification of graph topology sampling in training (up to) three-layer GCNs for semi-supervised node classification. We formally characterize some sufficient conditions on graph topology sampling such that GCN training leads to a diminishing generalization error. Moreover, our method tackles the nonconvex interaction of weights across layers, which is under-explored in the existing theoretical analyses of GCNs. This paper characterizes the impact of graph structures and topology sampling on the generalization performance and sample complexity explicitly, and the theoretical findings are also justified through numerical experiments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑