arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.15643cs.LGstat.ML
收藏

图卷积网络中度相关偏差的研究与缓解

Investigating and Mitigating Degree-Related Biases in Graph Convolutional Networks

The Pennsylvania State University(宾夕法尼亚州立大学);Michigan State University(密歇根州立大学);IBM T.J. Watson(IBM T.J. Watson 研究院)

Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Yiqi Wang, Jiliang Tang, Charu Aggarwal, Prasenjit Mitra, Suhang Wang

摘要:

图卷积网络在图的半监督学习任务中表现出良好的效果,因此与其他方法相比成为一种很好的方法。尽管GCN取得了显著的成功,但在监督不足的情况下很难训练GCN。当标记数据有限时,GCN的性能对于低度节点变得不令人满意。虽然一些先前的工作在整个模型级别上分析了GCN的成功和失败,但在单个节点级别上分析GCN仍然没有充分探索。在本文中,我们分析GCN的节点度分布。从经验观察到理论证明,我们确认GCN偏向于具有更高准确度的节点,即使在大多数图中高度节点的代表性不足。我们进一步开发了一种新的自监督学习度特定GCN(SL-DSGC),从模型和数据方面减轻GCN的度相关偏差。首先,我们提出了一个度特定的GCN层,捕获不同程度的节点的差异和相似性,这减少了GCN的内部模型方面的偏见所造成的共享相同的参数与所有节点。其次,我们设计了一个自监督学习算法,该算法使用贝叶斯神经网络在未标记的节点上创建具有不确定性分数的伪标签。伪标签增加了连接到低度节点的标记邻居的机会,从而从数据角度减少了GCN的偏差。进一步利用不确定性分数在SL-DSGC的随机梯度下降中动态地加权伪标签。在三个基准数据集上的实验表明,SL-DSGC不仅优于最先进的自训练/自监督学习GCN方法,而且大大提高了低度节点的GCN准确性。

英文摘要:

Graph Convolutional Networks (GCNs) show promising results for semi-supervised learning tasks on graphs, thus become favorable comparing with other approaches. Despite the remarkable success of GCNs, it is difficult to train GCNs with insufficient supervision. When labeled data are limited, the performance of GCNs becomes unsatisfying for low-degree nodes. While some prior work analyze successes and failures of GCNs on the entire model level, profiling GCNs on individual node level is still underexplored. In this paper, we analyze GCNs in regard to the node degree distribution. From empirical observation to theoretical proof, we confirm that GCNs are biased towards nodes with larger degrees with higher accuracy on them, even if high-degree nodes are underrepresented in most graphs. We further develop a novel Self-Supervised-Learning Degree-Specific GCN (SL-DSGC) that mitigate the degree-related biases of GCNs from model and data aspects. Firstly, we propose a degree-specific GCN layer that captures both discrepancies and similarities of nodes with different degrees, which reduces the inner model-aspect biases of GCNs caused by sharing the same parameters with all nodes. Secondly, we design a self-supervised-learning algorithm that creates pseudo labels with uncertainty scores on unlabeled nodes with a Bayesian neural network. Pseudo labels increase the chance of connecting to labeled neighbors for low-degree nodes, thus reducing the biases of GCNs from the data perspective. Uncertainty scores are further exploited to weight pseudo labels dynamically in the stochastic gradient descent for SL-DSGC. Experiments on three benchmark datasets show SL-DSGC not only outperforms state-of-the-art self-training/self-supervised-learning GCN methods, but also improves GCN accuracy dramatically for low-degree nodes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑