arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2210.13339cs.LG
收藏

层邻居采样——化解GNNs中的邻域爆炸

Layer-Neighbor Sampling -- Defusing Neighborhood Explosion in GNNs

NVIDIA(英伟达);Georgia Institute of Technology(佐治亚理工学院);Amazon Web Services(亚马逊云科技)

Muhammed Fatih Balın, Ümit V. Çatalyürek

AI总结:

提出LABOR采样算法,作为NS的直接替代方案,在保持相同fanout超参数和质量的同时显著减少采样顶点数,提高收敛速度和批大小。

摘要:

图神经网络(GNNs)近期受到了广泛关注,但大规模训练它们仍然是一个挑战。小批量训练结合采样被用于缓解这一挑战。然而,现有的方法要么受到邻域爆炸现象的影响,要么性能较差。为了解决这些问题,我们提出了一种名为LAyer-neighBOR sampling(LABOR)的新采样算法。它被设计为Neighbor Sampling(NS)的直接替代方案,具有相同的fanout超参数,同时采样的顶点最多减少7倍,且不牺牲质量。从单个顶点的角度来看,每个顶点估计量的方差与NS相匹配。此外,在相同的顶点采样预算约束下,LABOR比现有的层采样方法收敛更快,并且与NS相比,最多可以使用112倍大的批大小。

英文摘要:

Graph Neural Networks (GNNs) have received significant attention recently, but training them at a large scale remains a challenge. Mini-batch training coupled with sampling is used to alleviate this challenge. However, existing approaches either suffer from the neighborhood explosion phenomenon or have poor performance. To address these issues, we propose a new sampling algorithm called LAyer-neighBOR sampling (LABOR). It is designed to be a direct replacement for Neighbor Sampling (NS) with the same fanout hyperparameter while sampling up to 7 times fewer vertices, without sacrificing quality. By design, the variance of the estimator of each vertex matches NS from the point of view of a single vertex. Moreover, under the same vertex sampling budget constraints, LABOR converges faster than existing layer sampling approaches and can use up to 112 times larger batch sizes compared to NS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑