arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.01089cs.LG
收藏

具有近优后悔的有偏图神经网络采样器

A Biased Graph Neural Network Sampler with Near-Optimal Regret

Georgia Institute of Technology(佐治亚理工学院);Amazon Shanghai AI Lab(亚马逊上海人工智能实验室);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)

Qingru Zhang, David Wipf, Quan Gan, Le Song

摘要:

图神经网络(GNN)最近已经成为将深度网络架构应用于图和关系数据的工具。然而,考虑到工业数据集的规模越来越大,在许多实际情况下,跨GNN层共享信息所需的消息传递计算不再具有可扩展性。虽然已经引入了各种采样方法来在易于处理的预算内近似全图训练,但仍然存在未解决的复杂性,例如高方差和有限的理论保证。为了解决这些问题,我们在现有工作的基础上,将GNN邻居采样视为多臂强盗问题,但新设计的奖励函数引入了一定程度的偏差,旨在减少方差并避免不稳定,可能无界的支付。与之前的bandit-GNN用例不同,由此产生的策略在考虑SGD引入的GNN训练动态的同时会导致接近最优的遗憾。从实际的角度来看,这意味着在几个基准测试中,方差估计值更低,测试精度更高。

英文摘要:

Graph neural networks (GNN) have recently emerged as a vehicle for applying deep network architectures to graph and relational data. However, given the increasing size of industrial datasets, in many practical situations the message passing computations required for sharing information across GNN layers are no longer scalable. Although various sampling methods have been introduced to approximate full-graph training within a tractable budget, there remain unresolved complications such as high variances and limited theoretical guarantees. To address these issues, we build upon existing work and treat GNN neighbor sampling as a multi-armed bandit problem but with a newly-designed reward function that introduces some degree of bias designed to reduce variance and avoid unstable, possibly-unbounded pay outs. And unlike prior bandit-GNN use cases, the resulting policy leads to near-optimal regret while accounting for the GNN training dynamics introduced by SGD. From a practical standpoint, this translates into lower variance estimates and competitive or superior test accuracy across several benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑