arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.08176cs.LG
收藏

用于节点回归/分类的无限宽度图神经网络

Infinite Width Graph Neural Networks for Node Regression/ Classification

Technical University of Munich(慕尼黑工业大学)

Yunus Cobanoglu

AI总结:

本文研究了宽度趋近无穷大的图神经网络,推导了多种架构下高斯过程和核的闭式解,并在节点回归与分类任务上验证了其性能,同时利用谱稀疏化方法改善了运行效率。

摘要:

这项工作分析了图神经网络(Graph Neural Networks),它是全连接深度神经网络在图结构数据上的推广,研究其宽度(即每个全连接层中的节点数)趋近于无穷大的情况。无限宽度神经网络将深度学习与高斯过程和核方法联系起来,这两种机器学习框架具有悠久的传统和广泛的理论基础。高斯过程和核方法比神经网络拥有少得多的超参数,并且可以用于不确定性估计,这使得它们在应用中对用户更加友好。本研究扩展了将高斯过程和核方法与神经网络联系起来的研究。针对多种架构推导了核和高斯过程的闭式解,即标准图神经网络、带有跳跃拼接连接的图神经网络以及图注意力神经网络。所有架构都在多个数据集上进行了评估,任务为直推式节点回归和分类。此外,使用一种称为有效电阻的谱稀疏化方法来改善运行时间和内存需求。将该设定扩展到归纳式图学习任务(图回归/分类)是直接的,并在3.5节中进行了简要讨论。

英文摘要:

This work analyzes Graph Neural Networks, a generalization of Fully-Connected Deep Neural Nets on Graph structured data, when their width, that is the number of nodes in each fullyconnected layer is increasing to infinity. Infinite Width Neural Networks are connecting Deep Learning to Gaussian Processes and Kernels, both Machine Learning Frameworks with long traditions and extensive theoretical foundations. Gaussian Processes and Kernels have much less hyperparameters then Neural Networks and can be used for uncertainty estimation, making them more user friendly for applications. This works extends the increasing amount of research connecting Gaussian Processes and Kernels to Neural Networks. The Kernel and Gaussian Process closed forms are derived for a variety of architectures, namely the standard Graph Neural Network, the Graph Neural Network with Skip-Concatenate Connections and the Graph Attention Neural Network. All architectures are evaluated on a variety of datasets on the task of transductive Node Regression and Classification. Additionally, a Spectral Sparsification method known as Effective Resistance is used to improve runtime and memory requirements. Extending the setting to inductive graph learning tasks (Graph Regression/ Classification) is straightforward and is briefly discussed in 3.5.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑