arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.02040cs.LGcs.AI
收藏

DFA-GNN:通过直接反馈对齐的前向学习图神经网络

DFA-GNN: Forward Learning of Graph Neural Networks by Direct Feedback Alignment

School of Computer Science and Technology(计算机科学与技术学院);Beijing Jiaotong University(北京交通大学);Department of Computer Science(计算机科学系);Stony Brook University(石溪大学)

Gongpei Zhao, Tao Wang, Congyan Lang, Yi Jin, Yidong Li, Haibin Ling

AI总结:

针对图神经网络训练中反向传播的局限,提出DFA-GNN前向学习框架,通过直接反馈对齐和伪误差生成器实现高效半监督学习,在10个基准上超越BP方法且鲁棒性强。

摘要:

图神经网络因其在各种应用中的强大性能而受到认可,其中反向传播算法在大多数GNN模型的发展中扮演着核心角色。然而,尽管BP有效,但它存在局限性,这些局限性挑战了其生物学合理性,并影响了基于图的任务中神经网络训练的效率、可扩展性和并行性。虽然几种非BP训练算法,如直接反馈对齐,已成功应用于处理欧几里得数据的全连接和卷积网络组件,但直接将这种非BP框架应用于GNN模型中处理非欧几里得图数据却面临重大挑战。这些挑战主要源于图数据中i.i.d.假设的违反,以及难以访问图中所有样本(节点)的预测误差。为了克服这些障碍,在本文中我们提出了DFA-GNN,一种专为GNN设计的新型前向学习框架,并以半监督学习作为案例研究。所提出的方法通过使用专门的前向训练机制打破了BP的局限性。具体来说,DFA-GNN扩展了DFA的原理以适应图数据和GNN的独特架构,将图拓扑信息纳入反馈连接中,以适应图数据的非欧几里得特性。此外,对于半监督图学习任务,我们开发了一个伪误差生成器,将训练数据的残差误差传播开来,为每个未标记节点生成伪误差。然后利用这些伪误差通过DFA训练GNN。在10个公共基准上的大量实验表明,我们的学习框架不仅优于以前的非BP方法,也优于标准的BP方法,并且对各种类型的噪声和攻击表现出优异的鲁棒性。

英文摘要:

Graph neural networks are recognized for their strong performance across various applications, with the backpropagation algorithm playing a central role in the development of most GNN models. However, despite its effectiveness, BP has limitations that challenge its biological plausibility and affect the efficiency, scalability and parallelism of training neural networks for graph-based tasks. While several non-BP training algorithms, such as the direct feedback alignment, have been successfully applied to fully-connected and convolutional network components for handling Euclidean data, directly adapting these non-BP frameworks to manage non-Euclidean graph data in GNN models presents significant challenges. These challenges primarily arise from the violation of the i.i.d. assumption in graph data and the difficulty in accessing prediction errors for all samples (nodes) within the graph. To overcome these obstacles, in this paper we propose DFA-GNN, a novel forward learning framework tailored for GNNs with a case study of semi-supervised learning. The proposed method breaks the limitations of BP by using a dedicated forward training mechanism. Specifically, DFA-GNN extends the principles of DFA to adapt to graph data and unique architecture of GNNs, which incorporates the information of graph topology into the feedback links to accommodate the non-Euclidean characteristics of graph data. Additionally, for semi-supervised graph learning tasks, we developed a pseudo error generator that spreads residual errors from training data to create a pseudo error for each unlabeled node. These pseudo errors are then utilized to train GNNs using DFA. Extensive experiments on 10 public benchmarks reveal that our learning framework outperforms not only previous non-BP methods but also the standard BP methods, and it exhibits excellent robustness against various types of noise and attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑