GraphFormers：用于文本图表示学习的GNN嵌套Transformer
GraphFormers: GNN-nested Transformers for Representation Learning on Textual Graph
AI总结：
针对文本图表示学习中级联架构文本特征独立建模的局限，提出嵌套分层GNN的GraphFormers模型，结合渐进式学习策略，在三个大规模基准上以相当效率超越SOTA基线。
摘要：
文本图上的表示学习旨在基于节点各自的文本特征和邻域信息，为节点生成低维嵌入。预训练语言模型和图神经网络（GNN）的最新突破推动了相关技术的发展。现有研究主要依赖级联模型架构：首先由语言模型独立编码节点的文本特征，随后通过图神经网络聚合文本嵌入。然而，上述架构因对文本特征的独立建模而存在局限性。在本研究中，我们提出GraphFormers，将分层GNN组件嵌套在语言模型的Transformer块旁。通过该架构，文本编码与图聚合被融合为一个迭代工作流，使每个节点的语义能从全局视角被准确理解。此外，我们引入了一种渐进式学习策略，让模型依次在经过处理的数据和原始数据上训练，以增强其在图上整合信息的能力。我们在三个大规模基准数据集上开展了广泛评估，结果显示GraphFormers在运行效率相当的情况下，性能优于现有最优（SOTA）基线模型。
英文摘要：
The representation learning on textual graph is to generate low-dimensional embeddings for the nodes based on the individual textual features and the neighbourhood information. Recent breakthroughs on pretrained language models and graph neural networks push forward the development of corresponding techniques. The existing works mainly rely on the cascaded model architecture: the textual features of nodes are independently encoded by language models at first; the textual embeddings are aggregated by graph neural networks afterwards. However, the above architecture is limited due to the independent modeling of textual features. In this work, we propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models. With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow, {making} each node's semantic accurately comprehended from the global perspective. In addition, a {progressive} learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph. Extensive evaluations are conducted on three large-scale benchmark datasets, where GraphFormers outperform the SOTA baselines with comparable running efficiency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
