GraphGPT：生成式预训练图欧拉Transformer
GraphGPT: Generative Pre-trained Graph Eulerian Transformer
AI总结：
GraphGPT提出基于图欧拉Transformer的自监督生成式预训练模型，通过将图可逆地转换为token序列并进行下一token或掩码token预测预训练，在OGB多个数据集上达到领先性能，并可扩展至20亿参数，突破传统图模型的可扩展性瓶颈。
摘要：
我们提出了GraphGPT，这是一种基于图欧拉Transformer（Graph Eulerian Transformer，GET）的新型自监督生成式预训练模型，用于图学习。首先，我们提出了GET，它将标准Transformer编码器或解码器架构与一种创新的图到序列转换方法相结合。该方法利用欧拉路径，以可逆的方式将图或采样子图转换为表示节点、边和属性的token序列。我们使用两种自监督任务之一对GET进行预训练：下一token预测（next-token prediction，NTP）和调度掩码token预测（scheduled masked-token prediction，SMTP）。预训练模型随后针对下游任务（如图级、边级和节点级预测）进行微调。尽管设计简洁，GraphGPT在多个大规模开放图基准（Open Graph Benchmark，OGB）数据集上取得了与最先进方法相当或更优的性能。它在分子性质预测数据集PCQM4Mv2和蛋白质-蛋白质相互作用数据集ogbl-ppa上展现了卓越的结果。值得注意的是，生成式预训练使GraphGPT能够扩展到20亿参数，同时保持性能提升——这一突破克服了传统图神经网络（Graph Neural Networks，GNNs）和先前图Transformer（graph transformers，GTs）的可扩展性限制。为了推进图基础模型研究，并促进化学、材料科学及相关领域的科学发现，我们将发布源代码（https://github.com/alibaba/graph-gpt）和预训练检查点。
英文摘要：
We introduceGraphGPT, a novel self-supervised generative pre-trained model for graph learning based on the Graph Eulerian Transformer (GET). First, we propose GET, which combines a standard transformer encoder or decoder architecture with an innovative graph-to-sequence transformation method. This method converts graphs or sampled subgraphs into sequences of tokens representing nodes, edges, and attributes in a reversible manner using Eulerian paths. We pre-train GET using either of the two self-supervised tasks: next-token prediction (NTP) and scheduled masked-token prediction (SMTP). The pre-trained model is then fine-tuned for downstream tasks such as graph-, edge-, and node-level prediction. Despite its simplicity, GraphGPT achieves performance comparable to or surpassing state-of-the-art methods on multiple large-scale Open Graph Benchmark (OGB) datasets. It demonstrates exceptional results on the molecular property prediction dataset PCQM4Mv2 and the protein-protein interaction dataset ogbl-ppa. Notably, generative pre-training enables scaling GraphGPT to 2 billion parameters while maintaining performance gains - a breakthrough that overcomes the scalability limitations of traditional Graph Neural Networks (GNNs) and prior graph transformers (GTs). To advance research in graph foundation models and facilitate scientific discovery in chemistry, materials science, and related fields, we will release the source code (https://github.com/alibaba/graph-gpt) and pre-trained checkpoints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
