arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08043cs.LGcs.CL
收藏

GraphPrompt:统一图神经网络的预训练和下游任务

GraphPrompt: Unifying Pre-Training and Downstream Tasks for Graph Neural Networks

National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Singapore Management University(新加坡管理大学)

Zemin Liu, Xingtong Yu, Yuan Fang, Xinming Zhang

摘要:

图形可以对对象之间的复杂关系进行建模,从而支持大量的Web应用程序,例如在线页面/文章分类和社交推荐。虽然图神经网络(GNN)已经成为图表示学习的强大工具,但在端到端的监督环境中,它们的性能严重依赖于大量特定于任务的监督。为了减少标记要求,“预训练,微调”和“预训练,提示”范例已经变得越来越普遍。特别是,提示是自然语言处理中微调的流行替代方案,其旨在以特定于任务的方式缩小预训练和下游目标之间的差距。然而,现有的图上的提示的研究仍然是有限的,缺乏一个通用的处理呼吁不同的下游任务。在本文中,我们提出了GraphPrompt,一种新的图上的预训练和提示框架。GraphPrompt不仅将预训练和下游任务统一到一个通用的任务模板中,而且还采用可学习的提示来帮助下游任务以特定于任务的方式从预训练模型中定位最相关的知识。最后,我们在五个公共数据集上进行了广泛的实验,以评估和分析GraphPrompt。

英文摘要:

Graphs can model complex relationships between objects, enabling a myriad of Web applications such as online page/article classification and social recommendation. While graph neural networks(GNNs) have emerged as a powerful tool for graph representation learning, in an end-to-end supervised setting, their performance heavily rely on a large amount of task-specific supervision. To reduce labeling requirement, the "pre-train, fine-tune" and "pre-train, prompt" paradigms have become increasingly common. In particular, prompting is a popular alternative to fine-tuning in natural language processing, which is designed to narrow the gap between pre-training and downstream objectives in a task-specific manner. However, existing study of prompting on graphs is still limited, lacking a universal treatment to appeal to different downstream tasks. In this paper, we propose GraphPrompt, a novel pre-training and prompting framework on graphs. GraphPrompt not only unifies pre-training and downstream tasks into a common task template, but also employs a learnable prompt to assist a downstream task in locating the most relevant knowledge from the pre-train model in a task-specific manner. Finally, we conduct extensive experiments on five public datasets to evaluate and analyze GraphPrompt.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑