arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.01635cs.LGcs.AIcs.SI
收藏

图提示(Graph Prompt)有效吗?一个数据操作视角下的理论分析

Does Graph Prompt Work? A Data Operation Perspective with Theoretical Analysis

The Chinese University of Hong Kong(香港中文大学)

Qunzhong Wang, Xiangguo Sun, Hong Cheng

AI总结:

本文从数据操作视角建立理论框架,证明图提示能近似图变换算子,推导误差上界并分析误差分布,为图提示有效性提供理论保证。

摘要:

近年来,图提示(graph prompting)作为一个有前景的研究方向崭露头角,它允许学习附加到原始图上的额外标记(tokens)或子图,而无需在各种应用中重新训练预训练图模型。这一新颖范式,从传统的预训练和微调转向预训练和提示(prompting),在模拟图数据操作方面展现了显著的实证成功,应用范围从推荐系统到生物网络和图迁移。然而,尽管其潜力巨大,图提示的理论基础仍未得到充分探索,引发了关于其基本有效性的关键问题。缺乏严格的理论证明来解释其为何有效以及效果多大,更像是一朵笼罩在图提示领域进一步发展的乌云。为填补这一空白,本文引入了一个理论框架,从数据操作视角严格分析图提示。我们的贡献有三方面:首先,我们提供了一个形式化的保证定理,证明图提示近似图变换算子的能力,有效连接上游和下游任务。其次,我们推导了图提示对单个图进行数据操作误差的上界,并将这一讨论扩展到图模型训练中常见的图批次。第三,我们分析了数据操作误差的分布,将我们的理论发现从线性图模型(如GCN)扩展到非线性图模型(如GAT)。大量实验支持了我们的理论结果,并确认了这些保证的实际意义。

英文摘要:

In recent years, graph prompting has emerged as a promising research direction, enabling the learning of additional tokens or subgraphs appended to the original graphs without requiring retraining of pre-trained graph models across various applications. This novel paradigm, shifting from the traditional pretraining and finetuning to pretraining and prompting has shown significant empirical success in simulating graph data operations, with applications ranging from recommendation systems to biological networks and graph transferring. However, despite its potential, the theoretical underpinnings of graph prompting remain underexplored, raising critical questions about its fundamental effectiveness. The lack of rigorous theoretical proof of why and how much it works is more like a dark cloud over the graph prompt area to go further. To fill this gap, this paper introduces a theoretical framework that rigorously analyzes graph prompting from a data operation perspective. Our contributions are threefold: First, we provide a formal guarantee theorem, demonstrating graph prompts capacity to approximate graph transformation operators, effectively linking upstream and downstream tasks. Second, we derive upper bounds on the error of these data operations by graph prompts for a single graph and extend this discussion to batches of graphs, which are common in graph model training. Third, we analyze the distribution of data operation errors, extending our theoretical findings from linear graph models (e.g., GCN) to non-linear graph models (e.g., GAT). Extensive experiments support our theoretical results and confirm the practical implications of these guarantees.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑