arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01400cs.LGcs.AI
收藏

解耦加权与选择以整合多个图预训练任务

Decoupling Weighing and Selecting for Integrating Multiple Graph Pre-training Tasks

Westlake University(西湖大学);Zhejiang University(浙江大学)

Tianyu Fan, Lirong Wu, Yufei Huang, Haitao Lin, Cheng Tan, Zhangyang Gao, Stan Z. Li

AI总结:

针对多图预训练任务整合中选择与加权耦合的问题,提出WAS框架,用解耦孪生网络为每个实例自适应选择任务并学习实例级加权,在16个数据集上达到领先性能。

摘要:

近年来,图预训练在图表示学习中取得了巨大成功。随着数百种图预训练任务被提出,整合从多个预训练任务中获得的知识已成为一个热门研究课题。本文针对该课题识别出两个重要的协作过程:(1)选择:如何根据任务间的兼容性,从给定任务池中选择最优任务组合;(2)加权:如何根据所选任务的重要性对其进行加权。尽管目前已有大量工作聚焦于加权,但投入到选择上的努力相对较少。本文提出了一种新颖的实例级框架,用于整合多个图预训练任务,即 Weigh And Select(WAS);在该框架中,加权与选择这两个协作过程通过解耦的孪生网络结合起来。具体而言,它首先针对每个实例,从给定任务池中自适应地学习最优任务组合,并在此基础上学习定制化的实例级任务加权策略。在涵盖节点级和图级下游任务的16个图数据集上进行的大量实验表明,通过组合少数简单但经典的任务,WAS能够取得与其他领先同类方法相当的性能。代码可在 https://github.com/TianyuFan0504/WAS 获取。

英文摘要:

Recent years have witnessed the great success of graph pre-training for graph representation learning. With hundreds of graph pre-training tasks proposed, integrating knowledge acquired from multiple pre-training tasks has become a popular research topic. In this paper, we identify two important collaborative processes for this topic: (1) select: how to select an optimal task combination from a given task pool based on their compatibility, and (2) weigh: how to weigh the selected tasks based on their importance. While there currently has been a lot of work focused on weighing, comparatively little effort has been devoted to selecting. This paper proposes a novel instance-level framework for integrating multiple graph pre-training tasks, Weigh And Select (WAS), where the two collaborative processes, weighing and selecting, are combined by decoupled siamese networks. Specifically, it first adaptively learns an optimal combination of tasks for each instance from a given task pool, based on which a customized instance-level task weighing strategy is learned. Extensive experiments on 16 graph datasets across node-level and graph-level downstream tasks have demonstrated that by combining a few simple but classical tasks, WAS can achieve comparable performance to other leading counterparts. The code is available at https://github.com/TianyuFan0504/WAS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑