arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.01436cs.LGcs.AI
收藏

基于GPT-4的图神经架构搜索

Graph Neural Architecture Search with GPT-4

Zhejiang Key Laboratory of Accessible Perception and Intelligent Systems(浙江可感知智能系统重点实验室);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Public Health, Zhejiang University(浙江大学公共卫生学院);Cyberspace Institute of Advanced Technology, Guangzhou University(广州大学网络空间研究院);Department of Computer Science, University of Illinois at Chicago(伊利诺伊大学芝加哥分校计算机科学系)

Haishuai Wang, Yang Gao, Xin Zheng, Peng Zhang, Jiajun Bu, Philip S. Yu

AI总结:

本文提出GNAS-LLM,将大语言模型与图神经架构搜索结合,通过设计提示引导GPT-4生成架构,在四个基准数据集上平均提升验证集0.7%、测试集0.3%。

摘要:

图神经架构搜索(GNAS)在给定图数据集上寻找最佳图神经网络架构方面已显示出有前景的结果。然而,现有的GNAS方法在设计搜索空间和搜索策略时仍然需要大量的人工劳动和丰富的领域知识。为此,我们将大型语言模型(LLMs)集成到GNAS中,并提出了一种基于LLMs的新型GNAS模型(简称GNAS-LLM)。GNAS-LLM的基本思想是为LLMs设计一类新的GNAS提示,引导LLMs理解图神经架构的生成任务。这些提示包括GNAS的搜索空间、搜索策略和搜索反馈的描述。通过迭代运行带有这些提示的LLMs,GNAS-LLM能够生成更准确的图神经架构,并实现快速收敛。实验结果表明,在四个基准图数据集上,GNAS-LLM优于最先进的GNAS方法,在验证集上平均提升0.7%,在测试集上平均提升0.3%。此外,基于AutoGEL的搜索空间,GNAS-LLM在测试集上平均提升1.0%。

英文摘要:

Graph Neural Architecture Search (GNAS) has shown promising results in finding the best graph neural network architecture on a given graph dataset. However, existing GNAS methods still require intensive human labor and rich domain knowledge when designing the search space and search strategy. To this end, we integrate Large Language Models (LLMs) into GNAS and present a new GNAS model based on LLMs (GNAS-LLM for short). The basic idea of GNAS-LLM is to design a new class of GNAS prompts for LLMs to guide LLMs towards understanding the generative task of graph neural architectures. The prompts consist of descriptions of the search space, search strategy, and search feedback of GNAS. By iteratively running LLMs with the prompts, GNAS-LLM generates more accurate graph neural network architectures with fast convergence. Experimental results show that GNAS-LLM outperforms the state-of-the-art GNAS methods on four benchmark graph datasets, with an average improvement of 0.7% on the validation sets and 0.3% on the test sets. Besides, GNAS-LLM achieves an average improvement of 1.0% on the test sets based on the search space from AutoGEL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑