arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.11478cs.CLcs.AIcs.LG
收藏

每个图都是一门新语言:基于大语言模型的图学习

Each Graph is a New Language: Graph Learning with LLMs

First Author Affiliation(第一作者机构)

Huachi Zhou, Jiahe Du, Chuang Zhou, Chang Yang, Yilin Xiao, Yuxuan Xie, Xiao Huang

AI总结:

提出GDL4LLM框架,将图转化为图语言语料库并预训练大语言模型,使其能简洁充分地建模图结构,在节点分类任务上优于现有基线。

摘要:

近期的工作利用大语言模型(LLMs)对文本属性图结构进行建模,以完成节点分类任务。这些方法通过图结构描述让LLMs理解图,或通过图结构聚合LLM生成的文本属性嵌入。然而,这些方法在用LLMs建模图结构时面临两个主要局限:(i)图描述在描述高阶图结构时变得冗长;(ii)仅文本属性不包含足够的图结构信息。用LLMs简洁且充分地建模图结构具有挑战性。LLMs缺乏直接建模图结构的内置机制,也难以处理高阶节点与目标节点之间的复杂长程依赖。受LLMs在一种语言上预训练后,通过极少的额外训练即可在另一种语言上取得优异表现这一观察的启发,我们提出了图定义语言用于大语言模型(GDL4LLM)。这一新颖框架使LLMs能够将其强大的语言理解能力迁移到图结构数据上。GDL4LLM将图转换为图语言语料库而非图描述,并在该语料库上预训练LLMs以充分理解图结构。在微调阶段,该语料库仅用少量标记即可简洁地描述目标节点的结构信息。通过将图视为一种新语言,GDL4LLM使LLMs能够为节点分类任务充分且简洁地建模图结构。在三个真实世界数据集上的大量实验表明,GDL4LLM通过高效建模不同阶的图结构,优于基于描述和基于文本属性嵌入的基线方法。

英文摘要:

Recent efforts leverage Large Language Models (LLMs) for modeling text-attributed graph structures in node classification tasks. These approaches describe graph structures for LLMs to understand or aggregate LLM-generated textual attribute embeddings through graph structure. However, these approaches face two main limitations in modeling graph structures with LLMs. (i) Graph descriptions become verbose in describing high-order graph structure. (ii) Textual attributes alone do not contain adequate graph structure information. It is challenging to model graph structure concisely and adequately with LLMs. LLMs lack built-in mechanisms to model graph structures directly. They also struggle with complex long-range dependencies between high-order nodes and target nodes. Inspired by the observation that LLMs pre-trained on one language can achieve exceptional performance on another with minimal additional training, we propose \textbf{G}raph-\textbf{D}efined \textbf{L}anguage for \textbf{L}arge \textbf{L}anguage \textbf{M}odel (GDL4LLM). This novel framework enables LLMs to transfer their powerful language understanding capabilities to graph-structured data. GDL4LLM translates graphs into a graph language corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand graph structures. During fine-tuning, this corpus describes the structural information of target nodes concisely with only a few tokens. By treating graphs as a new language, GDL4LLM enables LLMs to model graph structures adequately and concisely for node classification tasks. Extensive experiments on three real-world datasets demonstrate that GDL4LLM outperforms description-based and textual attribute embeddings-based baselines by efficiently modeling different orders of graph structure with LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑