迈向通过重构图指令微调的大型语言模型的图-词化
Toward Graph-Tokenizing Large Language Models with Reconstructive Graph Instruction Tuning
AI总结：
本文提出RGLM方法，通过重构图信息和显式图监督改进图-词化LLMs的对齐效果，提升对图结构的理解与利用。
摘要：
大型语言模型（LLMs）的显著成功促使研究者将它们适应为各种图相关任务的通用预测器，最终目标是开发一个能够泛化各种场景的图基础模型。关键挑战是将图数据与语言空间对齐，使LLMs能够更好地理解图。作为流行范式，图-词化LLMs（GTokenLLMs）将复杂结构和长文本编码为图令牌序列，然后通过语言指令微调将其与文本令牌对齐。尽管它们最初的成功，我们的信息论分析揭示现有GTokenLLMs仅依赖语言指令中的文本监督，实现了仅隐式的图-文本对齐，导致文本主导偏差，未能充分利用图上下文。为克服这一限制，我们首先证明对齐目标上界由输入图与其隐藏表示之间的互信息决定，这促使我们改进这一上界以实现更好的对齐。为此，我们进一步提出一个重构图指令微调流程，RGLM。我们的核心思想是从LLM的图令牌输出中重构图信息，显式地将图监督纳入对齐过程。技术上，我们通过从两个互补视角探索三个不同的变体来体现RGLM：RGLM-Decoder从输入空间；RGLM-Similarizer和RGLM-Denoiser从潜在空间。此外，我们理论分析了每个变体的对齐有效性。在各种基准和任务场景上的广泛实验验证了所提RGLM的有效性，为GTokenLLMs对齐研究开辟了新方向。
英文摘要：
The remarkable success of large language models (LLMs) has motivated researchers to adapt them as universal predictors for various graph-related tasks, with the ultimate goal of developing a graph foundation model that generalizes diverse scenarios. The key challenge is to align graph data with language spaces so that LLMs can better comprehend graphs. As a popular paradigm, Graph-Tokenizing LLMs (GTokenLLMs) encode complex structures and lengthy texts into a graph token sequence, and then align them with text tokens via language instructions tuning. Despite their initial success, our information-theoretic analysis reveals that existing GTokenLLMs rely solely on text supervision from language instructions, which achieve only implicit graph-text alignment, resulting in a text-dominant bias that underutilizes graph context. To overcome this limitation, we first prove that the alignment objective is upper-bounded by the mutual information between the input graphs and their hidden representations in the LLM, which motivates us to improve this upper bound to achieve better alignment. To this end, we further propose a reconstructive graph instruction tuning pipeline, RGLM. Our key idea is to reconstruct the graph information from the LLM's graph token outputs, explicitly incorporating graph supervision to constrain the alignment process. Technically, we embody RGLM by exploring three distinct variants from two complementary perspectives: RGLM-Decoder from the input space; RGLM-Similarizer and RGLM-Denoiser from the latent space. Additionally, we theoretically analyze the alignment effectiveness of each variant. Extensive experiments on various benchmarks and task scenarios validate the effectiveness of the proposed RGLM, paving the way for new directions in GTokenLLMs' alignment research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
