arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.19084cs.CL
收藏

GCoder:改进大型语言模型以实现泛化图问题求解

GCoder: Improving Large Language Model for Generalized Graph Problem Solving

Hong Kong University of Science and Technology (GuangZhou)(香港科技大学(广州));State Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)

Qifan Zhang, Xiaobin Hong, Jianheng Tang, Nuo Chen, Yuhan Li, Wenzhong Li, Jing Tang, Jia Li

AI总结:

本文提出基于代码的 GCoder,通过 GraphWild 数据集、SFT 与 RLCF 多阶段训练及混合检索,提升 LLM 对多样图问题和大规模图的泛化求解能力,准确率较 GPT-4o 平均提升 16.42%。

摘要:

大型语言模型(LLM)已展现出强大的推理能力,使其适合图计算等复杂任务。用于图问题的传统推理步骤范式受到步骤不可验证、长期推理有限以及对图变化泛化能力差的阻碍。为克服这些限制,我们提出 GCoder,这是一种基于代码的 LLM,旨在增强泛化图计算问题中的求解能力。我们的方法包括构建大规模训练数据集 GraphWild,该数据集具有多样的图格式和算法。我们采用多阶段训练流程,包括监督微调(SFT)和来自编译器反馈的强化学习(RLCF),以精炼模型能力。对于未见过的任务,使用混合检索技术来增强性能。实验表明,GCoder 的表现优于 GPT-4o,在各类图计算问题上平均准确率提升 16.42%。此外,GCoder 可高效处理具有数百万节点和多种输入格式的大规模图,克服了以往聚焦推理步骤范式的模型的局限。这一进展为使用 LLM 更直观、有效地求解图问题铺平了道路。代码和数据可在 https://github.com/Bkight999/WWW25-GCoder/tree/master 获取。

英文摘要:

Large Language Models (LLMs) have demonstrated strong reasoning abilities, making them suitable for complex tasks such as graph computation. Traditional reasoning steps paradigm for graph problems is hindered by unverifiable steps, limited long-term reasoning, and poor generalization to graph variations. To overcome these limitations, we introduce GCoder, a code-based LLM designed to enhance problem-solving in generalized graph computation problems. Our method involves constructing an extensive training dataset, GraphWild, featuring diverse graph formats and algorithms. We employ a multi-stage training process, including Supervised Fine-Tuning (SFT) and Reinforcement Learning from Compiler Feedback (RLCF), to refine model capabilities. For unseen tasks, a hybrid retrieval technique is used to augment performance. Experiments demonstrate that GCoder outperforms GPT-4o, with an average accuracy improvement of 16.42% across various graph computational problems. Furthermore, GCoder efficiently manages large-scale graphs with millions of nodes and diverse input formats, overcoming the limitations of previous models focused on the reasoning steps paradigm. This advancement paves the way for more intuitive and effective graph problem-solving using LLMs. Code and data are available at here: https://github.com/Bklight999/WWW25-GCoder/tree/master.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑