GTA：用于大语言模型算法图推理的图论智能体与基准
GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMs
浏览论文内容
中文总结 AI 辅助
提出图论基准GT Bench和智能体GTA，GTA通过偏好训练表示选择器与计划分解，显著提升LLM在图算法推理上的准确率，并跨基准迁移。
中文摘要 AI 辅助
大型语言模型（LLMs）越来越多地被要求对图等结构化数据进行推理，然而它们在语言中执行多步图算法的可靠性仍不清楚。现有的评估往往使用小图上的简单任务，侧重于代码生成而非对图本身的推理，或者固定单一输入格式。我们引入了图论基准（GT Bench），该基准涵盖24个经典图问题，分布在44种任务结构设置中，包含超过100,000个示例，涵盖四种表示形式：自然语言、结构化语言、邻接表和邻接矩阵。在GT Bench上评估八个LLM表明，准确性与输入表示密切相关，最佳表示随图密度、大小和拓扑以及模型而变化，并且这种敏感性在最强推理模型中依然存在，尽管有所减弱。基于这些观察，我们提出了图论智能体（GTA），它将偏好训练的表示选择器与围绕冻结执行器LLM的计划与分解脚手架相结合。GTA将Phi-4在基准的简单分割上从53.5%提升至69.1%，在困难分割上从33.0%提升至41.5%，优于八个提示和智能体基线，并且无需重新训练即可迁移到GraCoRe和NLGraph。基准生成和评估的代码：此https URL。项目主页可在以下网址获取：此https URL。
英文摘要
Large Language Models (LLMs) are increasingly asked to reason over structured data such as graphs, yet how reliably they can carry out multi-step graph algorithms in language remains unclear. Existing evaluations tend to use simple tasks on small graphs, to score code generation rather than reasoning over the graph itself, or to fix a single input format. We introduce Graph Theory Bench (GT Bench), a benchmark covering 24 classical graph problems in 44 task-structure settings, with over 100,000 examples across four representations: natural language, structured language, adjacency list, and adjacency matrix. Evaluating eight LLMs on GT Bench shows that accuracy is strongly tied to the input representation, that the best representation shifts with graph density, size, and topology as well as with the model, and that this sensitivity persists, attenuated, in the strongest reasoning models. Building on these observations, we propose the Graph Theory Agent (GTA), which pairs a preference-trained representation selector with plan-and-decompose scaffolding around a frozen executor LLM. GTA lifts Phi-4 from 53.5% to 69.1% on the benchmark's easy split and from 33.0% to 41.5% on its hard split, outperforming eight prompting and agent baselines, and transfers without retraining to GraCoRe and NLGraph. Code for benchmark generation and evaluation: https://github.com/xzx34/GTA. The project homepage is available at https://xzx34.github.io/gta/.
发表机构
- University of Southern California（南加州大学）
- University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
- Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
- University of Notre Dame（圣母大学）
- University of Chicago（芝加哥大学）
机构由 AI 辅助整理，请以论文原文为准。
