arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.02518cs.LGcs.AIcs.CL
收藏

GraphDancer: 通过两阶段课程后训练训练LLMs在图上的探索与推理

GraphDancer: Training LLMs to Explore and Reason over Graphs via Curriculum Reinforcement Learning

Texas A&M University(德克萨斯大学A&M分校);University of Waterloo(滑铁卢大学);Lambda(Lambda公司);University of Oregon(俄勒冈大学)

Yuyang Bai, Zhuofeng Li, Ping Nie, Jianwen Xie, Yu Wang, Yu Zhang

AI总结:

提出GraphDancer两阶段后训练框架,通过图感知课程逐步增加任务难度,使LLMs学会在异构图上进行自然语言推理与函数调用交织的探索与推理,仅用3B骨干模型即在跨域基准上超越更强基线。

摘要:

大型语言模型(LLMs)越来越依赖外部知识来提高事实性,然而许多真实世界的知识源被组织为异构图而非纯文本。在此类图上进行推理要求模型通过精确的函数调用遵循模式定义的关系,并在多轮交互中聚合证据。我们提出GraphDancer,一个两阶段后训练框架,通过将自然语言推理与图函数执行交织来教导LLMs在图上的推理。第一阶段教导模型在基于规则的奖励下如何与图交互,而第二阶段进一步教导其偏好更基于事实且高效的交互轨迹。GraphDancer的关键创新在于一个图感知课程,该课程根据信息寻求轨迹的结构复杂性组织两个阶段,在训练期间逐步增加任务难度。我们在一个多领域基准上评估GraphDancer,仅在一个领域上训练,并在未见过的领域和分布外问题类型上进行测试。尽管仅使用3B骨干模型,GraphDancer仍优于配备更大/更强骨干的基线,展示了图探索和推理技能的强大跨域泛化能力。我们的代码可在https://github.com/leopoldwhite/GraphDancer找到。

英文摘要:

Large language models (LLMs) increasingly rely on external knowledge to improve factuality, yet many knowledge sources on the Web, such as scholarly indexes and product catalogs, are organized as heterogeneous graphs. Reasoning over such graph-structured knowledge poses two key challenges: (1) navigating structured, schema-defined relations requires precise function calls beyond similarity-based retrieval, and (2) answering complex questions often demands multi-hop evidence aggregation through iterative information seeking. We study how LLMs can learn these interaction capabilities and transfer them to unseen domains. We propose GraphDancer, a reinforcement learning (RL) framework that teaches LLMs to navigate graphs by interleaving reasoning and function execution. To improve RL for moderate-sized LLMs, we introduce a graph-aware curriculum that schedules training by the structural difficulty of questions using an easy-to-hard biased sampler. We evaluate GraphDancer on the multi-domain GRBench benchmark, training only on Academic and Biomedical graphs. Testing covers held-out questions from these domains as well as unseen E-commerce, Literature, and Legal graphs. With either Qwen2.5-3B-Instruct or Qwen3-14B as the backbone, GraphDancer outperforms graph-agent baselines using the same backbone across all evaluation domains. These results indicate that the learned graph exploration and reasoning capabilities transfer across domains. On average across unseen domains, GraphDancer with a 14B backbone also outperforms baselines powered by the closed-source models GPT-4.1-mini and GPT-5-nano. Trajectory analyses further show that GraphDancer retrieves answer-bearing evidence more often and achieves higher accuracy when such evidence is found, while substantially reducing invalid-protocol outcomes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑