arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.21167cs.AI
收藏

GRADS:具有可控难度的图调控智能体数据合成

GRADS: Graph-Regulated Agentic Data Synthesis with Controllable Difficulty

Johns Hopkins University(约翰斯·霍普金斯大学);Nanjing University(南京大学);Amazon(亚马逊)

Gaocheng Yang, Ruijie Tao, Kai Ye, Senhua Zhang, Zhengqin Zhu, Ziyang Li

AI总结:

针对多跳长 horizon 智能体任务的合成数据难度不匹配问题,提出 GRADS 框架,通过图调控实现可控难度数据生成,训练后可提升模型性能,4B 模型在多领域表现优异。

摘要:

深度研究智能体越来越多地被部署用于从大型语料库中检索精确信息,但多跳、长 horizon 任务仍然具有挑战性。合成数据生成已成为一种可扩展的解决方案,现有流程通常通过扩展种子任务来增加其表面深度和广度。然而,这种扩展并不能可靠地转化为必要的推理:新引入的证据可能仅与答案弱相关,导致预期难度与实际难度不匹配。为解决该问题,我们提出 GRADS,一种具有显式难度控制的图引导智能体数据合成框架。GRADS 运行种子选择、证据扩展、问答构建和验证的闭环,同时维护用于推导每个答案所需的证据和推理依赖关系图。在每一步,它都基于相关子图条件化生成,保留必要的推理而不积累无关上下文。在安全、软件工程和科学文档领域的实验表明,GRADS 始终生成具有经验分离难度分布的已验证任务,该分布由独立求解器测量。此外,在匹配的训练预算下,基于此数据进行后训练在各个层面均带来一致增益:SFT 将解决方案终止率提高至 95%以上,同时将工具调用和回合数减少多达 75%;RL 进一步将工具使用减少多达 19.5%;在我们保留的领域特定基准上,得到的 4B 模型在软件工程领域与 Sonnet 4 相当,在科学和安全领域则超越了它。

英文摘要:

Deep research agents are increasingly deployed to retrieve precise information from large corpora, yet multi-hop, long-horizon tasks remain challenging. Synthetic data generation has emerged as a scalable solution, with existing pipelines typically expanding seed tasks to increase their apparent depth and breadth. However, this expansion does not reliably translate into necessary reasoning: newly introduced evidence may be only weakly coupled to the answer, creating a mismatch between intended and empirical difficulty. To address this problem, we introduce GRADS, a graph-guided agentic data synthesis framework with explicit difficulty control. GRADS runs a closed loop of seed selection, evidence expansion, question--answer construction, and verification while maintaining a graph of the evidence and inference dependencies required to derive each answer. At each step, it conditions generation on a relevant subgraph, preserving indispensable inference without accumulating irrelevant context. Experiments across security, software-engineering, and scientific-document domains show that GRADS consistently generates verified tasks with empirically separated difficulty distributions, as measured by independent solvers. Moreover, under matched training budgets, post-training on this data delivers consistent gains at every level: SFT raises the solution-termination rate to over 95% while cutting tool calls and turns by up to 75%, RL trims tool use by up to a further 19.5%, and on our held-out, domain-specific benchmarks, the resulting 4B model matches Sonnet~4 on software engineering and surpasses it on the scientific and security domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑