arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.01290cs.CL
收藏

ToolComp:一个多工具推理与过程监督基准

ToolComp: A Multi-Tool Reasoning & Process Supervision Benchmark

Scale AI

Vaskar Nath, Pranav Raja, Claire Yoon, Sean Hendryx

AI总结:

ToolComp 基准通过人工验证的提示、答案与过程监督标签评估多步工具使用推理,并表明过程监督奖励模型比结果监督奖励模型具有显著更好的泛化能力。

摘要:

尽管人工智能近期取得进展,但开发能够执行涉及多个工具的复杂多步推理任务的系统仍然是一个重大挑战。现有基准未能充分捕捉工具使用推理在真实世界中的复杂性,在这种推理中,不仅验证最终答案的正确性,而且验证中间步骤的正确性,对于评估、开发以及在推理过程中识别失败都十分重要。为了弥合这一差距,我们提出了 ToolComp,一个旨在评估多步工具使用推理的综合基准。ToolComp 通过模型与人类标注者之间的协作开发而成,具有经过人工编辑/验证的提示、最终答案和过程监督标签,从而能够同时评估最终结果和中间推理。在六个不同模型家族上的评估表明了我们数据集的挑战性,大多数模型的准确率低于 50%。此外,我们生成合成训练数据,以比较结果监督奖励模型(ORMs)与过程监督奖励模型(PRMs)的性能,评估它们在 ToolComp 所衡量的复杂工具使用推理上的改进能力。我们的结果表明,PRMs 的泛化能力显著优于 ORMs,在对基础模型和微调模型轨迹进行排序时,rank@1 准确率分别提高了 19% 和 11%。这些发现凸显了过程监督在 AI 模型评估和训练中的关键作用,为在复杂的多步工具使用任务中构建更稳健、能力更强的系统铺平了道路。

英文摘要:

Despite recent advances in AI, the development of systems capable of executing complex, multi-step reasoning tasks involving multiple tools remains a significant challenge. Current benchmarks fall short in capturing the real-world complexity of tool-use reasoning, where verifying the correctness of not only the final answer but also the intermediate steps is important for evaluation, development, and identifying failures during inference time. To bridge this gap, we introduce ToolComp, a comprehensive benchmark designed to evaluate multi-step tool-use reasoning. ToolComp is developed through a collaboration between models and human annotators, featuring human-edited/verified prompts, final answers, and process supervision labels, allowing for the evaluation of both final outcomes and intermediate reasoning. Evaluation across six different model families demonstrates the challenging nature of our dataset, with the majority of models achieving less than 50% accuracy. Additionally, we generate synthetic training data to compare the performance of outcome-supervised reward models (ORMs) with process-supervised reward models (PRMs) to assess their ability to improve complex tool-use reasoning as evaluated by ToolComp. Our results show that PRMs generalize significantly better than ORMs, achieving a 19% and 11% improvement in rank@1 accuracy for ranking base and fine-tuned model trajectories, respectively. These findings highlight the critical role of process supervision in both the evaluation and training of AI models, paving the way for more robust and capable systems in complex, multi-step tool-use tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑