TIR-Bench：面向智能体式“以图思考”推理的综合基准
TIR-Bench: A Comprehensive Benchmark for Agentic Thinking-with-Images Reasoning
AI总结：
本文提出 TIR-Bench 综合基准，通过 13 个需要新颖工具使用的图像处理任务评估 22 个多模态大语言模型的智能体式“以图思考”能力，并初步比较直接微调与智能体式微调。
摘要：
视觉推理的前沿正在转向像 OpenAI o3 这样的模型，它们能够智能地创建和操作工具来变换图像以解决问题，这也被称为思维链中的“以图思考”。然而，现有基准未能充分捕捉这种高级能力。即便是当前“以图思考”方法最常用的 Visual Search 基准，也只测试定位和裁剪等基本操作，对更复杂、动态且依赖工具的推理几乎无法提供深入认识。我们提出 TIR-Bench，一个用于评估智能体式“以图思考”能力的综合基准，涵盖 13 个多样化任务，每个任务都要求在思维链中进行新颖的工具使用以完成图像处理与操作。我们评估了 22 个多模态大语言模型（MLLMs），包括领先的开源模型、闭源模型以及具有显式工具使用增强的模型。结果表明，TIR-Bench 对所有模型都具有普遍挑战性，而优异表现需要真正的“以图思考”能力。最后，我们展示了一项初步研究，比较直接微调与智能体式微调。
英文摘要：
The frontier of visual reasoning is shifting toward models like OpenAI o3, which can intelligently create and operate tools to transform images for problem-solving, also known as thinking-\textit{with}-images in chain-of-thought. Yet existing benchmarks fail to fully capture this advanced capability. Even Visual Search, the most common benchmark for current thinking-\textit{with}-images methods, tests only basic operations such as localization and cropping, offering little insight into more complex, dynamic, and tool-dependent reasoning. We introduce \textbf{TIR-Bench}, a comprehensive benchmark for evaluating agentic thinking-with-images across 13 diverse tasks, each requiring novel tool use for image processing and manipulation in chain-of-thought. We evaluate 22 multimodal large language models (MLLMs), from leading open-sourced and proprietary models to those with explicit tool-use augmentation. Results show that TIR-Bench is universally challenging, and strong performance requires genuine thinking-with-images capabilities. Finally, we present a pilot study comparing direct versus agentic fine-tuning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
