VTC-Bench：通过组合视觉工具链评估代理多模态模型
VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining
AI总结：
VTC-Bench通过组合视觉工具链评估多模态大语言模型的工具使用能力，揭示了当前模型在复杂任务中适应性和多工具组合方面的不足。
摘要：
近期进展将多模态大语言模型（MLLMs）从标准视觉问答扩展到利用外部工具进行高级视觉任务。尽管有进展，但精确执行和有效组合多样化工具仍是一个持续瓶颈。受限于稀疏的工具集和简单的工具使用轨迹，现有基准无法捕捉复杂的工具交互，无法在实际条件下评估模型性能。为此，我们引入了视觉工具链基准（VTC-Bench），该基准旨在评估MLLMs的工具使用能力。为了与现实计算机视觉管道对齐，我们的框架包含32种基于OpenCV的视觉操作。丰富的工具集使VTC-Bench能够广泛组合，从而严格评估多工具组合和长周期、多步骤计划执行。为了精确评估，我们提供了680个经过精心挑选的问题，分布在九个认知层次中，每个问题都有地面真相执行轨迹。在19种领先MLLMs上的广泛实验揭示了当前模型在视觉代理能力方面的关键限制。具体而言，模型在适应多样化工具集和泛化到未见过的操作方面存在困难，领先的模型Gemini-3.0-Pro仅在我们的基准上达到51%。此外，多工具组合仍是一个持续挑战。面对复杂任务时，模型难以制定高效的执行计划，依赖于狭窄的、次优的熟悉功能子集，而不是选择最优工具。通过识别这些根本挑战，VTC-Bench建立了严格的基准，以指导更通用的视觉代理模型的发展。
英文摘要：
Recent advancements extend Multimodal Large Language Models (MLLMs) beyond standard visual question answering to utilizing external tools for advanced visual tasks. Despite this progress, precisely executing and effectively composing diverse tools for complex tasks remain persistent bottleneck. Constrained by sparse tool-sets and simple tool-use trajectories, existing benchmarks fail to capture complex and diverse tool interactions, falling short in evaluating model performance under practical, real-world conditions. To bridge this gap, we introduce VisualToolChain-Bench(VTC-Bench), a comprehensive benchmark designed to evaluate tool-use proficiency in MLLMs. To align with realistic computer vision pipelines, our framework features 32 diverse OpenCV-based visual operations. This rich tool-set enables extensive combinations, allowing VTC-Bench to rigorously assess multi-tool composition and long-horizon, multi-step plan execution. For precise evaluation, we provide 680 curated problems structured across a nine-category cognitive hierarchy, each with ground-truth execution trajectories. Extensive experiments on 19 leading MLLMs reveal critical limitations in current models' visual agentic capabilities. Specifically, models struggle to adapt to diverse tool-sets and generalize to unseen operations, with the leading model Gemini-3.0-Pro only achieving 51% on our benchmark. Furthermore, multi-tool composition remains a persistent challenge. When facing complex tasks, models struggle to formulate efficient execution plans, relying heavily on a narrow, suboptimal subset of familiar functions rather than selecting the optimal tools. By identifying these fundamental challenges, VTC-Bench establishes a rigorous baseline to guide the development of more generalized visual agentic models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
