arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.03214cs.CLcs.AIcs.CV
收藏

iVISPAR——一个面向 VLM 的交互式视觉空间推理基准

iVISPAR -- An Interactive Visual-Spatial Reasoning Benchmark for VLMs

Institute of Cognitive Science, Osnabrück University(认知科学研究所,奥斯纳布吕克大学)

Julius Mayer, Mohamad Ballout, Serwan Jassim, Farbod Nosrat Nezami, Elia Bruni

AI总结:

该研究提出交互式多模态基准 iVISPAR,通过滑块拼图变体在 3D、2D 和文本模态下评估 VLM 智能体的空间推理,并结合最优解与人类基线揭示其视觉对齐差距。

摘要:

众所周知,视觉语言模型(VLM)在空间推理和视觉对齐方面存在困难。为帮助克服这些局限,我们提出 iVISPAR,这是一个交互式多模态基准,旨在评估 VLM 作为智能体时的空间推理能力。iVISPAR 基于滑块拼图的一种变体,这一经典问题需要逻辑规划、空间意识和多步推理。该基准支持视觉 3D、2D 和基于文本的输入模态,从而能够全面评估 VLM 的规划与推理技能。我们评估了一系列广泛的、最先进的开源和闭源 VLM,比较它们的表现,同时提供最优路径解和人类基线,以评估任务对人类的复杂性和可行性。结果表明,尽管 VLM 在 2D 任务上的表现优于 3D 或基于文本的设置,但它们在复杂空间配置上仍有困难,并且始终不及人类表现,这说明视觉对齐仍是持续存在的挑战。这凸显了当前 VLM 能力中的关键差距,揭示了它们在实现人类级认知方面的局限。项目网站:https://microcosm.ai/ivispar

英文摘要:

Vision-Language Models (VLMs) are known to struggle with spatial reasoning and visual alignment. To help overcome these limitations, we introduce iVISPAR, an interactive multimodal benchmark designed to evaluate the spatial reasoning capabilities of VLMs acting as agents. \mbox{iVISPAR} is based on a variant of the sliding tile puzzle, a classic problem that demands logical planning, spatial awareness, and multi-step reasoning. The benchmark supports visual 3D, 2D, and text-based input modalities, enabling comprehensive assessments of VLMs' planning and reasoning skills. We evaluate a broad suite of state-of-the-art open-source and closed-source VLMs, comparing their performance while also providing optimal path solutions and a human baseline to assess the task's complexity and feasibility for humans. Results indicate that while VLMs perform better on 2D tasks compared to 3D or text-based settings, they struggle with complex spatial configurations and consistently fall short of human performance, illustrating the persistent challenge of visual alignment. This underscores critical gaps in current VLM capabilities, highlighting their limitations in achieving human-level cognition. Project website: https://microcosm.ai/ivispar

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑