通过指代表达探索空间语言 grounding
Exploring Spatial Language Grounding Through Referring Expressions
AI总结：
本文提出用指代表达理解任务评估视觉-语言模型的空间推理能力，重点考察目标检测歧义、复杂多关系空间表达和否定表达，揭示了不同模型在拓扑、方向、邻近等空间语义类别上的挑战与行为差异。
摘要：
空间推理是人类认知的重要组成部分，也是最新视觉-语言模型（VLMs）表现出困难的一个领域。目前的分析工作使用图像描述任务和视觉问答。在这项工作中，我们提出改用指代表达理解任务作为评估 VLMs 空间推理的平台。该平台为在以下情况下更深入地分析空间理解和 grounding 能力提供了机会：1）目标检测中存在歧义；2）具有较长句子结构和多个空间关系的复杂空间表达；3）带否定（“不”）的表达。在我们的分析中，我们使用任务特定架构以及大型 VLMs，并突出它们在处理这些特定情况时的优势与不足。虽然所有这些模型都面临当前任务的挑战，但相对行为取决于底层模型以及空间语义的具体类别（拓扑、方向、邻近等）。我们的结果突显了这些挑战和行为，并为研究空白和未来方向提供了见解。
英文摘要：
Spatial Reasoning is an important component of human cognition and is an area in which the latest Vision-language models (VLMs) show signs of difficulty. The current analysis works use image captioning tasks and visual question answering. In this work, we propose using the Referring Expression Comprehension task instead as a platform for the evaluation of spatial reasoning by VLMs. This platform provides the opportunity for a deeper analysis of spatial comprehension and grounding abilities when there is 1) ambiguity in object detection, 2) complex spatial expressions with a longer sentence structure and multiple spatial relations, and 3) expressions with negation ('not'). In our analysis, we use task-specific architectures as well as large VLMs and highlight their strengths and weaknesses in dealing with these specific situations. While all these models face challenges with the task at hand, the relative behaviors depend on the underlying models and the specific categories of spatial semantics (topological, directional, proximal, etc.). Our results highlight these challenges and behaviors and provide insight into research gaps and future directions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
