LRR-Bench：左、右还是旋转？视觉-语言模型在空间理解任务上仍面临挑战
LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks
AI总结：
LRR-Bench通过合成数据评估视觉-语言模型在空间理解任务上的性能，发现其在复杂任务上的表现远低于人类水平。
摘要：
现实中的应用，如自动驾驶和人形机器人操作，需要精确的空间感知。然而，如何让视觉-语言模型（VLMs）识别空间关系和感知空间运动仍是一个未充分探索的领域。在本工作中，我们介绍了一种空间评估流程，并构建了相应的基准测试集。具体而言，我们将空间理解分为两大类：绝对空间理解，涉及查询图像中物体的绝对空间位置（例如左、右）；以及三维空间理解，包括移动和旋转。值得注意的是，我们的数据集完全合成，使在低成本下生成测试样本成为可能，同时防止数据集污染。我们在多个最先进的VLMs上进行了实验，观察到它们的空间理解能力仍有很大提升空间。具体而言，在我们的实验中，人类在所有任务上均表现接近完美，而当前的VLMs仅在两个最简单的任务上达到人类水平。对于其余任务，VLMs的表现明显低于人类。事实上，表现最好的视觉-语言模型在多个任务上甚至接近零分。数据集和代码可在https://github.com/kong13661/LRR-Bench上获取。
英文摘要：
Real-world applications, such as autonomous driving and humanoid robot manipulation, require precise spatial perception. However, it remains underexplored how Vision-Language Models (VLMs) recognize spatial relationships and perceive spatial movement. In this work, we introduce a spatial evaluation pipeline and construct a corresponding benchmark. Specifically, we categorize spatial understanding into two main types: absolute spatial understanding, which involves querying the absolute spatial position (e.g., left, right) of an object within an image, and 3D spatial understanding, which includes movement and rotation. Notably, our dataset is entirely synthetic, enabling the generation of test samples at a low cost while also preventing dataset contamination. We conduct experiments on multiple state-of-the-art VLMs and observe that there is significant room for improvement in their spatial understanding abilities. Explicitly, in our experiments, humans achieve near-perfect performance on all tasks, whereas current VLMs attain human-level performance only on the two simplest tasks. For the remaining tasks, the performance of VLMs is distinctly lower than that of humans. In fact, the best-performing Vision-Language Models even achieve near-zero scores on multiple tasks. The dataset and code are available on https://github.com/kong13661/LRR-Bench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
