SpatialPIN：通过提示与交互3D先验增强视觉语言模型的空间推理能力
SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors
AI总结：
本研究提出无需训练的零样本框架SpatialPIN，通过提示并交互多3D基础模型先验增强VLM空间推理，在多类空间VQA及下游机器人任务中表现优异。
摘要：
当前最先进的增强空间推理能力的视觉语言模型（VLM）经训练后在空间视觉问答（VQA）任务上表现出色。但我们认为，更高级的3D感知任务，比如清晰描述动态场景变化和运动规划，需要超越现有空间VQA数据集的基础且明确的3D理解能力。本研究提出SpatialPIN框架，该框架以零样本、无需训练的方式，通过提示并与多个3D基础模型的先验交互，来增强VLM的空间推理能力。大量实验表明，注入空间推理能力的VLM在各类空间VQA任务上表现良好，还可拓展应用于拾取堆叠、轨迹规划等多种下游机器人任务。
英文摘要：
Current state-of-the-art spatial reasoning-enhanced VLMs are trained to excel at spatial visual question answering (VQA). However, we believe that higher-level 3D-aware tasks, such as articulating dynamic scene changes and motion planning, require a fundamental and explicit 3D understanding beyond current spatial VQA datasets. In this work, we present SpatialPIN, a framework designed to enhance the spatial reasoning capabilities of VLMs through prompting and interacting with priors from multiple 3D foundation models in a zero-shot, training-free manner. Extensive experiments demonstrate that our spatial reasoning-imbued VLM performs well on various forms of spatial VQA and can extend to help in various downstream robotics tasks such as pick and stack and trajectory planning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
