SceneFunRI:为任务驱动的功能物体定位推理不可见区域
SceneFunRI: Reasoning the Invisible for Task-Driven Functional Object Localization
AI总结:
SceneFunRI通过半自动化流程构建了855个实例的基准,要求模型根据任务指令和常识推理推断不可见功能物体的位置,现有模型在推理稳定性上仍有不足。
摘要:
在现实场景中,目标物体可能位于不可见的区域。尽管人类可以通过上下文和常识知识推断被遮挡物体的位置,但这一能力仍是视觉-语言模型(VLMs)的主要挑战。为此,我们引入了SceneFunRI基准,用于推理不可见区域。基于SceneFun3D数据集,SceneFunRI通过半自动化流程将任务转化为二维空间推理问题,并包含855个实例。该基准要求模型根据任务指令和常识推理推断不可见功能物体的位置。最强基线模型(Gemini 3 Flash)仅达到CAcc@75为15.20,mIoU为0.74,Dist为28.65。我们将提示分析分为三类:强指令提示、基于推理的提示和空间排除过程(SPoE)。这些发现表明,当前VLMs在不可见区域推理能力上仍不稳定,推动未来工作在模型中更紧密地整合任务意图、常识先验、空间定位和不确定性感知搜索。
英文摘要:
In real-world scenes, target objects may reside in regions that are not visible. While humans can often infer the locations of occluded objects from context and commonsense knowledge, this capability remains a major challenge for vision-language models (VLMs). To address this gap, we introduce SceneFunRI, a benchmark for Reasoning the Invisible. Based on the SceneFun3D dataset, SceneFunRI formulates the task as a 2D spatial reasoning problem via a semi-automatic pipeline and comprises 855 instances. It requires models to infer the locations of invisible functional objects from task instructions and commonsense reasoning. The strongest baseline model (Gemini 3 Flash) only achieves an CAcc@75 of 15.20, an mIoU of 0.74, and a Dist of 28.65. We group our prompting analysis into three categories: Strong Instruction Prompting, Reasoning-based Prompting, and Spatial Process of Elimination (SPoE). These findings indicate that invisible-region reasoning remains an unstable capability in current VLMs, motivating future work on models that more tightly integrate task intent, commonsense priors, spatial grounding, and uncertainty-aware search.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。