arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.07555cs.CVcs.AI
收藏

VISOR:基于语言驱动的对象导航的视觉空间对象推理

VISOR: VIsual Spatial Object Reasoning for Language-driven Object Navigation

Polytechnic of Turin(都灵理工大学);Simon Fraser University(Simon Fraser大学);University of Verona(威尼斯大学);University of Reykjavik(雷克雅未克大学);University of California, Irvine(加州大学尔湾分校)

Francesco Taioli, Shiping Yang, Sonia Raychaudhuri, Marco Cristani, Unnat Jain, Angel X Chang

AI总结:

VISOR提出了一种紧凑的3B参数VLA智能体,通过显式图像基础推理实现人类般的具身推理,提升对象识别和动作选择的可解释性、泛化能力和导航效率。

摘要:

语言驱动的对象导航需要智能体解释目标对象的自然语言描述,这些描述结合了内在和外在属性用于实例识别和常识导航。现有方法要么(i)使用端到端训练的模型与视觉-语言嵌入,这些模型难以在训练数据之外泛化且缺乏动作层面的可解释性,要么(ii)依赖模块化零样本流程与大型语言模型(LLMs)和开集对象检测器,这些方法存在误差传播、计算成本高以及难以将推理整合回导航策略的问题。为此,我们提出了一种紧凑的3B参数视觉-语言-动作(VLA)智能体,该智能体在对象识别和动作选择上实现了类似人类的具身推理,消除了对拼接多模型流程的需要。不同于原始嵌入匹配,我们的智能体采用显式图像基础推理,直接回答“这是否是目标对象?”和“为何应采取此动作?”推理过程分为三个阶段:“思考”、“思考摘要”和“动作”,从而提高了可解释性、更强的泛化能力和更高效的导航。代码和数据集将在接受后提供。

英文摘要:

Language-driven object navigation requires agents to interpret natural language descriptions of target objects, which combine intrinsic and extrinsic attributes for instance recognition and commonsense navigation. Existing methods either (i) use end-to-end trained models with vision-language embeddings, which struggle to generalize beyond training data and lack action-level explainability, or (ii) rely on modular zero-shot pipelines with large language models (LLMs) and open-set object detectors, which suffer from error propagation, high computational cost, and difficulty integrating their reasoning back into the navigation policy. To this end, we propose a compact 3B-parameter Vision-Language-Action (VLA) agent that performs human-like embodied reasoning for both object recognition and action selection, removing the need for stitched multi-model pipelines. Instead of raw embedding matching, our agent employs explicit image-grounded reasoning to directly answer "Is this the target object?" and "Why should I take this action?" The reasoning process unfolds in three stages: "think", "think summary", and "action", yielding improved explainability, stronger generalization, and more efficient navigation. Code and dataset available upon acceptance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑