arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.11089cs.ROcs.AIcs.CV

EmbodiedVSR:面向视觉空间任务的动态场景图引导思维链推理

EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks

Beijing Innovation Center of Humanoid Robotics(北京人形机器人创新中心);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Hong Kong University of Science and Technology(香港科技大学);University of Technology Sydney(悉尼科技大学)
Yi Zhang, Qiang Zhang, Xiaozhu Ju, Zhaoyang Liu, Jilei Mao, Jingkai Sun, Jintao Wu, Shixiong Gao, Shihan Cai, Zhiyuan Qin, Linkai Liang, Jiaxu Wang, Yiqun Duan,… 展开作者

Yi Zhang, Qiang Zhang, Xiaozhu Ju, Zhaoyang Liu, Jilei Mao, Jingkai Sun, Jintao Wu, Shixiong Gao, Shihan Cai, Zhiyuan Qin, Linkai Liang, Jiaxu Wang, Yiqun Duan, Jiahang Cao, Renjing Xu, Jian Tang

AI总结:

本文提出 EmbodiedVSR 框架,利用动态场景图引导 CoT 实现零样本具身空间推理,并构建 eSpatial-Benchmark,显著提升长程视觉空间任务的准确率与推理连贯性。

摘要:

尽管多模态大语言模型(MLLMs)在具身智能领域取得了突破性进展,但它们在复杂长程任务的空间推理方面仍面临重大挑战。为弥补这一差距,我们提出 EmbodiedVSR(Embodied Visual Spatial Reasoning,具身视觉空间推理),这是一种新颖框架,融合动态场景图引导的思维链(Chain-of-Thought,CoT)推理,以增强具身智能体的空间理解能力。通过利用动态场景图显式构建结构化知识表示,我们的方法无需针对特定任务进行微调即可实现零样本空间推理。该方法不仅能够解耦复杂的空间关系,还能使推理步骤与可执行的环境动态保持一致。为严格评估性能,我们引入 eSpatial-Benchmark,这是一个综合性数据集,包含具有细粒度空间标注和自适应任务难度等级的真实世界具身场景。实验表明,我们的框架在准确率和推理连贯性方面显著优于现有基于 MLLM 的方法,尤其是在需要迭代式环境交互的长程任务中。结果揭示,当配备结构化、可解释的推理机制时,MLLMs 在具身智能中尚未开发的潜力,为在真实世界空间应用中更可靠地部署铺平道路。代码和数据集将很快发布。

英文摘要:

While multimodal large language models (MLLMs) have made groundbreaking progress in embodied intelligence, they still face significant challenges in spatial reasoning for complex long-horizon tasks. To address this gap, we propose EmbodiedVSR (Embodied Visual Spatial Reasoning), a novel framework that integrates dynamic scene graph-guided Chain-of-Thought (CoT) reasoning to enhance spatial understanding for embodied agents. By explicitly constructing structured knowledge representations through dynamic scene graphs, our method enables zero-shot spatial reasoning without task-specific fine-tuning. This approach not only disentangles intricate spatial relationships but also aligns reasoning steps with actionable environmental dynamics. To rigorously evaluate performance, we introduce the eSpatial-Benchmark, a comprehensive dataset including real-world embodied scenarios with fine-grained spatial annotations and adaptive task difficulty levels. Experiments demonstrate that our framework significantly outperforms existing MLLM-based methods in accuracy and reasoning coherence, particularly in long-horizon tasks requiring iterative environment interaction. The results reveal the untapped potential of MLLMs for embodied intelligence when equipped with structured, explainable reasoning mechanisms, paving the way for more reliable deployment in real-world spatial applications. The codes and datasets will be released soon.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑