arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.16579cs.AIcs.CV

弥合动态感知鸿沟:面向动态多模态空间推理的无训练草稿链式思维

Bridging the Dynamic Perception Gap: Training-Free Draft Chain-of-Thought for Dynamic Multimodal Spatial Reasoning

Shanghai Jiao Tong University(上海交通大学);Shanghai AI lab(上海人工智能实验室)

Siqu Ou, Hongcheng Liu, Pingjie Wang, Yusheng Liao, Chuan Xuan, Yanfeng Wang, Yu Wang

AI总结:

针对动态空间推理中现有CoT方法的不足,提出无训练框架D2R,通过叠加动态视觉草稿增强文本推理链,在GRASSLAND基准上显著提升性能,无需微调。

摘要:

虽然链式思维(CoT)已推进了多模态大语言模型(MLLMs)中的复杂推理,但现有方法仍局限于文本或静态视觉领域,在动态空间推理任务中往往表现不佳。为弥合这一鸿沟,我们提出了GRASSLAND,一个新颖的迷宫导航基准,旨在评估动态空间推理。我们的实验表明,在输入图像上叠加动态视觉草稿来增强文本推理链,显著优于传统方法,为演化环境中的空间推理提供了新见解。为推广这一能力,我们提出了D2R(动态草稿增强推理),一个无训练框架,将文本CoT与相应的视觉草稿无缝集成到MLLMs中。广泛评估表明,D2R在多种任务中持续提升性能,为动态空间推理建立了稳健基线,且无需模型微调。项目开源地址:https://github.com/Cratileo/D2R。

英文摘要:

While chains-of-thought (CoT) have advanced complex reasoning in multimodal large language models (MLLMs), existing methods remain confined to text or static visual domains, often faltering in dynamic spatial reasoning tasks. To bridge this gap, we present GRASSLAND, a novel maze navigation benchmark designed to evaluate dynamic spatial reasoning. Our experiments show that augmenting textual reasoning chains with dynamic visual drafts, overlaid on input images, significantly outperforms conventional approaches, offering new insights into spatial reasoning in evolving environments. To generalize this capability, we propose D2R (Dynamic Draft-Augmented Reasoning), a training-free framework that seamlessly integrates textual CoT with corresponding visual drafts into MLLMs. Extensive evaluations demonstrate that D2R consistently enhances performance across diverse tasks, establishing a robust baseline for dynamic spatial reasoning without requiring model fine-tuning. Project is open at https://github.com/Cratileo/D2R.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑