arXiv 2605.265202026-05-26cs.CVcs.AI

InterSketch: 一种具有自校正视觉草图和逐步奖励的交错推理模型

InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward

Shanghai Jiao Tong University（上海交通大学）；SenseTime Research（商汤科技研究院）；Shandong Normal University（山东师范大学）

Zhiwei Ning, Wenwen Tong, Xiangli Kong, Shengnan Ma, Ziyi Shang, Jingcheng Ni, Tao Hu, Yong Xien Chng, Jixuan Ying, Zehuan Wu, Hanming Deng, Jie Yang, Yuanjie Z… 展开作者

Zhiwei Ning, Wenwen Tong, Xiangli Kong, Shengnan Ma, Ziyi Shang, Jingcheng Ni, Tao Hu, Yong Xien Chng, Jixuan Ying, Zehuan Wu, Hanming Deng, Jie Yang, Yuanjie Zheng, Wei Liu, Lewei Lu

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对视觉-语言模型在长程视觉推理中文本中心范式局限性的问题，提出InterSketch模型，通过自校正和逐步奖励机制增强交错视觉-文本思维链能力，在视觉推理基准上超越Gemini-3-Pro等专有模型。

摘要：

尽管视觉-语言模型（VLM）已展现出多轮视觉推理能力，但其推理轨迹仍相对浅层且以文本为中心，限制了其在复杂视觉挑战中的适用性。相比之下，人类思维通常涉及长程推理，并伴有交错的视觉-文本思维链（VT-CoT）。为弥合这一差距，我们引入InterSketch，一种交错推理模型，通过自校正和逐步奖励机制增强VT-CoT能力。InterSketch使用外部工具动态生成中间视觉草图，并将其与文本推理交错进行，从而在长程视觉理解任务中实现有效的感知和逻辑推理。具体而言，在第一个冷启动阶段，我们提出了一个合成的高质量交错VT-CoT数据集，并引入反思机制，使模型具备多轮交错推理和自校正能力。在后续的强化学习（RL）阶段，我们设计了一种逐步奖励机制，以缓解长程推理中仅端到端监督固有的奖励信号稀疏性问题。在视觉推理基准上的大量实验证明了InterSketch的有效性，其性能甚至超越了Gemini-3-Pro等专有模型。

英文摘要：

While vision-language models (VLMs) have exhibited multi-turn visual reasoning capabilities, their reasoning trajectories remain relatively shallow and are dominated by a text-centric paradigm, limiting their applicability to complex visual challenges. In contrast, human-like thought typically involves long-horizon reasoning with an interleaved visual-textual chain-of-thought (VT-CoT). To bridge this gap, we introduce InterSketch, an interleaved reasoning model to enhance the VT-CoT capability via self-correcting and stepwise reward mechanisms. InterSketch dynamically generates intermediate visual sketches using external tools and interleaves them with textual reasoning, enabling effective perception and logical reasoning over long-horizon visual understanding tasks. Specifically, in the first cold-start stage, we propose a synthesized high-quality interleaved VT-CoT dataset and include a reflection mechanism to enable the model's capability in multi-turn interleaved reasoning and self-correction. In the subsequent reinforcement learning (RL) stage, we design a stepwise reward mechanism to mitigate the sparsity of reward signals inherent in end-only supervision over long-horizon reasoning. Extensive experiments on visual reasoning benchmarks demonstrate the effectiveness of InterSketch, even outperforming proprietary models such as Gemini-3-Pro.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。