MentalThink:在心理SVG世界中塑造思想
MentalThink: Shaping Thoughts in Mental SVG World
AI总结:
介绍MentalThink视觉符号推理范式,核心是think-with-SVG管道,通过两阶段训练框架实例化,在空间理解和推理基准测试中性能优越,为动态视角获取等提供可视化工作区。
摘要:
我们引入MentalThink,一种视觉符号推理范式,为多模态大语言模型(MLLMs)配备“心理”可视化的可执行机制。MentalThink的核心是一个用SVG思考的管道,模型学习生成、渲染和解释可扩展矢量图形(SVG)代码作为多轮推理的中间视觉表示。通过创建结构化矢量草图,模型可以外化空间假设,通过确定性渲染检查它们,并在受限几何空间内推理,有效地模仿人类心理意象过程。我们通过两阶段训练框架实例化此范式,将用于SVG句法对齐的监督微调(SFT)与多轮强化学习(RL)相结合,以鼓励对中间视觉假设进行迭代检查、修订和完善。广泛评估表明,MentalThink在空间理解和推理基准测试中取得了卓越性能(例如,在VSIBench上为55.1%,在MindCube上为76.0%),表明可执行矢量图形为动态视角获取、视觉反射和构图场景构建提供了可验证的视觉工作区。
英文摘要:
We introduce MentalThink, a visual-symbolic reasoning paradigm that equips Multimodal LLMs (MLLMs) with an executable mechanism for "mental" visualization. The core of MentalThink is a think-with-SVG pipeline, where the model learns to generate, render, and interpret scalable vector graphics (SVG) code as an intermediate visual representation for multi-turn reasoning. By creating structured vector sketches, the model can externalize spatial hypotheses, inspect them through deterministic rendering, and reason within a constrained geometric space, effectively mimicking the human process of mental imagery. We instantiate this paradigm through a two-stage training framework, combining Supervised Fine-Tuning (SFT) for SVG syntactic alignment with multi-turn Reinforcement Learning (RL) to encourage iterative inspection, revision, and refinement of intermediate visual hypotheses. Extensive evaluations demonstrate that MentalThink achieves superior performance on spatial understanding and reasoning benchmarks (e.g., 55.1% on VSIBench, 76.0% on MindCube), showing that executable vector graphics provide a verifiable visual workspace for dynamic perspective taking, visual reflection, and compositional scene construction.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。