Canvas-of-Thought:通过可变的结构化状态进行推理
Canvas-of-Thought: Grounding Reasoning via Mutable Structured States
AI总结:
Canvas-CoT通过引入HTML Canvas实现可变结构化状态,提升多模态推理效率与精度。
摘要:
尽管链式思维(CoT)提示方法显著提升了多模态大语言模型(MLLMs)的推理能力,但仅依赖线性文本序列在复杂任务中仍是一个瓶颈。我们发现,即使辅助视觉元素被交错插入,它们往往仍被视为一维、无结构的推理链中的静态快照。我们认为,此类方法将推理历史视为不可变的流:纠正局部错误需要要么生成冗长的下游修正,要么重新生成整个上下文。这迫使模型隐式地维护和跟踪状态更新,显著增加了token消耗和认知负担。这一限制在高维领域,如几何和SVG设计中尤为明显,因为CoT的文本表达缺乏显式的视觉指导,进一步限制了模型的推理精度。为弥合这一差距,我们引入了\textbf{Canvas-of-Thought(Canvas-CoT)}。通过利用HTML Canvas作为外部推理载体,Canvas-CoT使模型能够执行原子的、基于DOM的CRUD操作。该架构使模型能够不破坏周围上下文的情况下进行原地状态修订,允许模型显式地维护“真实值”。此外,我们集成了基于渲染的批评循环,作为硬约束验证器,提供明确的视觉反馈,以解决难以仅通过文本描述完成的复杂任务。在VCode、RBench-V和MathVista上的广泛实验表明,Canvas-CoT显著优于现有基线,建立了上下文高效多模态推理的新范式。
英文摘要:
While Chain-of-Thought (CoT) prompting has significantly advanced the reasoning capabilities of Multimodal Large Language Models (MLLMs), relying solely on linear text sequences remains a bottleneck for complex tasks. We observe that even when auxiliary visual elements are interleaved, they are often treated as static snapshots within a one-dimensional, unstructured reasoning chain. We argue that such approaches treat reasoning history as an immutable stream: correcting a local error necessitates either generating verbose downstream corrections or regenerating the entire context. This forces the model to implicitly maintain and track state updates, significantly increasing token consumption and cognitive load. This limitation is particularly acute in high-dimensional domains, such as geometry and SVG design, where the textual expression of CoT lacks explicit visual guidance, further constraining the model's reasoning precision. To bridge this gap, we introduce \textbf{Canvas-of-Thought (Canvas-CoT)}. By leveraging a HTML Canvas as an external reasoning substrate, Canvas-CoT empowers the model to perform atomic, DOM-based CRUD operations. This architecture enables in-place state revisions without disrupting the surrounding context, allowing the model to explicitly maintain the "ground truth". Furthermore, we integrate a rendering-based critique loop that serves as a hard constraint validator, providing explicit visual feedback to resolve complex tasks that are difficult to articulate through text alone. Extensive experiments on VCode, RBench-V, and MathVista demonstrate that Canvas-CoT significantly outperforms existing baselines, establishing a new paradigm for context-efficient multimodal reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。