让我们高效地用图像思考！一种具有动态和精确视觉思维的交错模态推理框架
Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts
AI总结：
本文提出DaP-ICoT框架，通过动态视觉思维整合和精确视觉思维引导，解决传统ICoT方法中视觉思维位置固定和表示不连贯的问题，提升推理效率和效果。
摘要：
最近，交错模态链式推理（ICoT）通过利用多模态输入和输出取得了显著成功，引起了越来越多的关注。尽管取得了有希望的性能，当前ICoT方法仍然面临两个主要限制：（1）静态视觉思维定位，即在固定步骤静态插入视觉信息，导致推理效率低下且不灵活；（2）断裂的视觉思维表示，涉及不连续且语义不连贯的视觉标记。为了解决这些限制，我们引入了具有动态和精确视觉思维的交错模态推理（DaP-ICoT），该方法结合了两个关键组件：（1）动态视觉思维整合，根据推理需求自适应地引入视觉输入，减少冗余并提高效率；（2）精确视觉思维引导，确保视觉语义连贯且上下文对齐的表示。在多个基准和模型上的实验表明，DaP-ICoT实现了最先进的性能。此外，DaP-ICoT显著减少了插入的图像数量，导致token消耗减少了72.6%，从而实现了更高效的ICoT推理。
英文摘要：
Recently, Interleaved-modal Chain-of-Thought (ICoT) reasoning has achieved remarkable success by leveraging both multimodal inputs and outputs, attracting increasing attention. While achieving promising performance, current ICoT methods still suffer from two major limitations: (1) Static Visual Thought Positioning, which statically inserts visual information at fixed steps, resulting in inefficient and inflexible reasoning; and (2) Broken Visual Thought Representation, which involves discontinuous and semantically incoherent visual tokens. To address these limitations, we introduce Interleaved-modal Chain-of-Thought reasoning with Dynamic and Precise Visual Thoughts (DaP-ICoT), which incorporates two key components: (1) Dynamic Visual Thought Integration adaptively introduces visual inputs based on reasoning needs, reducing redundancy and improving efficiency. (2) Precise Visual Thought Guidance ensures visual semantically coherent and contextually aligned representations. Experiments across multiple benchmarks and models demonstrate that DaP-ICoT achieves state-of-the-art performance. In addition, DaP-ICoT significantly reduces the number of inserted images, leading to a 72.6% decrease in token consumption, enabling more efficient ICoT reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
