拓宽视野并深入思考：协作式跨模态链式推理用于复杂视觉推理
Watch Wider and Think Deeper: Collaborative Cross-modal Chain-of-Thought for Complex Visual Reasoning
AI总结：
CoCoT通过动态多区域定位和关系感知推理，提升复杂视觉推理性能，在多个基准测试中实现显著准确率提升。
摘要：
多模态推理需要无缝整合视觉和语言线索，但现有链式推理方法在跨模态场景中存在两个关键限制：（1）过度依赖单一粗粒度图像区域，（2）推理步骤之间的语义碎片化。为解决这些问题，我们提出了CoCoT（协作跨模态思考）框架，基于两个关键创新：a）动态多区域定位，根据问题自适应检测最相关的图像区域；b）关系感知推理，通过迭代对齐视觉线索，实现多区域协作，形成连贯且逻辑的推理链。通过这种方法，我们构建了包含74,691个高质量样本的CoCoT-70K数据集，这些样本具有多区域标注和结构化的推理链。大量实验表明，CoCoT显著提升了复杂视觉推理能力，在六个具有挑战性的基准测试中，CoCoT在LLaVA-1.5上实现了平均准确率提升15.4%，在Qwen2-VL上提升了4.0%。数据和代码可在：https://github.com/deer-echo/CoCoT获取。
英文摘要：
Multi-modal reasoning requires the seamless integration of visual and linguistic cues, yet existing Chain-of-Thought methods suffer from two critical limitations in cross-modal scenarios: (1) over-reliance on single coarse-grained image regions, and (2) semantic fragmentation between successive reasoning steps. To address these issues, we propose the CoCoT (Collaborative Coross-modal Thought) framework, built upon two key innovations: a) Dynamic Multi-Region Grounding to adaptively detect the most relevant image regions based on the question, and b) Relation-Aware Reasoning to enable multi-region collaboration by iteratively aligning visual cues to form a coherent and logical chain of thought. Through this approach, we construct the CoCoT-70K dataset, comprising 74,691 high-quality samples with multi-region annotations and structured reasoning chains. Extensive experiments demonstrate that CoCoT significantly enhances complex visual reasoning, achieving an average accuracy improvement of 15.4% on LLaVA-1.5 and 4.0% on Qwen2-VL across six challenging benchmarks. The data and code are available at: https://github.com/deer-echo/CoCoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
