ViThinker: 通过动态感知查询实现主动视觉-语言推理
ViThinker: Active Vision-Language Reasoning via Dynamic Perceptual Querying
AI总结：
ViThinker通过动态感知查询实现主动视觉-语言推理，提升感知基础和推理准确性。
摘要：
链式推理（CoT）在语言模型中表现优异，但在视觉-语言模型中却因过早的视觉到文本转换而遇到困难，这种转换会丢弃诸如几何和空间布局等连续信息。尽管最近的方法通过静态枚举或基于注意力的选择来增强CoT，但它们仍然被动，即处理预计算的输入，而不是主动寻找与任务相关的信息。受人类主动感知的启发，我们引入了ViThinker框架，使视觉-语言模型能够自主生成决策（查询）令牌，以触发专家对齐的视觉特征的合成。ViThinker在训练期间内化了视觉专家的能力，在推理期间进行生成性心理模拟，而无需外部工具调用。通过两阶段课程：首先将冻结的专家知识蒸馏到模型参数中，然后通过稀疏惩罚学习任务驱动的查询，即ViThinker为每个推理步骤发现最小的必要感知。在视觉导向的基准测试中，评估显示了一致的改进，验证了主动查询生成在感知基础和推理准确性方面均优于被动方法。
英文摘要：
Chain-of-Thought (CoT) reasoning excels in language models but struggles in vision-language models due to premature visual-to-text conversion that discards continuous information such as geometry and spatial layout. While recent methods enhance CoT through static enumeration or attention-based selection, they remain passive, i.e., processing pre-computed inputs rather than actively seeking task-relevant details. Inspired by human active perception, we introduce ViThinker, a framework that enables vision-language models to autonomously generate decision (query) tokens triggering the synthesis of expert-aligned visual features on demand. ViThinker internalizes vision-expert capabilities during training, performing generative mental simulation during inference without external tool calls. Through a two-stage curriculum: first distilling frozen experts into model parameters, then learning task-driven querying via sparsity penalties, i.e., ViThinker discovers minimal sufficient perception for each reasoning step. Evaluations across vision-centric benchmarks demonstrate consistent improvements, validating that active query generation outperforms passive approaches in both perceptual grounding and reasoning accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
