探究多模态思维链的推理时扩展：一项初步研究
Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study
AI总结：
本研究首次探索多模态思维链的推理时扩展，在10个任务上比较采样与树搜索方法，发现多模态思维优于纯文本思维，但需更高令牌消耗。
摘要：
最近，思维链（CoT）的推理时扩展已被证明是解决多模态推理任务的一种有前景的方法。尽管现有研究主要集中在基于文本的思维上，但在推理过程中整合视觉和文本两种模态仍未被探索。在本研究中，我们率先探索了具有多模态思维的推理时扩展，旨在弥合这一差距。为了提供全面的分析，我们系统地研究了流行的基于采样和基于树搜索的推理时扩展方法，在跨越多个领域的10个具有挑战性的任务上进行了实验。此外，我们统一采用一致性增强的验证器，以确保在不同思维范式下对两种方法的有效指导。结果表明，多模态思维相比传统的纯文本思维能促进更好的性能，而混合两种类型的思维能激发更多样化的思考。尽管有这些优势，多模态思维需要更高的令牌消耗来处理更丰富的视觉输入，这在实际应用中引起了担忧。我们希望我们关于这一研究方向的优缺点发现能激励该领域的未来工作。
英文摘要：
Recently, inference-time scaling of chain-of-thought (CoT) has been demonstrated as a promising approach for addressing multi-modal reasoning tasks. While existing studies have predominantly centered on text-based thinking, the integration of both visual and textual modalities within the reasoning process remains unexplored. In this study, we pioneer the exploration of inference-time scaling with multi-modal thought, aiming to bridge this gap. To provide a comprehensive analysis, we systematically investigate popular sampling-based and tree search-based inference-time scaling methods on 10 challenging tasks spanning various domains. Besides, we uniformly adopt a consistency-enhanced verifier to ensure effective guidance for both methods across different thought paradigms. Results show that multi-modal thought promotes better performance against conventional text-only thought, and blending the two types of thought fosters more diverse thinking. Despite these advantages, multi-modal thoughts necessitate higher token consumption for processing richer visual inputs, which raises concerns in practical applications. We hope that our findings on the merits and drawbacks of this research line will inspire future works in the field.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
