M$^3$CoT：多领域多步骤多模态思维链的新型基准
M$^3$CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought
AI总结：
针对现有MCoT基准缺乏视觉推理、单步推理及领域缺失问题，提出多领域多步骤多模态思维链基准M$^3$CoT，并评估VLLMs，发现其与人类表现差距大，为相关研究奠定基础。
摘要：
多模态思维链（MCoT）要求模型利用文本和视觉两种模态的知识进行逐步推理，这日益受到关注。然而，当前的MCoT基准仍面临一些挑战：（1）缺乏视觉模态推理，（2）单步视觉模态推理，以及（3）领域缺失，从而阻碍了MCoT的发展。受此启发，我们引入了一个新型基准（M$^3$CoT）来应对上述挑战，推动多领域、多步骤和多模态CoT的发展。此外，我们对视觉大语言模型（VLLMs）上的大量MCoT方法进行了全面评估。另外，我们强调，尽管现有VLLMs在之前的MCoT基准上表现优异，但在M$^3$CoT上仍难以正确推理，且现有VLLMs与人类在M$^3$CoT上的表现之间仍存在较大差距。据我们所知，我们在MCoT中向多领域、多步骤和多模态场景迈出了有意义的第一步。我们希望M$^3$CoT能作为宝贵资源，为多领域、多步骤、多模态思维链研究提供开创性基础。
英文摘要：
Multi-modal Chain-of-Thought (MCoT) requires models to leverage knowledge from both textual and visual modalities for step-by-step reasoning, which gains increasing attention. Nevertheless, the current MCoT benchmark still faces some challenges: (1) absence of visual modal reasoning, (2) single-step visual modal reasoning, and (3) Domain missing, thereby hindering the development of MCoT. Motivated by this, we introduce a novel benchmark (M$^3$CoT) to address the above challenges, advancing the multi-domain, multi-step, and multi-modal CoT. Additionally, we conduct a thorough evaluation involving abundant MCoT approaches on Vision Large Language Models (VLLMs). In addition, we highlight that the current VLLMs still struggle to correctly reason in M$^3$CoT and there remains a large gap between existing VLLMs and human performance in M$^3$CoT, despite their superior results on previous MCoT benchmarks. To our knowledge, we take the first meaningful step toward the multi-domain, multi-step, and multi-modal scenario in MCoT. We hope that M$^3$CoT can serve as a valuable resource, providing a pioneering foundation in multi-domain, multi-step, multi-modal chain-of-thought research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
