Gemini 中的推理：揭示多模态大语言模型中的常识
Gemini in Reasoning: Unveiling Commonsense in Multimodal Large Language Models
AI总结：
针对 Gemini 在常识推理上落后于 GPT 的初步评估，本文通过 12 个数据集全面评测其跨模态常识推理能力，发现其表现具有竞争力，并指出了当前模型的共同挑战。
摘要：
对多模态大语言模型（MLLMs）日益增长的兴趣，例如 OpenAI 的 GPT-4V(ision)，已对学术界和工业界产生了显著影响。这些模型通过先进的视觉理解能力增强了大型语言模型（LLMs），促进了它们在各种多模态任务中的应用。最近，谷歌推出了 Gemini，这是一款专为多模态集成而设计的前沿 MLLM。尽管取得了进展，初步基准测试表明，Gemini 在常识推理任务上落后于 GPT 模型。然而，这一基于有限数据集（即 HellaSWAG）的评估并未完全捕捉 Gemini 真实的常识推理潜力。为弥补这一差距，我们的研究对 Gemini 在需要跨模态整合常识知识的复杂推理任务中的表现进行了全面评估。我们对 12 个常识推理数据集进行了综合分析，涵盖从通用到特定领域的任务。这包括 11 个仅关注语言的数据集，以及一个包含多模态元素的数据集。我们在四个 LLM 和两个 MLLM 上的实验表明，Gemini 具有竞争力的常识推理能力。此外，我们识别了当前 LLM 和 MLLM 在处理常识问题时所面临的共同挑战，强调了进一步增强这些模型常识推理能力的必要性。
英文摘要：
The burgeoning interest in Multimodal Large Language Models (MLLMs), such as OpenAI's GPT-4V(ision), has significantly impacted both academic and industrial realms. These models enhance Large Language Models (LLMs) with advanced visual understanding capabilities, facilitating their application in a variety of multimodal tasks. Recently, Google introduced Gemini, a cutting-edge MLLM designed specifically for multimodal integration. Despite its advancements, preliminary benchmarks indicate that Gemini lags behind GPT models in commonsense reasoning tasks. However, this assessment, based on a limited dataset (i.e., HellaSWAG), does not fully capture Gemini's authentic commonsense reasoning potential. To address this gap, our study undertakes a thorough evaluation of Gemini's performance in complex reasoning tasks that necessitate the integration of commonsense knowledge across modalities. We carry out a comprehensive analysis of 12 commonsense reasoning datasets, ranging from general to domain-specific tasks. This includes 11 datasets focused solely on language, as well as one that incorporates multimodal elements. Our experiments across four LLMs and two MLLMs demonstrate Gemini's competitive commonsense reasoning capabilities. Additionally, we identify common challenges faced by current LLMs and MLLMs in addressing commonsense problems, underscoring the need for further advancements in enhancing the commonsense reasoning abilities of these models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
