探索与评估多模态大语言模型的多模态知识推理一致性
Exploring and Evaluating Multimodal Knowledge Reasoning Consistency of Multimodal Large Language Models
AI总结：
针对多模态大语言模型跨模态知识推理不一致的问题，本文提出四个评估任务并构建新数据集，通过实验分析一致性退化程度及成因，为未来模型改进提供指导。
摘要：
近年来，多模态大语言模型（MLLMs）取得了重大突破，增强了对文本和视觉的理解。然而，当前的MLLMs在多模态知识推理过程中，仍面临有效跨模态整合知识的挑战，导致推理结果出现不一致。为系统地探索这一问题，我们提出了四个评估任务并构建了一个新数据集。我们在该数据集上进行了一系列实验，以分析和比较MLLMs内部多模态知识推理的一致性退化程度。基于实验结果，我们找出了导致一致性退化的因素。本研究为多模态知识推理的挑战提供了新见解，并为未来改进MLLMs提供了有价值的指导。
英文摘要：
In recent years, multimodal large language models (MLLMs) have achieved significant breakthroughs, enhancing understanding across text and vision. However, current MLLMs still face challenges in effectively integrating knowledge across these modalities during multimodal knowledge reasoning, leading to inconsistencies in reasoning outcomes. To systematically explore this issue, we propose four evaluation tasks and construct a new dataset. We conduct a series of experiments on this dataset to analyze and compare the extent of consistency degradation in multimodal knowledge reasoning within MLLMs. Based on the experimental results, we identify factors contributing to the observed degradation in consistency. Our research provides new insights into the challenges of multimodal knowledge reasoning and offers valuable guidance for future efforts aimed at improving MLLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
