arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.26348cs.AI
收藏

我们在多模态大语言模型评估中缺失了什么?

What We are Missing in Multimodal LLM Evaluation?

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Po-han Li, Shenghui Chen, Sandeep Chinchali, Ufuk Topcu

AI总结:

本文分析现有多模态大语言模型评估基准的不足,识别出时空连贯性、物理世界理解、多模态一致性和选择性注意等关键缺失,强调填补这些空白对衡量多模态智能真实进展的重要性。

摘要:

多模态大语言模型(MLLMs)可以处理多样化的输入,例如文本、图像、音频和视频,并生成文本响应。尽管它们的能力迅速发展,但对此类模型的评估却未能跟上步伐。大多数现有的评估基准局限于孤立的任务,并且很少揭示模型是否跨模态整合信息。我们审视了当前评估MLLMs的方法,并回顾了现有基准分类法,以识别差距,包括时空连贯性、物理世界理解、多模态一致性和选择性注意。填补这些差距对于衡量多模态智能的真实进展和揭示能力边界至关重要。

英文摘要:

Multimodal large language models (MLLMs) can process diverse inputs, e.g., text, images, audio, and video, and generate textual responses. While their capabilities have advanced rapidly, evaluation of such models has not kept pace. Most existing evaluation benchmarks are limited to isolated tasks and reveal little about whether a model integrates information across modalities. We examine current means for evaluating MLLMs and review the existing benchmark taxonomy to identify gaps, including temporal-spatial coherence, physical world understanding, multimodal consistency, and selective attention. Addressing these gaps is essential for measuring real progress in multimodal intelligence and exposing capability boundaries.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑