arXiv 2412.000602025-11-14cs.CVcs.AI

MOSABench：用于评估多模态大语言模型对复杂图像理解能力的多目标情感分析基准

MOSABench: Multi-Object Sentiment Analysis Benchmark for Evaluating Multimodal Large Language Models Understanding of Complex Image

College of Computer, National University of Defense Technology（国防科技大学计算机学院）；School of Computer and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；CCNU（中国地质大学）

Shezheng Song, Chengxiang He, Shan Zhao, Chengyu Wang, Qian Wan, Tianwei Yan, Meng Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有多模态大语言模型缺乏多目标情感分析标准化评估基准的问题，构建含约千张多目标图像的MOSABench基准，实验揭示当前模型的相关性能局限，为该领域研究提供基础评测工具。

摘要：

多模态大语言模型（MLLMs）在视觉问答、图像字幕生成、情感识别等高层语义任务上已取得显著进展。尽管相关技术不断发展，目前仍缺乏标准化基准来评估MLLMs在多目标情感分析这一语义理解核心任务上的性能。为填补这一空白，我们推出了专门面向多目标情感分析设计的新型评估数据集MOSABench。该基准包含约1000张含多个目标的图像，要求MLLMs独立评估每个目标的情感，以此反映真实场景的复杂性。MOSABench的核心创新包括基于距离的目标标注、用于标准化输出的评估后处理流程，以及改进的评分机制。我们的实验揭示了当前MLLMs存在的明显局限：尽管mPLUG-owl、Qwen-VL2等部分模型能有效关注情感相关特征，但其余模型注意力分散、性能出现下降，尤其当目标间空间距离增大时表现下滑更明显。本研究强调MLLMs需提升复杂多目标情感分析任务的准确率，同时MOSABench可作为推进MLLMs情感分析能力发展的基础工具。

英文摘要：

Multimodal large language models (MLLMs) have shown remarkable progress in high-level semantic tasks such as visual question answering, image captioning, and emotion recognition. However, despite advancements, there remains a lack of standardized benchmarks for evaluating MLLMs performance in multi-object sentiment analysis, a key task in semantic understanding. To address this gap, we introduce MOSABench, a novel evaluation dataset designed specifically for multi-object sentiment analysis. MOSABench includes approximately 1,000 images with multiple objects, requiring MLLMs to independently assess the sentiment of each object, thereby reflecting real-world complexities. Key innovations in MOSABench include distance-based target annotation, post-processing for evaluation to standardize outputs, and an improved scoring mechanism. Our experiments reveal notable limitations in current MLLMs: while some models, like mPLUG-owl and Qwen-VL2, demonstrate effective attention to sentiment-relevant features, others exhibit scattered focus and performance declines, especially as the spatial distance between objects increases. This research underscores the need for MLLMs to enhance accuracy in complex, multi-object sentiment analysis tasks and establishes MOSABench as a foundational tool for advancing sentiment analysis capabilities in MLLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。