arXiv 2604.037652026-04-13cs.CV

ITIScore: 一种图像到文本到图像评分框架，用于评估多模态大语言模型的图像描述能力

ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs

Shanghai Jiao Tong University（上海交通大学）；University of Electronic and Science Technology of China（电子科技大学）；Beijing University of Technology（北京工业大学）；Institut Universitaire de France (IUF), University of Nantes（法国大学研究院（IUF），南特大学）

Zitong Xu, Huiyu Duan, Shengyao Qin, Guangyu Yang, Guangji Ma, Xiongkuo Min, Ke Gu, Guangtao Zhai, Patrick Le Callet

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出ICBenchmark图像描述基准，包含12类内容和2K图像生成的40K短长描述，提出ITIScore自动评估指标，通过图像到文本到图像框架衡量描述质量，实验显示其与人类判断一致且具有强泛化能力。

摘要：

近年来，多模态大语言模型（MLLMs）在图像理解和描述能力方面取得了显著进展。然而，现有图像描述基准通常面临描述长度多样性有限、缺乏最新先进MLLMs以及人工标注不足等问题，可能引入偏差并限制对现代MLLMs性能的全面评估。为解决这些问题，我们提出一个新的大规模图像描述基准，称为ICBenchmark，涵盖12类内容，包含由10个先进MLLMs在2000张图像上生成的短长描述，总计40000条描述。我们进行了广泛的主观人类研究以获得跨细粒度评估维度的平均意见分数（MOSs），其中短描述在流畅性、相关性和简洁性方面进行评估，而长描述则基于流畅性、相关性和完整性进行评估。此外，我们提出了一种基于图像到文本到图像框架的自动化评估指标ITIScore，通过重建一致性来衡量描述质量。实验结果表明，我们的自动指标与人类判断高度一致，并且在其他公开描述数据集上具有强大的零样本泛化能力。数据集和模型将在发表时发布。

英文摘要：

Recent advances in multimodal large language models (MLLMs) have greatly improved image understanding and captioning capabilities. However, existing image captioning benchmarks typically suffer from limited diversity in caption length, the absence of recent advanced MLLMs, and insufficient human annotations, which potentially introduces bias and limits the ability to comprehensively assess the performance of modern MLLMs. To address these limitations, we present a new large-scale image captioning benchmark, termed, ICBench, which covers 12 content categories and consists of both short and long captions generated by 10 advanced MLLMs on 2K images, resulting in 40K captions in total. We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions, where short captions are assessed in terms of fluency, relevance, and conciseness, while long captions are evaluated based on fluency, relevance, and completeness. Furthermore, we propose an automated evaluation metric, \textbf{ITIScore}, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency. Experimental results demonstrate strong alignment between our automatic metric and human judgments, as well as robust zero-shot generalization ability on other public captioning datasets. Both the dataset and model will be released upon publication.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。