arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2506.02494cs.CLcs.AIcs.CV
收藏

MINOS:一种用于图像与文本双向生成的多模态评估模型

MINOS: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text

Wangxuan Institute of Computer Technology, Peking University(北京大学王萱计算机技术研究所);School of Physics, Peking University(北京大学物理学院)

Junzhe Zhang, Huixuan Zhang, Xinyu Hu, Li Lin, Mingqi Gao, Shi Qiu, Xiaojun Wan

AI总结:

本文提出MINOS模型,通过严格质量控制策略构建Minos-57K多模态评估数据集,结合SFT和偏好对齐训练策略,在16个跨领域数据集中取得最佳性能。

摘要:

评估对多模态生成任务至关重要,而传统多模态评估指标存在诸多局限。随着MLLMs的快速发展,应用MLLMs构建通用评估系统日益受到关注。然而,现有研究往往仅收集大规模评估数据进行训练,忽视了评估数据的质量。此外,当前提出的评估模型在图像到文本(I2T)和文本到图像(T2I)任务中难以保持一致的强性能。本文通过严格的质量控制策略,构建了一个包含15个数据集的综合多模态评估数据集Minos-57K,以开发多模态评估模型MINOS,采用SFT和偏好对齐训练策略。值得注意的是,尽管使用的训练数据规模不到先前工作的半数,我们的模型在16个跨领域数据集中均取得了最先进的评估性能,涵盖了I2T和T2I任务,并在所有开源多模态评估模型中保持竞争力,与闭源模型相抗衡。大量实验表明,利用质量控制过程、联合训练I2T和T2I生成任务的评估数据以及进一步的偏好对齐的重要性。

英文摘要:

Evaluation is important for multimodal generation tasks, while traditional multimodal evaluation metrics suffer from several limitations. With the rapid progress of MLLMs, there is growing interest in applying MLLMs to build general evaluation systems. However, existing researches often simply collect large-scale evaluation data for training, while overlooking the quality of evaluation data. What's more, current proposed evaluation models often struggle to achieve consistently strong performance across both image-to-text (I2T) and text-to-image (T2I) tasks. In this paper, through rigorous quality control strategies, we construct a comprehensive multimodal evaluation dataset, Minos-57K, with evaluation samples across 15 datasets, for developing the multimodal evaluation model Minos with SFT and preference alignment training strategies. Notably, despite using less than half the scale of the training data of prior work, our model achieves state-of-the-art evaluation performance across 16 out-of-domain datasets covering both I2T and T2I tasks among all open-source multimodal evaluation models and remain competitive with closed-source models. Extensive experiments demonstrate the importance of leveraging quality control process, jointly training on evaluation data from both I2T and T2I generation tasks and further preference alignment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑