arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.16562cs.CVcs.CL
收藏

EVALALIGN:使用人类对齐数据监督微调多模态大语言模型以评估文本生成图像模型

EVALALIGN: Supervised Fine-Tuning Multimodal LLMs with Human-Aligned Data for Evaluating Text-to-Image Models

Shanghai Academy of AI for Science(上海人工智能科学研究院);Carnegie Mellon University(卡内基梅隆大学);Fudan University(复旦大学)

Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, Mengping Yang, Cheng Zhang, Hao Li

AI总结:

针对文本生成图像模型缺乏精准细粒度评估指标的问题,提出基于人类对齐数据监督微调MLLM的EvalAlign指标,在24个模型上验证其稳定性与人类偏好对齐度均优于现有指标。

摘要:

文本生成图像模型近年来进展显著,但该领域缺乏能准确反映模型性能的评估指标,尤其缺少可指导模型优化的细粒度指标。本文提出EvalAlign,一种兼具准确性、稳定性与细粒度特性的评估指标。我们的方法利用在海量数据上预训练的多模态大语言模型(Multimodal Large Language Models, MLLMs)的能力,围绕图像忠实度与文本-图像对齐两个核心维度构建评估协议。每个协议包含一组关联特定评分选项的详细细粒度指令,可对生成图像进行精准人工评分。我们对MLLM进行监督微调(Supervised Fine-Tune, SFT),使其与人类评估判断对齐,得到鲁棒的评估模型。我们在24个文本生成图像模型上的评估表明,EvalAlign不仅指标稳定性更优,且相比现有指标更贴合人类偏好,证实了其在模型评估中的有效性与实用性。

英文摘要:

The recent advancements in text-to-image generative models have been remarkable. Yet, the field suffers from a lack of evaluation metrics that accurately reflect the performance of these models, particularly lacking fine-grained metrics that can guide the optimization of the models. In this paper, we propose EvalAlign, a metric characterized by its accuracy, stability, and fine granularity. Our approach leverages the capabilities of Multimodal Large Language Models (MLLMs) pre-trained on extensive data. We develop evaluation protocols that focus on two key dimensions: image faithfulness and text-image alignment. Each protocol comprises a set of detailed, fine-grained instructions linked to specific scoring options, enabling precise manual scoring of the generated images. We supervised fine-tune (SFT) the MLLM to align with human evaluative judgments, resulting in a robust evaluation model. Our evaluation across 24 text-to-image generation models demonstrate that EvalAlign not only provides superior metric stability but also aligns more closely with human preferences than existing metrics, confirming its effectiveness and utility in model assessment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑