arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.14705cs.CV
收藏

学习文本到图像生成的多维人类偏好

Learning Multi-dimensional Human Preference for Text-to-Image Generation

Kuaishou Technology(快手科技)

Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, Zhongyuan Wang

AI总结:

本文提出多维偏好评分(MPS)模型,基于包含91.8万偏好选择的MHP数据集,在CLIP上引入偏好条件模块,从美学、语义对齐、细节质量和整体评估四维度评估文本到图像生成,在3个数据集上优于现有方法。

摘要:

当前的文本到图像模型评估指标通常依赖于统计指标,这些指标不能充分代表人类的真实偏好。尽管最近的工作尝试通过人类标注图像来学习这些偏好,但它们将丰富的人类偏好简化为单一的整体评分。然而,当人类从不同方面评估图像时,偏好结果会有所不同。因此,为了学习多维人类偏好,我们提出了多维偏好评分(MPS),这是首个用于评估文本到图像模型的多维偏好评分模型。MPS在CLIP模型上引入了偏好条件模块,以学习这些多样化的偏好。它基于我们的多维人类偏好(MHP)数据集进行训练,该数据集包含对607,541张图像在四个维度(即美学、语义对齐、细节质量和整体评估)上的918,315个人类偏好选择。这些图像由一系列最新的文本到图像模型生成。MPS在3个数据集的4个维度上优于现有的评分方法,使其成为评估和改进文本到图像生成的一个有前景的指标。

英文摘要:

Current metrics for text-to-image models typically rely on statistical metrics which inadequately represent the real preference of humans. Although recent work attempts to learn these preferences via human annotated images, they reduce the rich tapestry of human preference to a single overall score. However, the preference results vary when humans evaluate images with different aspects. Therefore, to learn the multi-dimensional human preferences, we propose the Multi-dimensional Preference Score (MPS), the first multi-dimensional preference scoring model for the evaluation of text-to-image models. The MPS introduces the preference condition module upon CLIP model to learn these diverse preferences. It is trained based on our Multi-dimensional Human Preference (MHP) Dataset, which comprises 918,315 human preference choices across four dimensions (i.e., aesthetics, semantic alignment, detail quality and overall assessment) on 607,541 images. The images are generated by a wide range of latest text-to-image models. The MPS outperforms existing scoring methods across 3 datasets in 4 dimensions, enabling it a promising metric for evaluating and improving text-to-image generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑