arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.15098cs.CV
收藏

CLIP-AGIQA:利用CLIP提升AI生成图像质量评估的性能

CLIP-AGIQA: Boosting the Performance of AI-Generated Image Quality Assessment with CLIP

New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences(模式识别新实验室,自动化研究所,中国科学院)

Zhenchen Tang, Zichuan Wang, Bo Peng, Jing Dong

AI总结:

本文提出CLIP-AGIQA,一种基于CLIP的回归模型,利用多类别可学习提示充分挖掘视觉-文本知识,在AGIQA-3K和AIGCIQA2023基准上超越现有IQA模型,显著提升AI生成图像质量评估性能。

摘要:

随着生成技术的快速发展,AI生成图像(AIGIs)已广泛应用于日常生活的各个方面。然而,由于技术尚不成熟,生成图像的质量参差不齐,因此开发针对生成图像的质量评估技术具有重要意义。尽管已有一些模型被提出用于评估生成图像的质量,但面对日益增多且类别多样的生成图像,这些模型仍显不足。因此,迫切需要开发更先进、更有效的模型来评估生成图像的质量。近期研究探索了视觉语言模型CLIP在图像质量评估中的巨大潜力,发现其在评估自然图像质量方面表现良好。然而,其在生成图像上的应用尚未得到深入研究。在本文中,我们基于这一思路,进一步探索CLIP在评估生成图像质量方面的潜力。我们设计了CLIP-AGIQA,一种基于CLIP的回归模型,用于生成图像的质量评估,充分利用了CLIP中蕴含的丰富视觉和文本知识。特别地,我们实现了多类别可学习提示(multi-category learnable prompts),以充分利用CLIP中的文本知识进行质量评估。在多个生成图像质量评估基准(包括AGIQA-3K和AIGCIQA2023)上进行的大量实验表明,CLIP-AGIQA优于现有的IQA模型,在评估生成图像质量方面取得了优异的结果。

英文摘要:

With the rapid development of generative technologies, AI-Generated Images (AIGIs) have been widely applied in various aspects of daily life. However, due to the immaturity of the technology, the quality of the generated images varies, so it is important to develop quality assessment techniques for the generated images. Although some models have been proposed to assess the quality of generated images, they are inadequate when faced with the ever-increasing and diverse categories of generated images. Consequently, the development of more advanced and effective models for evaluating the quality of generated images is urgently needed. Recent research has explored the significant potential of the visual language model CLIP in image quality assessment, finding that it performs well in evaluating the quality of natural images. However, its application to generated images has not been thoroughly investigated. In this paper, we build on this idea and further explore the potential of CLIP in evaluating the quality of generated images. We design CLIP-AGIQA, a CLIP-based regression model for quality assessment of generated images, leveraging rich visual and textual knowledge encapsulated in CLIP. Particularly, we implement multi-category learnable prompts to fully utilize the textual knowledge in CLIP for quality assessment. Extensive experiments on several generated image quality assessment benchmarks, including AGIQA-3K and AIGCIQA2023, demonstrate that CLIP-AGIQA outperforms existing IQA models, achieving excellent results in evaluating the quality of generated images.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑