arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.05381cs.CV
收藏

基于视觉-语言模型适配的小样本图像质量评估

Few-Shot Image Quality Assessment via Adaptation of Vision-Language Models

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(中国教育部多媒体可信感知与高效计算重点实验室,厦门大学);Beijing Institute of Technology(北京理工大学);Tencent Youtu Lab, Shanghai, China(腾讯优图实验室,中国)

Xudong Li, Zihao Huang, Yan Zhang, Yunhang Shen, Ke Li, Xiawu Zheng, Liujuan Cao, Rongrong Ji

AI总结:

本文提出GRMP-IQA框架,通过元提示预训练和质量感知梯度正则化高效适配CLIP,在仅用20%训练数据的小样本条件下实现与全监督方法相当的图像质量评估性能。

摘要:

图像质量评估(IQA)由于复杂失真、多样化的图像内容以及有限的数据可用性,仍然是计算机视觉中一个尚未解决的挑战。现有的盲图像质量评估(BIQA)方法在很大程度上依赖大量人工标注,而由于创建IQA数据集的苛刻性质,这些标注劳动密集且成本高昂。为了减少这种依赖,我们提出了梯度调控元提示IQA框架(GRMP-IQA),旨在高效地将视觉-语言预训练模型CLIP适配到IQA任务中,即使在数据有限的情况下也能实现高精度。GRMP-IQA由两个核心模块组成:(i)元提示预训练模块和(ii)质量感知梯度正则化。元提示预训练模块利用元学习范式来预训练软提示,使其在不同失真之间共享元知识,从而能够快速适配各种IQA任务。另一方面,质量感知梯度正则化旨在微调过程中调整更新梯度,使模型的注意力集中在与质量相关的特征上,并防止过拟合语义信息。在标准BIQA数据集上的大量实验表明,在有限数据设置下,该方法优于最先进的BIQA方法。值得注意的是,仅使用20%的训练数据,GRMP-IQA就能与大多数现有的全监督BIQA方法相媲美。

英文摘要:

Image Quality Assessment (IQA) remains an unresolved challenge in computer vision due to complex distortions, diverse image content, and limited data availability. Existing Blind IQA (BIQA) methods largely rely on extensive human annotations, which are labor-intensive and costly due to the demanding nature of creating IQA datasets. To reduce this dependency, we propose the Gradient-Regulated Meta-Prompt IQA Framework (GRMP-IQA), designed to efficiently adapt the visual-language pre-trained model, CLIP, to IQA tasks, achieving high accuracy even with limited data. GRMP-IQA consists of two core modules: (i) Meta-Prompt Pre-training Module and (ii) Quality-Aware Gradient Regularization. The Meta Prompt Pre-training Module leverages a meta-learning paradigm to pre-train soft prompts with shared meta-knowledge across different distortions, enabling rapid adaptation to various IQA tasks. On the other hand, the Quality-Aware Gradient Regularization is designed to adjust the update gradients during fine-tuning, focusing the model's attention on quality-relevant features and preventing overfitting to semantic information. Extensive experiments on standard BIQA datasets demonstrate the superior performance to the state-of-the-art BIQA methods under limited data setting. Notably, utilizing just 20% of the training data, GRMP-IQA is competitive with most existing fully supervised BIQA approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑