arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.02002cs.CV

面向盲图像质量评估的统计与视觉-语言特征的失真感知融合

Parameter-Efficient Adaptation of a Multi-Stream Vision-Language Framework for Blind Image Quality Assessment

Northwestern Polytechnical University(西北工业大学)

Bishr Omer Adam, Xu Li

AI总结:

提出一种失真感知融合框架,通过乘法门控机制动态加权NSS统计特征与VLM嵌入,在三个基准上取得最优或竞争性能,并揭示NSS对不同失真的贡献差异。

摘要:

盲图像质量评估(BIQA)旨在无参考图像的情况下预测感知图像质量。经典的自然场景统计(NSS)描述符和现代视觉语言模型(VLM)嵌入从根本不同的角度解决这一问题,但两者结合是否能产生互补优势以及如何根据输入图像加权其贡献尚待探索。我们提出一种失真感知融合框架,通过乘法门控机制将138维NSS描述符与两种互补的VLM嵌入(SigLIP和CLIP-H)集成,该门控机制学习基于图像内容的每输入流权重。与静态拼接融合不同,所提出的门控网络根据输入抑制或放大每个流的贡献,产生的权重与在KADID-10k上通过独立消融测量的每失真NSS贡献呈正相关(Spearman秩相关系数ρ=0.33)。该框架无需对VLM骨干网络进行端到端微调,并使用结合均方误差、Pearson线性相关和成对排序目标的混合损失进行训练。我们在三个标准基准上评估:KonIQ-10k(SROCC=0.9142,PLCC=0.9279)、KADID-10k(SROCC=0.9715,PLCC=0.9733,超越近期最先进方法)和LIVE Challenge in-the-Wild(通过跨数据集预训练和微调,SROCC=0.8527,PLCC=0.8802)。在KADID-10k上的每失真分析表明,NSS特征对噪声和色彩偏移失真(像素统计直接影响)贡献最大,对感知失真(如色彩饱和度变化)贡献最小。学习到的门控值验证了这些发现,确认模型自主发现了与手动每失真研究一致的失真-流亲和模式。

英文摘要:

Blind image quality assessment (BIQA) predicts perceived image quality without access to a pristine reference and is fundamental to applications such as image compression, transmission, and restoration. Recent BIQA methods increasingly rely on large vision-language models (VLMs). Although frozen VLMs provide an efficient alternative to computationally expensive full fine-tuning, it remains unclear how much performance is sacrificed by not adapting the backbone and, more importantly, under what conditions such adaptation is truly beneficial. Answering this question, however, is complicated by the widespread use of image-level splitting on synthetic-distortion benchmarks, where distorted versions of the same reference image can appear in both training and test partitions. This content overlap artificially inflates the apparent performance of frozen representations, masking their true generalization ability and potentially leading to incorrect conclusions about the value of backbone adaptation. We therefore address these two issues jointly. We develop an efficient BIQA framework that fuses a natural-scene-statistics descriptor with frozen SigLIP and CLIP-H embeddings through a lightweight regression head, and then apply parameter-efficient Low-Rank Adaptation (LoRA) to the SigLIP backbone, training only $0.23\%$ of its parameters. Evaluating both frozen and adapted models across six datasets under image-level and reference-level protocols, we find that image-level splitting inflates frozen-feature SROCC by up to $0.44$ and masks wide variation in true difficulty, which reference-level evaluation reveals. Under this content-independent protocol, LoRA adaptation recovers performance in proportion to the exposed difficulty, with the largest gains where frozen features generalize poorly (up to $+0.357$ SROCC on TID2013) and little benefit where they are already strong.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑