Q-CLIP：通过统一跨模态适配释放视觉-语言模型在视频质量评估中的力量
Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation
AI总结：
提出Q-CLIP，首个完全基于视觉-语言模型的视频质量评估框架，通过共享跨模态适配器和可学习质量提示，以极低训练成本实现多数据集上的优异性能。
摘要：
准确且高效的视频质量评估（VQA）一直是一个关键的研究挑战。当前主流的VQA方法通常通过在大规模分类数据集（如ImageNet、Kinetics-400）上进行预训练，然后在VQA数据集上进行微调来提升性能。然而，这种策略带来了两个重大挑战：（1）仅迁移预训练中学到的语义知识不足以应对VQA，因为视频质量取决于多种因素（如语义、失真、运动、美学）；（2）在大规模数据集上进行预训练需要巨大的计算资源，其成本通常是直接在VQA数据集上训练的数十倍甚至数百倍。近年来，视觉-语言模型（VLMs）在广泛的视觉任务中展现出了显著的泛化能力，并开始在质量评估领域显示出有前景的潜力。在这项工作中，我们提出了Q-CLIP，这是首个完全基于VLMs的VQA框架。Q-CLIP通过共享跨模态适配器（SCMA）增强视觉和文本表示，该适配器仅包含极少数可训练参数，并且是唯一需要训练的部分。这种设计显著降低了计算成本。此外，我们引入了一组五个可学习的质量级别提示，以引导VLMs感知细微的质量变化，从而进一步增强模型对视频质量的敏感性。进一步地，我们研究了不同帧采样策略对VQA性能的影响，并发现基于帧差的采样策略在数据集间能带来更好的泛化性能。大量实验表明，Q-CLIP在多个VQA数据集上表现出优异的性能。
英文摘要：
Accurate and efficient Video Quality Assessment (VQA) has long been a key research challenge. Current mainstream VQA methods typically improve performance by pretraining on large-scale classification datasets (e.g., ImageNet, Kinetics-400), followed by fine-tuning on VQA datasets. However, this strategy presents two significant challenges: (1) merely transferring semantic knowledge learned from pretraining is insufficient for VQA, as video quality depends on multiple factors (e.g., semantics, distortion, motion, aesthetics); (2) pretraining on large-scale datasets demands enormous computational resources, often dozens or even hundreds of times greater than training directly on VQA datasets. Recently, Vision-Language Models (VLMs) have shown remarkable generalization capabilities across a wide range of visual tasks, and have begun to demonstrate promising potential in quality assessment. In this work, we propose Q-CLIP, the first fully VLMs-based framework for VQA. Q-CLIP enhances both visual and textual representations through a Shared Cross-Modal Adapter (SCMA), which contains only a minimal number of trainable parameters and is the only component that requires training. This design significantly reduces computational cost. In addition, we introduce a set of five learnable quality-level prompts to guide the VLMs in perceiving subtle quality variations, thereby further enhancing the model's sensitivity to video quality. Furthermore, we investigate the impact of different frame sampling strategies on VQA performance, and find that frame-difference-based sampling leads to better generalization performance across datasets. Extensive experiments demonstrate that Q-CLIP exhibits excellent performance on several VQA datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
