arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2405.03333cs.CV
收藏

Light-VQA+:一种带视觉-语言引导的面向曝光校正的视频质量评估模型

Light-VQA+: A Video Quality Assessment Model for Exposure Correction with Vision-Language Guidance

Shanghai Jiao Tong University(上海交通大学);Nanyang Technological University(南洋理工大学)

Xunchu Zhou, Xiaohong Liu, Yunlong Dong, Tengchuan Kou, Yixuan Gao, Zicheng Zhang, Chunyi Li, Haoning Wu, Guangtao Zhai

AI总结:

本研究针对现有通用VQA模型不适配曝光校正效果评估的问题,扩展构建VEC-QA数据集,提出带视觉-语言引导与HVS模块的Light-VQA+模型,在多个数据集上性能超越现有SOTA VQA模型。

摘要:

近年来,用户生成内容(UGC)视频在日常生活中日益普及。但受拍摄设备与技术限制,UGC视频常存在曝光不佳的问题。为此研究者提出了视频曝光校正(VEC)算法,涵盖低光视频增强(LLVE)与过曝视频恢复(OEVR)两类。与VEC同等重要的是视频质量评估(VQA),但现有几乎所有VQA模型均为通用型,从综合维度衡量视频质量。针对这一问题,此前有研究在LLVE-QA数据集上训练了Light-VQA模型用于评估低光视频增强效果。本研究拓展了Light-VQA的工作:将LLVE-QA数据集扩展为视频曝光校正质量评估(VEC-QA)数据集,新增了过曝视频及其对应校正版本;同时提出专门用于评估VEC效果的VQA模型Light-VQA+。Light-VQA+与Light-VQA的核心差异在于,其在特征提取阶段引入CLIP模型与视觉-语言引导机制,后续还新增了参考人类视觉系统(HVS)的模块以提升评估准确性。大量实验结果表明,在VEC-QA数据集及其他公开数据集上,本模型的性能均优于当前最先进(SOTA)的VQA模型。

英文摘要:

Recently, User-Generated Content (UGC) videos have gained popularity in our daily lives. However, UGC videos often suffer from poor exposure due to the limitations of photographic equipment and techniques. Therefore, Video Exposure Correction (VEC) algorithms have been proposed, Low-Light Video Enhancement (LLVE) and Over-Exposed Video Recovery (OEVR) included. Equally important to the VEC is the Video Quality Assessment (VQA). Unfortunately, almost all existing VQA models are built generally, measuring the quality of a video from a comprehensive perspective. As a result, Light-VQA, trained on LLVE-QA, is proposed for assessing LLVE. We extend the work of Light-VQA by expanding the LLVE-QA dataset into Video Exposure Correction Quality Assessment (VEC-QA) dataset with over-exposed videos and their corresponding corrected versions. In addition, we propose Light-VQA+, a VQA model specialized in assessing VEC. Light-VQA+ differs from Light-VQA mainly from the usage of the CLIP model and the vision-language guidance during the feature extraction, followed by a new module referring to the Human Visual System (HVS) for more accurate assessment. Extensive experimental results show that our model achieves the best performance against the current State-Of-The-Art (SOTA) VQA models on the VEC-QA dataset and other public datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑