VQA$^2$：面向视频质量评估的视觉问答
VQA$^2$: Visual Question Answering for Video Quality Assessment
AI总结：
本文构建首个面向视频质量评估的视觉问答指令数据集 VQA2 Instruction Dataset，并提出交错视觉与运动 token 的 VQA2 系列模型，在质量评分与理解任务上表现优异，VQA2-Assistant 的质量理解能力超过 GPT-4o。
摘要：
大型多模态模型（LMMs）的出现与普及为计算机视觉带来了新的范式，将各类任务转化为统一的视觉问答框架。视频质量评估（VQA）是低层视觉感知中的经典领域，最初关注定量的视频质量评分。然而，在 LMMs 发展的推动下，它如今正朝着更全面的视觉质量理解任务发展。近期在图像领域的研究表明，视觉问答（VQA）能够显著增强低层视觉质量评价。尽管如此，相关工作尚未在视频领域得到探索，仍留有巨大的改进空间。为弥补这一空白，我们提出了 VQA2 Instruction Dataset——首个专注于视频质量评估的视觉问答指令数据集。该数据集由 3 个子集组成，覆盖多种视频类型，包含 157,755 个指令问答对。随后，基于这一基础，我们提出了 VQA2 系列模型。VQA2 系列模型通过交错排列视觉 token 与运动 token，增强对视频中空时质量细节的感知。我们在视频质量评分和理解任务上进行了大量实验，结果表明 VQA2 系列模型在两类任务中均取得优异性能。值得注意的是，我们的最终模型 VQA2-Assistant 在视觉质量理解任务上超过了著名的 GPT-4o，同时在质量评分任务中保持强劲竞争力。本工作为将低层视频质量评估与理解同 LMMs 相结合提供了基础和可行路径。
英文摘要：
The advent and proliferation of large multi-modal models (LMMs) have introduced new paradigms to computer vision, transforming various tasks into a unified visual question answering framework. Video Quality Assessment (VQA), a classic field in low-level visual perception, focused initially on quantitative video quality scoring. However, driven by advances in LMMs, it is now progressing toward more holistic visual quality understanding tasks. Recent studies in the image domain have demonstrated that Visual Question Answering (VQA) can markedly enhance low-level visual quality evaluation. Nevertheless, related work has not been explored in the video domain, leaving substantial room for improvement. To address this gap, we introduce the VQA2 Instruction Dataset - the first visual question answering instruction dataset that focuses on video quality assessment. This dataset consists of 3 subsets and covers various video types, containing 157,755 instruction question-answer pairs. Then, leveraging this foundation, we present the VQA2 series models. The VQA2 series models interleave visual and motion tokens to enhance the perception of spatial-temporal quality details in videos. We conduct extensive experiments on video quality scoring and understanding tasks, and results demonstrate that the VQA2series models achieve excellent performance in both tasks. Notably, our final model, the VQA2-Assistant, exceeds the renowned GPT-4o in visual quality understanding tasks while maintaining strong competitiveness in quality scoring tasks. Our work provides a foundation and feasible approach for integrating low-level video quality assessment and understanding with LMMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
