arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.00212eess.IVcs.CV
收藏

MVAD:一种用于视频流的多重视觉伪影检测器

MVAD: A Multiple Visual Artifact Detector for Video Streaming

Visual Information Lab, University of Bristol(视觉信息实验室,布里斯托大学);Amazon Prime Video(亚马逊Prime视频)

Chen Feng, Duolikun Danier, Fan Zhang, Alex Mackin, Andrew Collins, David Bull

AI总结:

本文提出MVAD多重视觉伪影检测器,首次在单一框架中结合ADFE特征提取器和RMViT时间建模模块,不依赖质量评估模型即可同时检测视频流中的多种伪影,并在两个数据库上验证了优于现有方法的性能。

摘要:

由于内容制作和传输过程中的普遍条件,流媒体视频内容中经常会引入视觉伪影。这些伪影会降低用户体验质量,因此自动且准确地检测它们对于实现有效的质量测量和增强至关重要。现有的检测方法通常只关注单一类型的伪影,并且/或者通过对客观质量指标进行阈值化来判断伪影是否存在。据报道,这类方法在预测性能上表现不一致,而且在多种伪影共存并相互作用的实际应用中也缺乏实用性。在本文中,我们提出了一种用于视频流的多重视觉伪影检测器MVAD,它首次能够在单一框架中检测多种伪影,并且不依赖于视频质量评估模型。我们的方法采用了一种新的伪影感知动态特征提取器(ADFE),用于在每个帧内获取与多种伪影类型相关的空间特征。提取到的特征进一步由循环记忆视觉Transformer(RMViT)模块处理,该模块能够捕获输入视频中的短期和长期时间信息。所提出的网络架构基于一个新的、大规模且多样化的训练数据库进行端到端优化,该数据库通过模拟视频流管道并基于对抗性数据增强生成。该模型已在两个视频伪影数据库Maxwell和BVI-Artifact上进行了评估,与七种现有的单一和多伪影检测器相比,在十种目标视觉伪影上取得了一致且改进的预测结果。源代码和训练数据库将在https://chenfeng-bristol.github.io/MVAD/上公开。

英文摘要:

Visual artifacts are often introduced into streamed video content, due to prevailing conditions during content production and delivery. Since these can degrade the quality of the user's experience, it is important to automatically and accurately detect them in order to enable effective quality measurement and enhancement. Existing detection methods often focus on a single type of artifact and/or determine the presence of an artifact through thresholding objective quality indices. Such approaches have been reported to offer inconsistent prediction performance and are also impractical for real-world applications where multiple artifacts co-exist and interact. In this paper, we propose a Multiple Visual Artifact Detector, MVAD, for video streaming which, for the first time, is able to detect multiple artifacts using a single framework that is not reliant on video quality assessment models. Our approach employs a new Artifact-aware Dynamic Feature Extractor (ADFE) to obtain artifact-relevant spatial features within each frame for multiple artifact types. The extracted features are further processed by a Recurrent Memory Vision Transformer (RMViT) module, which captures both short-term and long-term temporal information within the input video. The proposed network architecture is optimized in an end-to-end manner based on a new, large and diverse training database that is generated by simulating the video streaming pipeline and based on Adversarial Data Augmentation. This model has been evaluated on two video artifact databases, Maxwell and BVI-Artifact, and achieves consistent and improved prediction results for ten target visual artifacts when compared to seven existing single and multiple artifact detectors. The source code and training database will be available at https://chenfeng-bristol.github.io/MVAD/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑