Priorformer:一种结合内容先验与失真先验的UGC-VQA方法
Priorformer: A UGC-VQA Method with content and distortion priors
AI总结:
针对UGC视频复杂降质与内容导致现有盲视频质量评估模型适配性不足的问题,提出结合内容和失真先验的PriorFormer模型,在三个公开数据集上取得SOTA性能。
摘要:
用户生成内容(UGC)视频易受复杂多样的降质和内容影响,现有盲视频质量评估(BVQA)模型因对失真和内容的适配性不足,难以取得良好性能。为缓解这一问题,我们提出一种新型先验增强感知视觉Transformer(PriorFormer),用于UGC的盲视频质量评估,提升其对不同内容和失真的适配性与表征能力。具体而言,我们通过两个预训练特征提取器提取内容嵌入和失真嵌入,引入两种强先验,即内容先验和失真先验。随后我们将这两种强嵌入作为自适应先验令牌,与隐式质量特征一同传输至视觉Transformer骨干网络。基于上述策略,所提PriorFormer在三个公开UGC VQA数据集(包括KoNViD-1K、LIVE-VQC和YouTube-UGC)上取得了当前最优性能。
英文摘要:
User Generated Content (UGC) videos are susceptible to complicated and variant degradations and contents, which prevents the existing blind video quality assessment (BVQA) models from good performance since the lack of the adapability of distortions and contents. To mitigate this, we propose a novel prior-augmented perceptual vision transformer (PriorFormer) for the BVQA of UGC, which boots its adaptability and representation capability for divergent contents and distortions. Concretely, we introduce two powerful priors, i.e., the content and distortion priors, by extracting the content and distortion embeddings from two pre-trained feature extractors. Then we adopt these two powerful embeddings as the adaptive prior tokens, which are transferred to the vision transformer backbone jointly with implicit quality features. Based on the above strategy, the proposed PriorFormer achieves state-of-the-art performance on three public UGC VQA datasets including KoNViD-1K, LIVE-VQC and YouTube-UGC.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。