arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.00356eess.IVcs.CV
收藏

理解视频预测的感知质量

Understanding the Perceived Quality of Video Predictions

Indian Institute of Science(印度科学学院)

Nagabhushan Somraj, Manoj Surya Kashi, S. P. Arun, Rajiv Soundararajan

AI总结:

本研究针对视频预测质量评估难题,构建IISc PVQA数据库,提出基于运动补偿余弦相似度和帧差异深度特征的新方法,实现与人类判断高度一致的最优质量预测。

摘要:

视频预测模型的研究被认为是视频表示学习的一种基本方法。尽管存在大量生成模型,用于根据过去的几帧预测未来帧的像素值,但预测帧的定量评估已被发现极具挑战性。在此背景下,我们研究了预测视频的质量评估问题。我们创建了印度科学研究所预测视频质量评估(IISc PVQA)数据库,该数据库包含300个视频,这些视频是通过在不同数据集上应用不同预测模型获得的,并附有人类意见评分。我们收集了50名人类参与者对这些视频的主观质量评分。我们的主观研究表明,人类观察者在对预测视频质量的判断上高度一致。我们评估了几种常用的视频预测评估度量,并表明它们与这些主观评分的相关性不足。我们引入了两个新特征来有效捕捉预测视频的质量:预测帧与过去帧的深度特征的运动补偿余弦相似度,以及从重缩放帧差异中提取的深度特征。我们表明,我们的特征设计在IISc PVQA数据库上达到了与人类判断一致的最先进的质量预测。该数据库和代码可在我们的项目网站上公开获取:https://nagabhushansn95.github.io/publications/2020/pvqa

英文摘要:

The study of video prediction models is believed to be a fundamental approach to representation learning for videos. While a plethora of generative models for predicting the future frame pixel values given the past few frames exist, the quantitative evaluation of the predicted frames has been found to be extremely challenging. In this context, we study the problem of quality assessment of predicted videos. We create the Indian Institute of Science Predicted Videos Quality Assessment (IISc PVQA) Database consisting of 300 videos, obtained by applying different prediction models on different datasets, and accompanying human opinion scores. We collected subjective ratings of quality from 50 human participants for these videos. Our subjective study reveals that human observers were highly consistent in their judgments of quality of predicted videos. We benchmark several popularly used measures for evaluating video prediction and show that they do not adequately correlate with these subjective scores. We introduce two new features to effectively capture the quality of predicted videos, motion-compensated cosine similarities of deep features of predicted frames with past frames, and deep features extracted from rescaled frame differences. We show that our feature design leads to state of the art quality prediction in accordance with human judgments on our IISc PVQA Database. The database and code are publicly available on our project website: https://nagabhushansn95.github.io/publications/2020/pvqa

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑