arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.03409eess.IVcs.AIcs.CVcs.MM
收藏

VFHQ:视频人脸超分辨率的高质量数据集和基准

VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution

Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);Tencent PCG(腾讯平台与内容事业群)

Liangbin Xie. Xintao Wang, Honglun Zhang, Chao Dong, Ying Shan

摘要:

大多数现有的视频人脸超分辨率(VFSR)方法都是在VoxCeleb1上训练和评估的,VoxCeleb1是专门为说话人识别而设计的,该数据集中的帧质量较低。因此,在此数据集上训练的VFSR模型无法输出视觉上令人愉悦的结果。在本文中,我们开发了一个自动和可扩展的管道来收集高质量的视频人脸数据集(VFHQ),其中包含超过$16,000$高保真剪辑的不同的采访场景。为了验证VFHQ的必要性,我们进一步进行了实验,并证明在我们的VFHQ数据集上训练的VFSR模型可以生成比在VoxCeleb1上训练的模型具有更清晰边缘和更精细纹理的结果。此外,我们表明,时间信息在消除视频一致性问题以及进一步提高视觉性能方面起着关键作用。在VFHQ的基础上,通过对双三次盲环境下几种最新算法的基准测试研究进行分析。查看我们的项目页面:https://liangbinxie.github.io/projects/vfhq

英文摘要:

Most of the existing video face super-resolution (VFSR) methods are trained and evaluated on VoxCeleb1, which is designed specifically for speaker identification and the frames in this dataset are of low quality. As a consequence, the VFSR models trained on this dataset can not output visual-pleasing results. In this paper, we develop an automatic and scalable pipeline to collect a high-quality video face dataset (VFHQ), which contains over $16,000$ high-fidelity clips of diverse interview scenarios. To verify the necessity of VFHQ, we further conduct experiments and demonstrate that VFSR models trained on our VFHQ dataset can generate results with sharper edges and finer textures than those trained on VoxCeleb1. In addition, we show that the temporal information plays a pivotal role in eliminating video consistency issues as well as further improving visual performance. Based on VFHQ, by analyzing the benchmarking study of several state-of-the-art algorithms under bicubic and blind settings. See our project page: https://liangbinxie.github.io/projects/vfhq

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑