arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2211.14005cs.CV
收藏

高分辨率视频帧内插的高效特征提取

Efficient Feature Extraction for High-resolution Video Frame Interpolation

TU Darmstadt(达姆施塔特工业大学)

Moritz Nottebaum, Stefan Roth, Simone Schaub-Meyer

摘要:

大多数用于视频帧插值的深度学习方法包括三个主要部分:特征提取、运动估计和图像合成。现有的方法主要是在如何设计这些模块方面的区别。然而,当内插高分辨率图像时,例如在4K下,在合理的存储器要求内实现高精度的设计选择是有限的。特征提取层有助于压缩输入并为后面的阶段提取相关信息,例如运动估计。然而,这些层在参数、计算时间和存储器方面通常是昂贵的。我们展示了如何结合轻量级优化的降维思想可以用来压缩输入表示,同时保持提取的信息适合帧内插。此外,我们既不需要预先训练的流网络,也不需要合成网络,另外减少了可训练参数的数量和所需的内存。在三个4K基准测试中进行评估时,我们在没有预训练流的情况下实现了最先进的图像质量,同时具有最低的网络复杂度和整体内存需求。

英文摘要:

Most deep learning methods for video frame interpolation consist of three main components: feature extraction, motion estimation, and image synthesis. Existing approaches are mainly distinguishable in terms of how these modules are designed. However, when interpolating high-resolution images, e.g. at 4K, the design choices for achieving high accuracy within reasonable memory requirements are limited. The feature extraction layers help to compress the input and extract relevant information for the latter stages, such as motion estimation. However, these layers are often costly in parameters, computation time, and memory. We show how ideas from dimensionality reduction combined with a lightweight optimization can be used to compress the input representation while keeping the extracted information suitable for frame interpolation. Further, we require neither a pretrained flow network nor a synthesis network, additionally reducing the number of trainable parameters and required memory. When evaluating on three 4K benchmarks, we achieve state-of-the-art image quality among the methods without pretrained flow while having the lowest network complexity and memory requirements overall.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑