arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.09134cs.CV
收藏

基于运动理解的单幅模糊图像视频帧恢复

Restoration of Video Frames from a Single Blurred Image with Motion Understanding

KAIST(韩国科学技术院)

Dawit Mureja Argaw, Junsik Kim, Francois Rameau, Chaoning Zhang, In So Kweon

摘要:

我们提出了一个新的框架,从一个单一的运动模糊图像生成干净的视频帧。虽然广泛的文献集中在从模糊图像中恢复单个图像,但在这项工作中,我们解决了一个更具挑战性的任务,即从模糊图像中恢复视频。我们制定从一个单一的模糊图像的视频恢复作为一个逆问题,通过设置清洁的图像序列和它们各自的运动作为潜在的因素,和模糊的图像作为一个观察。我们的框架是基于空间Transformer网络模块的编码器-解码器结构,以端到端的方式恢复视频序列及其底层运动。我们设计了一个损失函数和正则化与互补性质,以稳定的训练和分析所提出的网络的变体模型。通过对两种不同类型的数据集进行大量实验,突出了我们网络的有效性和可移植性:全景场景中生成的相机旋转模糊和高速视频中的动态运动模糊。

英文摘要:

We propose a novel framework to generate clean video frames from a single motion-blurred image. While a broad range of literature focuses on recovering a single image from a blurred image, in this work, we tackle a more challenging task i.e. video restoration from a blurred image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors, and the blurred image as an observation. Our framework is based on an encoder-decoder structure with spatial transformer network modules to restore a video sequence and its underlying motion in an end-to-end manner. We design a loss function and regularizers with complementary properties to stabilize the training and analyze variant models of the proposed network. The effectiveness and transferability of our network are highlighted through a large set of experiments on two different types of datasets: camera rotation blurs generated from panorama scenes and dynamic motion blurs in high speed videos.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑