arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2201.11632cs.CVcs.AI
收藏

用于视频一致性和传播的深度视频先验

Deep Video Prior for Video Consistency and Propagation

The Hong Kong University of Science and Technology(香港科技大学)

Chenyang Lei, Yazhou Xing, Hao Ouyang, Qifeng Chen

摘要:

将图像处理算法独立地应用于每个视频帧通常导致所得视频中的时间不一致性。为了解决这个问题,我们提出了一种新的和一般的方法盲视频时间一致性。我们的方法只直接在一对原始和处理过的视频上训练,而不是在一个大的数据集上训练。与大多数以前的方法不同,我们证明了时间一致性可以通过在具有深度视频先验(DVP)的视频上训练卷积神经网络来实现。此外,一个精心设计的迭代重新加权的训练策略,提出了解决具有挑战性的多模态不一致性问题。我们证明了我们的方法的有效性7计算机视觉任务的视频。大量的定量和感知实验表明,我们的方法在盲视频时间一致性方面比最先进的方法获得了更优越的性能。我们进一步扩展DVP的视频传播,并证明其有效性传播三种不同类型的信息(颜色,艺术风格,对象分割)。提出了一种基于伪标签的渐进传播策略,以提高DVP在视频传播中的性能。我们的源代码可在https://github.com/ChenyangLEI/deep-video-prior上公开获取。

英文摘要:

Applying an image processing algorithm independently to each video frame often leads to temporal inconsistency in the resulting video. To address this issue, we present a novel and general approach for blind video temporal consistency. Our method is only trained on a pair of original and processed videos directly instead of a large dataset. Unlike most previous methods that enforce temporal consistency with optical flow, we show that temporal consistency can be achieved by training a convolutional neural network on a video with Deep Video Prior (DVP). Moreover, a carefully designed iteratively reweighted training strategy is proposed to address the challenging multimodal inconsistency problem. We demonstrate the effectiveness of our approach on 7 computer vision tasks on videos. Extensive quantitative and perceptual experiments show that our approach obtains superior performance than state-of-the-art methods on blind video temporal consistency. We further extend DVP to video propagation and demonstrate its effectiveness in propagating three different types of information (color, artistic style, and object segmentation). A progressive propagation strategy with pseudo labels is also proposed to enhance DVP's performance on video propagation. Our source codes are publicly available at https://github.com/ChenyangLEI/deep-video-prior.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑