arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2401.03395cs.CV

基于深度学习的图像和视频修复:综述

Deep Learning-based Image and Video Inpainting: A Survey

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Sichuan University(四川大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学)

Weize Quan, Jiaxi Chen, Yanli Liu, Dong-Ming Yan, Peter Wonka

AI总结:

本文综述了基于深度学习的图像与视频修复方法,按修复流程分类介绍CNN、GAN等架构及模块设计,评估基准与指标,并探讨应用现状与未来挑战。

摘要:

图像和视频修复是计算机视觉和计算机图形学中的一个经典问题,旨在为图像和视频的缺失区域填补合理且逼真的内容。随着深度学习的发展,该问题近期取得了显著进展。本文旨在全面回顾基于深度学习的图像和视频修复方法。具体而言,我们从高层修复流程的视角将现有方法划分为不同类别,介绍了包括CNN、VAE、GAN、扩散模型等在内的不同深度学习架构,我们回顾了训练目标和常用基准数据集,介绍了低级像素和高级感知相似度的评估指标,进行了性能评估,并讨论了代表性修复方法的的优势与不足。我们还讨论了相关的实际应用。最后,我们讨论了开放性挑战并提出了潜在的未来研究方向。

英文摘要:

Image and video inpainting is a classic problem in computer vision and computer graphics, aiming to fill in the plausible and realistic content in the missing areas of images and videos. With the advance of deep learning, this problem has achieved significant progress recently. The goal of this paper is to comprehensively review the deep learning-based methods for image and video inpainting. Specifically, we sort existing methods into different categories from the perspective of their high-level inpainting pipeline, present different deep learning architectures, including CNN, VAE, GAN, diffusion models, etc., and summarize techniques for module design. We review the training objectives and the common benchmark datasets. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods. We also discuss related real-world applications. Finally, we discuss open challenges and suggest potential future research directions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑