arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01598eess.IVcs.AIcs.CV
收藏

APISR:受动画制作启发的真实世界动画超分辨率

APISR: Anime Production Inspired Real-World Anime Super-Resolution

University of Michigan(密歇根大学);Zhejiang University(浙江大学);Yale University(耶鲁大学)

Boyang Wang, Fengyu Yang, Xihang Yu, Chao Zhang, Hanbin Zhao

AI总结:

本文借鉴动画制作流程构建API数据集,并通过退化压缩建模、线条增强伪真值和平衡孪生感知损失,提升真实世界动画超分辨率效果。

摘要:

尽管真实世界动画超分辨率(SR)在超分辨率领域受到越来越多的关注,但现有方法仍然采用来自照片级真实感领域的技术。本文分析了动画制作工作流,并重新思考如何利用其特征来服务于真实世界动画超分辨率。首先,我们认为由于手绘帧会被重复使用,动画超分辨率并不需要视频网络和数据集。相反,我们提出了一种动画图像收集流程,通过从视频源中选择压缩程度最低且信息量最高的帧来实现。基于该流程,我们构建了面向动画制作的图像(Anime Production-oriented Image,API)数据集。此外,我们识别出两个动画特有的挑战:失真且微弱的手绘线条,以及不希望出现的色彩伪影。针对第一个问题,我们在图像退化模型中引入了面向预测的压缩模块,并采用增强手绘线条的伪真值准备方法。此外,我们提出了结合动画与照片级真实感高层特征的平衡孪生感知损失,以缓解不希望出现的色彩伪影并提升视觉清晰度。我们在公开基准上通过大量实验评估了所提方法,结果表明该方法优于在最先进动画数据集上训练的方法。

英文摘要:

While real-world anime super-resolution (SR) has gained increasing attention in the SR community, existing methods still adopt techniques from the photorealistic domain. In this paper, we analyze the anime production workflow and rethink how to use characteristics of it for the sake of the real-world anime SR. First, we argue that video networks and datasets are not necessary for anime SR due to the repetition use of hand-drawing frames. Instead, we propose an anime image collection pipeline by choosing the least compressed and the most informative frames from the video sources. Based on this pipeline, we introduce the Anime Production-oriented Image (API) dataset. In addition, we identify two anime-specific challenges of distorted and faint hand-drawn lines and unwanted color artifacts. We address the first issue by introducing a prediction-oriented compression module in the image degradation model and a pseudo-ground truth preparation with enhanced hand-drawn lines. In addition, we introduce the balanced twin perceptual loss combining both anime and photorealistic high-level features to mitigate unwanted color artifacts and increase visual clarity. We evaluate our method through extensive experiments on the public benchmark, showing our method outperforms state-of-the-art anime dataset-trained approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑