Scenimefy：通过半监督图像到图像翻译学习制作动漫场景
Scenimefy: Learning to Craft Anime Scene via Semi-Supervised Image-to-Image Translation
AI总结：
针对真实图像转动漫场景任务中语义保留差、风格化不足及数据稀缺问题，提出半监督框架Scenimefy，借助伪配对数据、对比损失等提升效果，并贡献高分辨率动漫场景数据集，性能超越现有基线。
摘要：
从复杂的真实世界图像自动高质量渲染动漫场景具有重要的实用价值。该任务的挑战在于场景的复杂性、动漫风格的独特特征，以及缺乏高质量数据集来弥合域差距。尽管已有颇具前景的尝试，但此前的研究仍无法取得令人满意的结果，难以同时实现一致的语义保留、显著的风格化效果和精细的细节。本研究中，我们提出Scenimefy——一种新型半监督图像到图像翻译框架，以应对这些挑战。我们的方法利用结构一致的伪配对数据引导学习，简化了纯无监督设置。这些伪数据由受语义约束的StyleGAN（利用CLIP等丰富的模型先验）独特生成。我们进一步应用分割引导的数据选择，以获得高质量的伪监督。我们还引入了逐块对比风格损失，以提升风格化效果和精细细节。此外，我们贡献了一个高分辨率动漫场景数据集，以推动未来的研究。我们的大量实验表明，该方法在感知质量和定量性能方面均优于现有最先进基线。
英文摘要：
Automatic high-quality rendering of anime scenes from complex real-world images is of significant practical value. The challenges of this task lie in the complexity of the scenes, the unique features of anime style, and the lack of high-quality datasets to bridge the domain gap. Despite promising attempts, previous efforts are still incompetent in achieving satisfactory results with consistent semantic preservation, evident stylization, and fine details. In this study, we propose Scenimefy, a novel semi-supervised image-to-image translation framework that addresses these challenges. Our approach guides the learning with structure-consistent pseudo paired data, simplifying the pure unsupervised setting. The pseudo data are derived uniquely from a semantic-constrained StyleGAN leveraging rich model priors like CLIP. We further apply segmentation-guided data selection to obtain high-quality pseudo supervision. A patch-wise contrastive style loss is introduced to improve stylization and fine details. Besides, we contribute a high-resolution anime scene dataset to facilitate future research. Our extensive experiments demonstrate the superiority of our method over state-of-the-art baselines in terms of both perceptual quality and quantitative performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
