学习结合纹理显著性自适应注意力以实现图像卡通化
Learning to Incorporate Texture Saliency Adaptive Attention to Image Cartoonization
AI总结：
本文提出了一种新颖的卡通纹理显著性采样器（CTSS）模块，通过在对抗学习中引入纹理显著性自适应注意力机制，在局部显著区域进行区域级对抗学习，仅使用基本对抗损失即可有效增强高分辨率图像的卡通化效果。
摘要：
图像卡通化近期由生成对抗网络（GANs）在无监督图像到图像转换的视角下主导，其中一个固有的挑战是精确捕捉并充分转移特征性的卡通风格（例如，清晰的边缘、平滑的颜色阴影、抽象的精细结构等）。现有的先进模型试图通过学习对抗性地促进边缘、引入风格迁移损失或学习从多个表示空间对齐风格来增强卡通化效果。本文表明，仅使用基本的对抗损失即可轻松实现更鲜明、更生动的卡通化效果。观察到卡通风格在卡通纹理显著的局部图像区域更为明显，我们构建了一个与正常图像级分支并行的区域级对抗学习分支，该分支在卡通纹理显著的局部图块上约束对抗学习，以更好地感知和转移卡通纹理特征。为此，提出了一种新颖的卡通纹理显著性采样器（CTSS）模块，用于从训练数据中动态采样卡通纹理显著的图块。通过大量实验，我们证明对抗学习中的纹理显著性自适应注意力作为图像卡通化相关方法中缺失的成分，在促进和增强图像卡通风格化方面具有重要意义，尤其是对于高分辨率输入图片。
英文摘要：
Image cartoonization is recently dominated by generative adversarial networks (GANs) from the perspective of unsupervised image-to-image translation, in which an inherent challenge is to precisely capture and sufficiently transfer characteristic cartoon styles (e.g., clear edges, smooth color shading, abstract fine structures, etc.). Existing advanced models try to enhance cartoonization effect by learning to promote edges adversarially, introducing style transfer loss, or learning to align style from multiple representation space. This paper demonstrates that more distinct and vivid cartoonization effect could be easily achieved with only basic adversarial loss. Observing that cartoon style is more evident in cartoon-texture-salient local image regions, we build a region-level adversarial learning branch in parallel with the normal image-level one, which constrains adversarial learning on cartoon-texture-salient local patches for better perceiving and transferring cartoon texture features. To this end, a novel cartoon-texture-saliency-sampler (CTSS) module is proposed to dynamically sample cartoon-texture-salient patches from training data. With extensive experiments, we demonstrate that texture saliency adaptive attention in adversarial learning, as a missing ingredient of related methods in image cartoonization, is of significant importance in facilitating and enhancing image cartoon stylization, especially for high-resolution input pictures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
