arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.19387eess.IVcs.CV
收藏

SeD:用于图像超分辨率的语义感知判别器

SeD: Semantic-Aware Discriminator for Image Super-Resolution

University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);Galbot(智位机器人)

Bingchen Li, Xin Li, Hanxin Zhu, Yeying Jin, Ruoyu Feng, Zhizheng Zhang, Zhibo Chen

AI总结:

针对GAN超分辨率中粗粒度分布学习易产生虚拟纹理的问题,本文提出SeD,利用PVM语义与空间交叉注意力引导判别器进行细粒度真假判别,提升SR与Real SR的真实感生成效果。

摘要:

生成对抗网络(GANs)已被广泛用于在图像超分辨率(SR)任务中恢复生动纹理。具体而言,研究中使用一个判别器,使SR网络能够以对抗训练的方式学习真实高质量图像的分布。然而,这种分布学习过于粗粒度,容易受到虚拟纹理的影响,并产生违背直觉的生成结果。为缓解该问题,我们提出了简单且有效的语义感知判别器(Semantic-aware Discriminator,记为SeD),通过引入图像语义作为条件,促使SR网络学习细粒度分布。具体来说,我们旨在从训练良好的语义提取器中挖掘图像语义。在不同语义下,判别器能够单独且自适应地区分真假图像,从而引导SR网络学习更细粒度的语义感知纹理。为获得准确且丰富的语义,我们充分利用近期流行的、在大规模数据集上训练的预训练视觉模型(PVMs),随后通过精心设计的空间交叉注意力模块将其语义特征整合到判别器中。通过这种方式,我们提出的语义感知判别器使SR网络能够生成更具照片真实感且更令人愉悦的图像。在SR和Real SR两个典型任务上的大量实验证明了我们所提方法的有效性。

英文摘要:

Generative Adversarial Networks (GANs) have been widely used to recover vivid textures in image super-resolution (SR) tasks. In particular, one discriminator is utilized to enable the SR network to learn the distribution of real-world high-quality images in an adversarial training manner. However, the distribution learning is overly coarse-grained, which is susceptible to virtual textures and causes counter-intuitive generation results. To mitigate this, we propose the simple and effective Semantic-aware Discriminator (denoted as SeD), which encourages the SR network to learn the fine-grained distributions by introducing the semantics of images as a condition. Concretely, we aim to excavate the semantics of images from a well-trained semantic extractor. Under different semantics, the discriminator is able to distinguish the real-fake images individually and adaptively, which guides the SR network to learn the more fine-grained semantic-aware textures. To obtain accurate and abundant semantics, we take full advantage of recently popular pretrained vision models (PVMs) with extensive datasets, and then incorporate its semantic features into the discriminator through a well-designed spatial cross-attention module. In this way, our proposed semantic-aware discriminator empowered the SR network to produce more photo-realistic and pleasing images. Extensive experiments on two typical tasks, i.e., SR and Real SR have demonstrated the effectiveness of our proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑