arXiv 2307.024572023-07-05cs.CVcs.AIcs.MM

DeSRA：检测并删除基于GAN的真实世界超分辨率模型伪影

DeSRA: Detect and Delete the Artifacts of GAN-based Real-World Super-Resolution Models

University of Macau（澳门大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Tencent PCG（腾讯平台与内容事业群）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Tencent Online Video（腾讯在线视频）

Liangbin Xie, Xintao Wang, Xiangyu Chen, Gen Li, Ying Shan, Jiantao Zhou, Chao Dong

备注：: The code and models will be made publicly at https://github.com/TencentARC/DeSRA

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出DeSRA方法，通过相对局部方差距离和语义感知阈值检测GAN超分辨率模型在真实场景中产生的伪影，并用少量样本微调模型以删除伪影，提升实际应用能力。

摘要：

基于生成对抗网络（GAN）的图像超分辨率（SR）在恢复逼真细节方面取得了巨大成功。然而，众所周知，基于GAN的SR模型不可避免地会产生令人不快且不理想的伪影，尤其是在实际场景中。以往的工作通常在训练阶段通过额外的损失惩罚来抑制伪影，但这些方法仅对训练过程中生成的分布内伪影类型有效。当应用于真实世界场景时，我们观察到这些改进的方法在推理过程中仍然会产生明显令人厌烦的伪影。在本文中，我们分析了在没有真实标签的未见测试数据中产生的GAN伪影的原因和特征。然后，我们开发了一种新方法，即DeSRA，用于在实践中检测并删除这些SR伪影。具体来说，我们提出测量MSE-SR结果与GAN-SR结果之间的相对局部方差距离，并基于上述距离和语义感知阈值定位问题区域。在检测到伪影区域后，我们开发了一种微调程序，用少量样本改进基于GAN的SR模型，使其能够处理更多未见真实数据中类似类型的伪影。借助我们的DeSRA，我们可以成功消除推理中的伪影，并提高SR模型在真实世界场景中的应用能力。代码将在https://github.com/TencentARC/DeSRA上提供。

英文摘要：

Image super-resolution (SR) with generative adversarial networks (GAN) has achieved great success in restoring realistic details. However, it is notorious that GAN-based SR models will inevitably produce unpleasant and undesirable artifacts, especially in practical scenarios. Previous works typically suppress artifacts with an extra loss penalty in the training phase. They only work for in-distribution artifact types generated during training. When applied in real-world scenarios, we observe that those improved methods still generate obviously annoying artifacts during inference. In this paper, we analyze the cause and characteristics of the GAN artifacts produced in unseen test data without ground-truths. We then develop a novel method, namely, DeSRA, to Detect and then Delete those SR Artifacts in practice. Specifically, we propose to measure a relative local variance distance from MSE-SR results and GAN-SR results, and locate the problematic areas based on the above distance and semantic-aware thresholds. After detecting the artifact regions, we develop a finetune procedure to improve GAN-based SR models with a few samples, so that they can deal with similar types of artifacts in more unseen real data. Equipped with our DeSRA, we can successfully eliminate artifacts from inference and improve the ability of SR models to be applied in real-world scenarios. The code will be available at https://github.com/TencentARC/DeSRA.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。