SuperInpaint：学习细节增强的注意力隐式表示以实现超分辨率图像修复
SuperInpaint: Learning Detail-Enhanced Attentional Implicit Representation for Super-resolutional Image Inpainting
AI总结：
本文提出了SuperInpaint任务及细节增强的注意力隐式表示（DEAR）方法，通过单一模型结合自适应高通滤波、unmask-attentional模块和隐式表示，实现了低分辨率图像缺失区域的高质量任意超分辨率重建，在四项指标上显著优于现有方法。
摘要：
在这项工作中，我们引入了一项具有挑战性的图像恢复任务，称为SuperInpaint，旨在重建低分辨率图像中的缺失区域，并生成具有任意更高分辨率的完整图像。我们发现，通过堆叠最先进的超分辨率和图像修复方法无法有效解决该任务，因为它们会放大彼此的缺陷，导致产生明显的伪影。为了克服这些限制，我们提出了细节增强的注意力隐式表示（DEAR），它能够通过单一模型实现SuperInpaint，从而生成具有任意分辨率的高质量完整图像。具体而言，我们使用深度卷积网络提取输入图像的潜在嵌入，然后通过自适应高通滤波器增强潜在嵌入的高频分量。这产生了细节增强的语义嵌入。我们进一步将语义嵌入输入到一个抑制无效掩码像素嵌入的unmask-attentional模块中。此外，我们提取了一个像素级重要性图，指示哪些像素应被用于图像重建。给定我们想要重建的像素坐标，我们首先在输入图像中收集其相邻像素，并提取它们细节增强的语义嵌入、unmask-attentional语义嵌入、重要性值以及到目标像素的空间距离。然后，我们将上述所有项输入到一个隐式表示中，并生成指定像素的颜色。为了评估我们的方法，我们针对这项新任务扩展了三个现有数据集，并使用SOTA修复和超分辨率方法构建了18个有意义的基准。大量实验结果表明，我们的方法在四个广泛使用的指标上以显著优势优于所有现有方法。
英文摘要：
In this work, we introduce a challenging image restoration task, referred to as SuperInpaint, which aims to reconstruct missing regions in low-resolution images and generate completed images with arbitrarily higher resolutions. We have found that this task cannot be effectively addressed by stacking state-of-the-art super-resolution and image inpainting methods as they amplify each other's flaws, leading to noticeable artifacts. To overcome these limitations, we propose the detail-enhanced attentional implicit representation (DEAR) that can achieve SuperInpaint with a single model, resulting in high-quality completed images with arbitrary resolutions. Specifically, we use a deep convolutional network to extract the latent embedding of an input image and then enhance the high-frequency components of the latent embedding via an adaptive high-pass filter. This leads to detail-enhanced semantic embedding. We further feed the semantic embedding into an unmask-attentional module that suppresses embeddings from ineffective masked pixels. Additionally, we extract a pixel-wise importance map that indicates which pixels should be used for image reconstruction. Given the coordinates of a pixel we want to reconstruct, we first collect its neighboring pixels in the input image and extract their detail-enhanced semantic embeddings, unmask-attentional semantic embeddings, importance values, and spatial distances to the desired pixel. Then, we feed all the above terms into an implicit representation and generate the color of the specified pixel. To evaluate our method, we extend three existing datasets for this new task and build 18 meaningful baselines using SOTA inpainting and super-resolution methods. Extensive experimental results demonstrate that our method outperforms all existing methods by a significant margin on four widely used metrics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
