RestoreFormer++：利用未退化键值对实现真实世界盲人脸复原
RestoreFormer++: Towards Real-World Blind Face Restoration from Undegraded Key-Value Pairs
AI总结：
本文提出 RestoreFormer++，通过多尺度多头交叉注意力、重建导向字典和扩展退化模型，增强上下文与先验交互并缩小合成到真实差距，在多类数据集上取得最优盲人脸复原效果。
摘要：
盲人脸复原旨在从具有未知退化的人脸图像中恢复高质量人脸图像。当前算法主要引入先验来补充高质量细节，并取得了令人瞩目的进展。然而，这些算法大多忽略了人脸中丰富的上下文信息及其与先验之间的相互作用，从而导致性能次优。此外，它们较少关注合成场景与真实世界场景之间的差距，限制了在真实世界应用中的鲁棒性和泛化能力。在本文中，我们提出 RestoreFormer++：一方面，它引入全空间注意力机制来建模上下文信息及其与先验的相互作用；另一方面，它探索一种扩展退化模型，以帮助生成更逼真的退化人脸图像，从而缓解合成到真实世界的差距。与当前算法相比，RestoreFormer++ 具有若干关键优势。首先，不同于传统视觉 transformer 使用多头自注意力机制，我们在多尺度特征上引入多头交叉注意力，以充分探索受损信息与高质量先验之间的空间交互。通过这种方式，它能够促使 RestoreFormer++ 以更高的真实性和保真度恢复人脸图像。其次，与面向识别的字典相比，我们学习一个面向重建的字典作为先验，该字典包含更加多样的高质量面部细节，并且更符合复原目标。第三，我们引入一种扩展退化模型，其中包含更逼真的退化场景用于训练数据合成，因此有助于增强 RestoreFormer++ 模型的鲁棒性和泛化能力。大量实验表明，RestoreFormer++ 在合成数据集和真实世界数据集上均优于最先进算法。
英文摘要：
Blind face restoration aims at recovering high-quality face images from those with unknown degradations. Current algorithms mainly introduce priors to complement high-quality details and achieve impressive progress. However, most of these algorithms ignore abundant contextual information in the face and its interplay with the priors, leading to sub-optimal performance. Moreover, they pay less attention to the gap between the synthetic and real-world scenarios, limiting the robustness and generalization to real-world applications. In this work, we propose RestoreFormer++, which on the one hand introduces fully-spatial attention mechanisms to model the contextual information and the interplay with the priors, and on the other hand, explores an extending degrading model to help generate more realistic degraded face images to alleviate the synthetic-to-real-world gap. Compared with current algorithms, RestoreFormer++ has several crucial benefits. First, instead of using a multi-head self-attention mechanism like the traditional visual transformer, we introduce multi-head cross-attention over multi-scale features to fully explore spatial interactions between corrupted information and high-quality priors. In this way, it can facilitate RestoreFormer++ to restore face images with higher realness and fidelity. Second, in contrast to the recognition-oriented dictionary, we learn a reconstruction-oriented dictionary as priors, which contains more diverse high-quality facial details and better accords with the restoration target. Third, we introduce an extending degrading model that contains more realistic degraded scenarios for training data synthesizing, and thus helps to enhance the robustness and generalization of our RestoreFormer++ model. Extensive experiments show that RestoreFormer++ outperforms state-of-the-art algorithms on both synthetic and real-world datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
