arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.00418cs.CV
收藏

基于深度多尺度分量字典的盲人脸恢复

Blind Face Restoration via Deep Multi-scale Component Dictionaries

Faculty of Computing, Harbin Institute of Technology(哈尔滨工业大学计算学部);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算学系);Department of Computer Science, The University of Hong Kong(香港大学计算机科学系);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院);Peng Cheng Lab, Shenzhen(鹏城实验室)

Xiaoming Li, Chaofeng Chen, Shangchen Zhou, Xianhui Lin, Wangmeng Zuo, Lei Zhang

摘要:

近年来,基于参考的人脸恢复方法因其在真实低质量图像上恢复高频细节的能力而受到广泛关注。然而,这些方法中的大多数需要相同身份的高质量参考图像,使得它们仅适用于有限的场景。为了解决这个问题,本文提出了一种深度人脸字典网络(称为DFDNet)来指导退化观测的恢复过程。首先,我们使用K-means从高质量图像中为感知上重要的面部组件(\ie,左/右眼,鼻子和嘴巴)生成深度字典。接下来,对于降级的输入,我们从相应的字典中匹配并选择最相似的组件特征,并通过建议的字典特征传输(DFT)块将高质量的细节传输到输入。特别是,组件AdaIN被用来消除输入和字典特征(\eg,照明)之间的风格多样性,并提出了一个置信度分数,自适应地融合输入的字典特征。最后,多尺度字典采用渐进的方式,使粗到精的恢复。实验表明,我们提出的方法可以实现合理的性能在定量和定性评价,更重要的是,可以产生真实的和有前途的结果,而不需要一个身份归属参考的真实退化图像。源代码和模型可在\url{https://github.com/csxmli2016/DFDNet}上获得。

英文摘要:

Recent reference-based face restoration methods have received considerable attention due to their great capability in recovering high-frequency details on real low-quality images. However, most of these methods require a high-quality reference image of the same identity, making them only applicable in limited scenes. To address this issue, this paper suggests a deep face dictionary network (termed as DFDNet) to guide the restoration process of degraded observations. To begin with, we use K-means to generate deep dictionaries for perceptually significant face components (\ie, left/right eyes, nose and mouth) from high-quality images. Next, with the degraded input, we match and select the most similar component features from their corresponding dictionaries and transfer the high-quality details to the input via the proposed dictionary feature transfer (DFT) block. In particular, component AdaIN is leveraged to eliminate the style diversity between the input and dictionary features (\eg, illumination), and a confidence score is proposed to adaptively fuse the dictionary feature to the input. Finally, multi-scale dictionaries are adopted in a progressive manner to enable the coarse-to-fine restoration. Experiments show that our proposed method can achieve plausible performance in both quantitative and qualitative evaluation, and more importantly, can generate realistic and promising results on real degraded images without requiring an identity-belonging reference. The source code and models are available at \url{https://github.com/csxmli2016/DFDNet}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑