arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2106.01863cs.CVcs.LGeess.IV
收藏

基于C2-Matching的鲁棒参考超分辨率

Robust Reference-based Super-Resolution via C2-Matching

Nanyang Technological University(南洋理工大学)

Yuming Jiang, Kelvin C. K. Chan, Xintao Wang, Chen Change Loy, Ziwei Liu

摘要:

基于参考的超分辨率(Ref-SR)最近已经成为一种有前途的范例,通过引入额外的高分辨率(HR)参考图像来增强低分辨率(LR)输入图像。现有的Ref-SR方法大多依赖于隐式对应匹配从参考图像中借用HR纹理来补偿输入图像中的信息损失。然而,由于输入图像和参考图像之间存在两个间隙:变换间隙(例如,缩放和旋转)和分辨率间隙(例如,HR和LR),因此难以执行局部转移。为了应对这些挑战,我们提出了C2-Matching,它产生显式的鲁棒匹配交叉变换和分辨率。1)对于变换间隙,我们提出了一个对比对应网络,它使用输入图像的增强视图来学习变换鲁棒对应。2)对于分辨率差距,我们采用了师生相关蒸馏,从更容易的HR-HR匹配中提取知识,以指导更模糊的LR-HR匹配。3)最后,我们设计了一个动态聚合模块来解决潜在的不对齐问题。此外,为了忠实地评估Ref-SR在现实环境下的性能,我们贡献了Webly引用的SR(WR-SR)数据集,模仿实际使用场景。大量的实验表明,我们提出的C2匹配显着优于国家的艺术超过1dB的标准CUFED 5基准。值得注意的是,它在WR-SR数据集上也表现出很强的泛化能力,以及在大规模和旋转变换中的鲁棒性。

英文摘要:

Reference-based Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image by introducing an additional high-resolution (HR) reference image. Existing Ref-SR methods mostly rely on implicit correspondence matching to borrow HR textures from reference images to compensate for the information loss in input images. However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e.g. scale and rotation) and the resolution gap (e.g. HR and LR). To tackle these challenges, we propose C2-Matching in this work, which produces explicit robust matching crossing transformation and resolution. 1) For the transformation gap, we propose a contrastive correspondence network, which learns transformation-robust correspondences using augmented views of the input image. 2) For the resolution gap, we adopt a teacher-student correlation distillation, which distills knowledge from the easier HR-HR matching to guide the more ambiguous LR-HR matching. 3) Finally, we design a dynamic aggregation module to address the potential misalignment issue. In addition, to faithfully evaluate the performance of Ref-SR under a realistic setting, we contribute the Webly-Referenced SR (WR-SR) dataset, mimicking the practical usage scenario. Extensive experiments demonstrate that our proposed C2-Matching significantly outperforms state of the arts by over 1dB on the standard CUFED5 benchmark. Notably, it also shows great generalizability on WR-SR dataset as well as robustness across large scale and rotation transformations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑