arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.13697cs.CVcs.AIcs.LG
收藏

基于注意力的多参考学习图像超分辨率

Attention-based Multi-Reference Learning for Image Super-Resolution

Centre for Vision, Speech and Signal Processing(视觉、语音与信号处理中心);University of Surrey(萨里大学)

Marco Pesavento, Marco Volino, Adrian Hilton

摘要:

本文提出了一种新的基于注意力的多参考超分辨率网络(AMRSR),该网络在给定低分辨率图像的情况下,学习自适应地将多个参考图像中最相似的纹理传输到超分辨率输出,同时保持空间相干性。使用多个参考图像与基于注意力的采样一起被证明在多个基准数据集上实现比最先进的参考超分辨率方法显著提高的性能。参考超分辨率方法最近被提出来通过从高分辨率参考图像提供附加信息来克服图像超分辨率的不适定问题。多参考超分辨率通过提供更多样化的图像特征池来扩展这种方法,以克服固有的信息不足,同时保持存储器效率。提出了一种新的基于注意力的分层采样方法,该方法基于感知损失来学习低分辨率图像特征与多个参考图像之间的相似性。消融演示了多参考和分层注意抽样的整体性能的贡献。感知和定量地面实况评估表明,即使参考图像偏离目标图像显着的性能显着改善。该项目的网站可以在https://marcopesavento.github.io/AMRSR/上找到

英文摘要:

This paper proposes a novel Attention-based Multi-Reference Super-resolution network (AMRSR) that, given a low-resolution image, learns to adaptively transfer the most similar texture from multiple reference images to the super-resolution output whilst maintaining spatial coherence. The use of multiple reference images together with attention-based sampling is demonstrated to achieve significantly improved performance over state-of-the-art reference super-resolution approaches on multiple benchmark datasets. Reference super-resolution approaches have recently been proposed to overcome the ill-posed problem of image super-resolution by providing additional information from a high-resolution reference image. Multi-reference super-resolution extends this approach by providing a more diverse pool of image features to overcome the inherent information deficit whilst maintaining memory efficiency. A novel hierarchical attention-based sampling approach is introduced to learn the similarity between low-resolution image features and multiple reference images based on a perceptual loss. Ablation demonstrates the contribution of both multi-reference and hierarchical attention-based sampling to overall performance. Perceptual and quantitative ground-truth evaluation demonstrates significant improvement in performance even when the reference images deviate significantly from the target image. The project website can be found at https://marcopesavento.github.io/AMRSR/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑