arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1807.10547cs.CV
收藏

CrossNet:一种基于跨尺度翘曲的端到端参考超分辨率网络

CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping

Tsinghua University(清华大学);Tsinghua-Berkeley Shenzhen Institute(清华-伯克利深圳学院);Hong Kong University of Science and Technology(香港科技大学)

Haitian Zheng, Mengqi Ji, Haoqian Wang, Yebin Liu, Lu Fang

摘要:

基于参考的超分辨率(RefSR)在给定外部高分辨率(HR)参考图像的情况下对低分辨率(LR)图像进行超分辨率处理,其中参考图像和LR图像共享相似的视点,但具有显著的分辨率差距x8。现有的RefSR方法以级联的方式工作,例如块匹配,然后是具有两个独立定义的目标函数的合成流水线,导致块间不对齐,网格效应和低效的优化。为了解决这些问题,我们提出了CrossNet,这是一个使用跨尺度翘曲的端到端和全卷积深度神经网络。我们的网络包含图像编码器,跨尺度扭曲层和融合解码器:编码器用于从LR和参考图像中提取多尺度特征;跨尺度扭曲层在空间上将参考特征图与LR特征图对齐;解码器最终聚合来自两个域的特征图以合成HR输出。使用跨尺度扭曲,我们的网络能够以端到端的方式在像素级执行空间对齐,这在精度(约2dB-4dB)和效率(快100倍以上)方面都提高了现有方案。

英文摘要:

The Reference-based Super-resolution (RefSR) super-resolves a low-resolution (LR) image given an external high-resolution (HR) reference image, where the reference image and LR image share similar viewpoint but with significant resolution gap x8. Existing RefSR methods work in a cascaded way such as patch matching followed by synthesis pipeline with two independently defined objective functions, leading to the inter-patch misalignment, grid effect and inefficient optimization. To resolve these issues, we present CrossNet, an end-to-end and fully-convolutional deep neural network using cross-scale warping. Our network contains image encoders, cross-scale warping layers, and fusion decoder: the encoder serves to extract multi-scale features from both the LR and the reference images; the cross-scale warping layers spatially aligns the reference feature map with the LR feature map; the decoder finally aggregates feature maps from both domains to synthesize the HR output. Using cross-scale warping, our network is able to perform spatial alignment at pixel-level in an end-to-end fashion, which improves the existing schemes both in precision (around 2dB-4dB) and efficiency (more than 100 times faster).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑