arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.08357eess.IVcs.CV
收藏

学习频率感知动态网络实现高效超分辨率

Learning Frequency-aware Dynamic Network for Efficient Super-Resolution

School of Software, Tsinghua University(清华大学软件学院);The University of Sydney(悉尼大学)

Wenbin Xie, Dehua Song, Chang Xu, Chunjing Xu, Hui Zhang, Yunhe Wang

摘要:

基于深度学习的方法,特别是卷积神经网络(CNN),已经成功应用于单图像超分辨率(SISR)领域。为了获得更好的逼真度和视觉质量,大多数现有的网络都是繁重的设计和大量的计算。然而,现代移动设备的计算资源是有限的,这不能轻易地支持昂贵的成本。为此,本文探讨了一种新的频率感知的动态网络划分成多个部分的输入,根据其系数在离散余弦变换(DCT)域。在实践中,高频部分将使用昂贵的操作来处理,而低频部分被分配以廉价的操作来减轻计算负担。由于属于低频区域的像素或图像块包含相对较少的纹理细节,因此这种动态网络不会影响所产生的超分辨率图像的质量。此外,我们将预测器嵌入到所提出的动态网络中,以端到端微调手工制作的频率感知掩码。在基准SISR模型和数据集上进行的大量实验表明,频率感知动态网络可以用于各种SISR神经架构,以获得视觉质量和计算复杂度之间的更好权衡。例如,我们可以将SR模型的FLOP降低约50%,同时保持最先进的SISR性能。

英文摘要:

Deep learning based methods, especially convolutional neural networks (CNNs) have been successfully applied in the field of single image super-resolution (SISR). To obtain better fidelity and visual quality, most of existing networks are of heavy design with massive computation. However, the computation resources of modern mobile devices are limited, which cannot easily support the expensive cost. To this end, this paper explores a novel frequency-aware dynamic network for dividing the input into multiple parts according to its coefficients in the discrete cosine transform (DCT) domain. In practice, the high-frequency part will be processed using expensive operations and the lower-frequency part is assigned with cheap operations to relieve the computation burden. Since pixels or image patches belong to low-frequency areas contain relatively few textural details, this dynamic network will not affect the quality of resulting super-resolution images. In addition, we embed predictors into the proposed dynamic network to end-to-end fine-tune the handcrafted frequency-aware masks. Extensive experiments conducted on benchmark SISR models and datasets show that the frequency-aware dynamic network can be employed for various SISR neural architectures to obtain the better tradeoff between visual quality and computational complexity. For instance, we can reduce the FLOPs of SR models by approximate 50% while preserving state-of-the-art SISR performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑