Lightweight image super-resolution (SR) networks have the utmost significance for real-world applications. There are several deep learning based SR methods with remarkable performance, but their memory and computational cost are hindrances in practical usage. To tackle this problem, we propose a Multi-Attentive Feature Fusion Super-Resolution Network (MAFFSRN). MAFFSRN consists of proposed feature fusion groups (FFGs) that serve as a feature extraction block. Each FFG contains a stack of proposed multi-attention blocks (MAB) that are combined in a novel feature fusion structure. Further, the MAB with a cost-efficient attention mechanism (CEA) helps us to refine and extract the features using multiple attention mechanisms. The comprehensive experiments show the superiority of our model over the existing state-of-the-art. We participated in AIM 2020 efficient SR challenge with our MAFFSRN model and won 1st, 3rd, and 4th places in memory usage, floating-point operations (FLOPs) and number of parameters, respectively.
Tencent（腾讯）；JD Explore Academy（京东探索研究院）；University of Southern California（南加州大学）；Xidian University（西安电子科技大学）
Single Image Super-Resolution (SISR) tasks have achieved significant performance with deep neural networks. However, the large number of parameters in CNN-based met-hods for SISR tasks require heavy computations. Although several efficient SISR models have been recently proposed, most are handcrafted and thus lack flexibility. In this work, we propose a novel differentiable Neural Architecture Search (NAS) approach on both the cell-level and network-level to search for lightweight SISR models. Specifically, the cell-level search space is designed based on an information distillation mechanism,…
Nanjing University of Posts and Telecommunications（南京邮电大学）；East China Normal University（华东师范大学）；National Institute of Informatics（国立情报学研究所）；Kyushu Institute of Technology（九州工业大学）
Recently, the single image super-resolution (SISR) approaches with deep and complex convolutional neural network structures have achieved promising performance. However, those methods improve the performance at the cost of higher memory consumption, which is difficult to be applied for some mobile devices with limited storage and computing resources. To solve this problem, we present a lightweight multi-scale feature interaction network (MSFIN). For lightweight SISR, MSFIN expands the receptive field and adequately exploits the informative features of the low-resolution observed images from va…
Lightweight super resolution networks have extremely importance for real-world applications. In recent years several SR deep learning approaches with outstanding achievement have been introduced by sacrificing memory and computational cost. To overcome this problem, a novel lightweight super resolution network is proposed, which improves the SOTA performance in lightweight SR and performs roughly similar to computationally expensive networks. Multi-Path Residual Network designs with a set of Residual concatenation Blocks stacked with Adaptive Residual Blocks: ($i$) to adaptively extract inform…
Nanjing Normal University（南京师范大学）；University of Leicester（莱斯特大学）
Lightweight neural networks for single-image super-resolution (SISR) tasks have made substantial breakthroughs in recent years. Compared to low-frequency information, high-frequency detail is much more difficult to reconstruct. Most SISR models allocate equal computational resources for low-frequency and high-frequency information, which leads to redundant processing of simple low-frequency information and inadequate recovery of more challenging high-frequency information. We propose a novel High-Frequency Focused Network (HFFN) through High-Frequency Focused Blocks (HFFBs) that selectively en…
Nanjing University of Science and Technology（南京理工大学）；Southeast University（东南大学）；Hong Kong University of Science and Technology（香港科技大学）
Recently, deep convolutional neural network methods have achieved an excellent performance in image superresolution (SR), but they can not be easily applied to embedded devices due to large memory cost. To solve this problem, we propose a pyramidal dense attention network (PDAN) for lightweight image super-resolution in this paper. In our method, the proposed pyramidal dense learning can gradually increase the width of the densely connected layer inside a pyramidal dense block to extract deep features efficiently. Meanwhile, the adaptive group convolution that the number of groups grows linear…
Single-Image Super Resolution (SISR) is a classical computer vision problem and it has been studied for over decades. With the recent success of deep learning methods, recent work on SISR focuses solutions with deep learning methodologies and achieves state-of-the-art results. However most of the state-of-the-art SISR methods contain millions of parameters and layers, which limits their practical applications. In this paper, we propose a hardware (Synaptics Dolphin NPU) limitation aware, extremely lightweight quantization robust real-time super resolution network (XLSR). The proposed model's b…
We propose a lightweight, single image super-resolution network for mobile devices, named XCAT. XCAT introduces Heterogeneous Group Convolution Blocks with Cross Concatenations (HXBlock). The heterogeneous split of the input channels to the group convolution blocks reduces the number of operations, and cross concatenation allows for information flow between the intermediate input tensors of cascaded HXBlocks. Cross concatenations inside HXBlocks can also avoid using more expensive operations like 1x1 convolutions. To further prev ent expensive tensor copy operations, XCAT utilizes non-trainabl…
Recent advances in single image super-resolution (SISR) explored the power of convolutional neural network (CNN) to achieve a better performance. Despite the great success of CNN-based methods, it is not easy to apply these methods to edge devices due to the requirement of heavy computation. To solve this problem, various fast and lightweight CNN models have been proposed. The information distillation network is one of the state-of-the-art methods, which adopts the channel splitting operation to extract distilled features. However, it is not clear enough how this operation helps in the design …
Dual-domain Modulation Network for Lightweight Image Super-Resolution
Pattern Recognition and Intelligent System Laboratory, School of Artificial Intelligence, Beijing University of Posts and Telecommunications（模式识别与智能系统实验室，人工智能学院，北京邮电大学）；School of Microelectronics, Tianjin University（微电子学院，天津大学）；Institute of Advanced Technology, Nanjing University of Posts and Telecommunications（先进技术研究院，南京邮电大学）
$L^2$FMamba: Lightweight Light Field Image Super-Resolution with State Space Model
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Bigo Technology Pte. Ltd.（Bigo科技私人有限公司）；Explorer Global (Suzhou) Artificial Intelligence Technology Co., Ltd.（探索全球（苏州）人工智能科技有限公司）
Super-resolution (SR) has achieved great success due to the development of deep convolutional neural networks (CNNs). However, as the depth and width of the networks increase, CNN-based SR methods have been faced with the challenge of computational complexity in practice. Moreover, most of them train a dedicated model for each target resolution, losing generality and increasing memory requirements. To address these limitations we introduce OverNet, a deep but lightweight convolutional network to solve SISR at arbitrary scale factors with a single model. We make the following contributions: fir…
Towards Lightweight Hyperspectral Image Super-Resolution with Depthwise Separable Dilated Convolutional Network
Department of Computer Science, Aalto University, Finland（艾尔沃斯大学计算机科学系）；School of Electrical and Electronic Engineering, University of Sheffield, United Kingdom（谢菲尔德大学电子与电气工程学院）
Deep convolutional neural networks demonstrate impressive results in the super-resolution domain. A series of studies concentrate on improving peak signal noise ratio (PSNR) by using much deeper layers, which are not friendly to constrained resources. Pursuing a trade-off between the restoration capacity and the simplicity of models is still non-trivial. Recent contributions are struggling to manually maximize this balance, while our work achieves the same goal automatically with neural architecture search. Specifically, we handle super-resolution with a multi-objective approach. We also propo…
Deep learning methods have shown outstanding performance in many applications, including single-image super-resolution (SISR). With residual connection architecture, deeply stacked convolutional neural networks provide a substantial performance boost for SISR, but their huge parameters and computational loads are impractical for real-world applications. Thus, designing lightweight models with acceptable performance is one of the major tasks in current SISR research. The objective of lightweight network design is to balance a computational load and reconstruction performance. Most of the previo…
Deep learning has been successfully applied to the single-image super-resolution (SISR) task with great performance in recent years. However, most convolutional neural network based SR models require heavy computation, which limit their real-world applications. In this work, a lightweight SR network, named Adaptive Weighted Super-Resolution Network (AWSRN), is proposed for SISR to address this issue. A novel local fusion block (LFB) is designed in AWSRN for efficient residual learning, which consists of stacked adaptive weighted residual units (AWRU) and a local residual fusion unit (LRFU). Mo…
UHDRes: Ultra-High-Definition Image Restoration via Dual-Domain Decoupled Spectral Modulation
College of Computer Science and Artificial Intelligence, Wenzhou University（计算机科学与人工智能学院，温州大学）；State Key Laboratory for Novel Software Technology, Nanjing University（新型软件技术国家重点实验室，南京大学）；College of Engineering and Computer Science, Australian National University（工程与计算机科学学院，澳大利亚国立大学）
In recent years, deep learning methods have been successfully applied to single-image super-resolution tasks. Despite their great performances, deep learning methods cannot be easily applied to real-world applications due to the requirement of heavy computation. In this paper, we address this issue by proposing an accurate and lightweight deep network for image super-resolution. In detail, we design an architecture that implements a cascading mechanism upon a residual network. We also present variant models of the proposed cascading residual network to further improve efficiency. Our extensive…
University Of Electronic Science And Technology Of China（电子科技大学）
Image super-resolution reconstruction is an important task in the field of image processing technology, which can restore low resolution image to high quality image with high resolution. In recent years, deep learning has been applied in the field of image super-resolution reconstruction. With the continuous development of deep neural network, the quality of the reconstructed images has been greatly improved, but the model complexity has also been increased. In this paper, we propose two lightweight models named as MSwinSR and UGSwinSR based on Swin Transformer. The most important structure in…
Nanjing University of Posts and Telecommunications（南京邮电大学）；The Chinese University of Hong Kong（香港中文大学）；Kyushu Institute of Technology（九州工业大学）；National Institute of Informatics（国立信息学研究所）
Convolutional neural networks based single-image super-resolution (SISR) has made great progress in recent years. However, it is difficult to apply these methods to real-world scenarios due to the computational and memory cost. Meanwhile, how to take full advantage of the intermediate features under the constraints of limited parameters and calculations is also a huge challenge. To alleviate these issues, we propose a lightweight yet efficient Feature Distillation Interaction Weighted Network (FDIWN). Specifically, FDIWN utilizes a series of specially designed Feature Shuffle Weighted Groups (…
Convolutional neural network (CNN)-based methods have achieved great success for single-image superresolution (SISR). However, most models attempt to improve reconstruction accuracy while increasing the requirement of number of model parameters. To tackle this problem, in this paper, we study reducing the number of parameters and computational cost of CNN-based SISR methods while maintaining the accuracy of super-resolution reconstruction performance. To this end, we introduce a novel network architecture for SISR, which strikes a good trade-off between reconstruction quality and low computati…
Ritsumeikan University（立命馆大学）；Zhejiang University（浙江大学）；Dalian university of technology（大连理工大学）；Zhejiang Lab（之江实验室）
Deep learning-based super-resolution (SR) techniques have generally achieved excellent performance in the computer vision field. Recently, it has been proven that three-dimensional (3D) SR for medical volumetric data delivers better visual results than conventional two-dimensional (2D) processing. However, deepening and widening 3D networks increases training difficulty significantly due to the large number of parameters and small number of training samples. Thus, we propose a 3D convolutional neural network (CNN) for SR of medical volumetric data called ParallelNet using parallel connections.…
s-LWSR: Super Lightweight Super-Resolution Network
University of Chinese Academy of Sciences（中国科学院大学）；Chinese Academy of Sciences（中国科学院）；University of International Business and Economics（对外经济贸易大学）；Texas A&M University（得克萨斯农工大学）；University of Nebraska（内布拉斯加大学）
Convolutional neural network (CNN) has achieved great success on image super-resolution (SR). However, most deep CNN-based SR models take massive computations to obtain high performance. Downsampling features for multi-resolution fusion is an efficient and effective way to improve the performance of visual recognition. Still, it is counter-intuitive in the SR task, which needs to project a low-resolution input to high-resolution. In this paper, we propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task. The network c…
Infrared Image Super-Resolution via Lightweight Information Split Network
School of Computer Science and Technology, Hangzhou Dianzi University（杭州电子科技大学计算机科学与技术学院）；Shenzhen Research Institute of Big Data（深圳大数据研究院）；ETH Zurich（苏黎世联邦理工学院）；School of Information and Electronic Engineering, Zhejiang University of Science and Technology（浙江科技学院信息与电子工程学院）；Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）
The University of Adelaide（阿德莱德大学）；Northwestern Polytechnical University（西北工业大学）
Deep neural networks have achieved remarkable success in single image super-resolution (SISR). The computing and memory requirements of these methods have hindered their application to broad classes of real devices with limited computing power, however. One approach to this problem has been lightweight network architectures that bal- ance the super-resolution performance and the computation burden. In this study, we revisit this problem from an orthog- onal view, and propose a novel learning strategy to maxi- mize the pixel-wise fitting capacity of a given lightweight network architecture. Con…
Recently, several deep learning-based image super-resolution methods have been developed by stacking massive numbers of layers. However, this leads too large model sizes and high computational complexities, thus some recursive parameter-sharing methods have been also proposed. Nevertheless, their designs do not properly utilize the potential of the recursive operation. In this paper, we propose a novel, lightweight, and efficient super-resolution method to maximize the usefulness of the recursive architecture, by introducing block state-based recursive network. By taking advantage of utilizing…
EchoSR: Efficient Context Harnessing for Lightweight Image Super-Resolution
College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou 325000, China（温州大学计算机科学与人工智能学院）；vivo BlueImage Lab, vivo Mobile Communication Co., Ltd, Shanghai 200100, China（vivo蓝影实验室，vivo移动通信有限公司，上海200100，中国）；College of Engineering and Computer Science, Australian National University, Canberra, Australia（工程与计算机科学学院，澳大利亚国立大学，堪培拉，澳大利亚）；The AI/Analytics Team, Nasdaq, St. John’s, Canada（AI/分析团队，纳斯达克，圣约翰，加拿大）
In recent years, single image super-resolution (SISR) methods using deep convolution neural network (CNN) have achieved impressive results. Thanks to the powerful representation capabilities of the deep networks, numerous previous ways can learn the complex non-linear mapping between low-resolution (LR) image patches and their high-resolution (HR) versions. However, excessive convolutions will limit the application of super-resolution technology in low computing power devices. Besides, super-resolution of any arbitrary scale factor is a critical issue in practical applications, which has not b…
In recent years, the use of large convolutional kernels has become popular in designing convolutional neural networks due to their ability to capture long-range dependencies and provide large receptive fields. However, the increase in kernel size also leads to a quadratic growth in the number of parameters, resulting in heavy computation and memory requirements. To address this challenge, we propose a neighborhood attention (NA) module that upgrades the standard convolution with a self-attention mechanism. The NA module efficiently extracts long-range dependencies in a sliding window pattern, …
Multinex: Lightweight Low-light Image Enhancement via Multi-prior Retinex
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；ETcTI, University Politehnica Timisoara（蒂米șoara工业大学ETcTI）；West University of Timisoara（蒂米șoara西大学）
Fujian Normal University（福建师范大学）；Jimei University（集美大学）；Fuzhou University（福州大学）；China Design Group Co Ltd（中设设计集团股份有限公司）
Recently, deep learning has been successfully applied to the single-image super-resolution (SISR) with remarkable performance. However, most existing methods focus on building a more complex network with a large number of layers, which can entail heavy computational costs and memory storage. To address this problem, we present a lightweight Self-Calibrated Efficient Transformer (SCET) network to solve this problem. The architecture of SCET mainly consists of the self-calibrated module and efficient transformer block, where the self-calibrated module adopts the pixel attention mechanism to extr…
Nanjing University of Posts and Telecommunications（南京邮电大学）；The Chinese University of Hong Kong（香港中文大学）；National Institute of Informatics（国立信息学研究所）
Single-image super-resolution (SISR) has achieved significant breakthroughs with the development of deep learning. However, these methods are difficult to be applied in real-world scenarios since they are inevitably accompanied by the problems of computational and memory costs caused by the complex operations. To solve this issue, we propose a Lightweight Bimodal Network (LBNet) for SISR. Specifically, an effective Symmetric CNN is designed for local feature extraction and coarse image reconstruction. Meanwhile, we propose a Recursive Transformer to fully learn the long-term dependence of imag…
Deep learning-based algorithms have greatly improved the performance of remote sensing image (RSI) super-resolution (SR). However, increasing network depth and parameters cause a huge burden of computing and storage. Directly reducing the depth or width of existing models results in a large performance drop. We observe that the SR difficulty of different regions in an RSI varies greatly, and existing methods use the same deep network to process all regions in an image, resulting in a waste of computing resources. In addition, existing SR methods generally predefine integer scale factors and ca…
UCAN: Unified Convolutional Attention Network for Expansive Receptive Fields in Lightweight Super-Resolution
Ho Chi Minh City Open University（胡志明市开放大学）；AI Tech Lab, Ho Chi Minh City University of Technology（胡志明市科技大学人工智能技术实验室）；Code Mely AI Research Team（Code Mely 人工智能研究团队）；Global College, Shanghai Jiao Tong University（上海交通大学全球学院）；Ha Noi University of Science and Technology（河内科技大学）；University of Manitoba（曼尼托巴大学）
Sun Yat-sen University（中山大学）；City University of Hong Kong（香港城市大学）；Shanghai Jiao Tong University（上海交通大学）；Northwestern University（西北大学）
Despite convolutional network-based methods have boosted the performance of single image super-resolution (SISR), the huge computation costs restrict their practical applicability. In this paper, we develop a computation efficient yet accurate network based on the proposed attentive auxiliary features (A$^2$F) for SISR. Firstly, to explore the features from the bottom layers, the auxiliary feature from all the previous layers are projected into a common space. Then, to better utilize these projected auxiliary features and filter the redundant information, the channel attention is employed to s…
FreeTransformSR: Efficient Lightweight Image Super-Resolution via Free Low-Rank Learnable Transform
Chinese Academy of Sciences Changchun Institute of Optics Fine Mechanics and Physics（中国科学院长春光学精密机械与物理研究所）；University of the Chinese Academy of Sciences（中国科学院大学）
Tsinghua University（清华大学）；China Three Gorges University（三峡大学）
Single image super-resolution(SISR) has witnessed great progress as convolutional neural network(CNN) gets deeper and wider. However, enormous parameters hinder its application to real world problems. In this letter, We propose a lightweight feature fusion network (LFFN) that can fully explore multi-scale contextual information and greatly reduce network parameters while maximizing SISR results. LFFN is built on spindle blocks and a softmax feature fusion module (SFFM). Specifically, a spindle block is composed of a dimension extension unit, a feature exploration unit and a feature refinement …
The Ohio State University（俄亥俄州立大学）；The University of Arizona（亚利桑那大学）
The rapid and seemingly endless expansion of COVID-19 can be traced back to the inefficiency and shortage of testing kits that offer accurate results in a timely manner. An emerging popular technique, which adopts improvements made in mobile ultrasound technology, allows for healthcare professionals to conduct rapid screenings on a large scale. We present an image-based solution that aims at automating the testing process which allows for rapid mass testing to be conducted with or without a trained medical professional that can be applied to rural environments and third world countries. Our co…
