图像超分辨率（SR）已经见证了从 CNN 到 transformer 架构的大量神经网络设计。然而，当前主流 SR 模型存在令人难以承受的内存占用和密集计算问题，这限制了其在边缘设备上的进一步部署。本工作研究了面向超分辨率的网络剪枝潜力，以利用现成的网络设计并降低底层计算开销。将剪枝方法应用于 SR 仍存在两个主要挑战。首先，被广泛使用的滤波器剪枝技术粒度有限，并且对多样化网络结构的适应性受限。其次，现有剪枝方法通常在预训练网络上运行以确定稀疏结构，难以摆脱传统 SR 范式中的稠密模型训练。为应对这些挑战，我们采用非结构化剪枝，并直接从零开始训练稀疏模型。具体而言，我们提出了一种新颖的迭代软收缩百分比（Iterative Soft Shrinkage-Percentage，ISS-P）方法，在每次迭代中优化随机初始化网络的稀疏结构，并按照与幅值尺度成比例的小量即时调整不重要的权重。我们观察到，所提出的 ISS-P 能够动态学习适应优化过程的稀疏结构，并通过产生更具正则化的梯度通量来保持稀疏模型的可训练性。在基准数据集上的实验证明了所提出 ISS-P 在多种网络架构上的有效性。代码可在 https://github.com/Jiamian-Wang/Iterative-Soft-Shrinkage-SR 获取。
英文摘要：
Image super-resolution (SR) has witnessed extensive neural network designs from CNN to transformer architectures. However, prevailing SR models suffer from prohibitive memory footprint and intensive computations, which limits further deployment on edge devices. This work investigates the potential of network pruning for super-resolution to take advantage of off-the-shelf network designs and reduce the underlying computational overhead. Two main challenges remain in applying pruning methods for SR. First, the widely-used filter pruning technique reflects limited granularity and restricted adaptability to diverse network structures. Second, existing pruning methods generally operate upon a pre-trained network for the sparse structure determination, hard to get rid of dense model training in the traditional SR paradigm. To address these challenges, we adopt unstructured pruning with sparse models directly trained from scratch. Specifically, we propose a novel Iterative Soft Shrinkage-Percentage (ISS-P) method by optimizing the sparse structure of a randomly initialized network at each iteration and tweaking unimportant weights with a small amount proportional to the magnitude scale on-the-fly. We observe that the proposed ISS-P can dynamically learn sparse structures adapting to the optimization process and preserve the sparse model's trainability by yielding a more regularized gradient throughput. Experiments on benchmark datasets demonstrate the effectiveness of the proposed ISS-P over diverse network architectures. Code is available at https://github.com/Jiamian-Wang/Iterative-Soft-Shrinkage-SR
Entropy Regularized Iterative Weighted Shrinkage-Thresholding Algorithm (ERIWSTA): An Application to CT Image Restoration
College of Medicine and Biological Information Engineering, Northeastern University（东北大学医学与生物信息工程学院）；Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education（教育部医学影像智能计算重点实验室）；Department of Electrical and Computer Engineering, Stevens Institute of Technology（史蒂文斯理工学院电气与计算机工程系）
Closed-Form Nonlocal Shrinkage for Multiplicative Image Denoising and SAR Despeckling
School of Electronic Engineering, Xidian University（西安电子科技大学电子工程学院）；Kunshan Innovation Institute of Xidian University（西安电子科技大学昆山创新研究院）；University of Montenegro（黑山大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）；Macquarie University（麦考瑞大学）
A Multi-scale Generalized Shrinkage Threshold Network for Image Blind Deblurring in Remote Sensing
School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）；School of Automation and Electronic Information, Xiangtan University（湘潭大学自动化与电子信息学院）
