Deep LoRA-Unfolding Networks for Image Restoration
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（计算机科学与技术学院，哈尔滨工业大学（深圳））；Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（图像通信与网络工程院，上海交通大学）；School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学）；School of Electronics and Information Engineering, Harbin Institute of Technology (Shenzhen)（电子与信息工程学院，哈尔滨工业大学（深圳））
深度展开网络（DUNs）将传统的迭代优化算法与深度神经网络结合，形成多阶段框架，在图像恢复（IR）中取得了显著成就，例如光谱成像重建、压缩感知和超分辨率。它将迭代优化步骤展开为一系列依次连接的块。每个块包含一个梯度下降模块（GDM）和一个近端映射模块（PMM），从贝叶斯视角看，PMM相当于一个去噪器，对已知水平的高斯噪声进行操作。然而，现有 DUNs 存在两个关键限制：（i）它们的 PMMs 在各阶段具有相同的架构和去噪目标，忽略了不同噪声水平阶段适应的需要；（ii）结构上重复的块链导致严重的参数冗余和高内存消耗，阻碍了在大规模或资源受限场景中的部署。为了解决这些挑战，我们引入了通用的深度低秩适应（LoRA）展开网络用于图像恢复，称为 LoRun，通过压缩内存使用实现更高效的 DUN。LoRun 引入了一种新的范式，其中预训练的基去噪器在所有阶段共享，同时轻量级的阶段特定 LoRA 适配器被注入到 PMMs 中，以根据每个展开步骤的噪声水平动态调节去噪行为。这种设计将核心恢复能力与任务特定的适应分离，实现了对去噪强度的精确控制，而无需复制完整的网络参数，从而在 N 阶 DUN 中实现多达 N 倍的参数减少，性能与或优于现有方法。在三个 IR 任务上进行的大量实验验证了该方法的效率。
英文摘要：
Deep unfolding networks (DUNs), combining conventional iterative optimization algorithms and deep neural networks into a multi-stage framework, have achieved remarkable accomplishments in Image Restoration (IR), such as spectral imaging reconstruction, compressive sensing and super-resolution.It unfolds the iterative optimization steps into a stack of sequentially linked blocks.Each block consists of a Gradient Descent Module (GDM) and a Proximal Mapping Module (PMM) which is equivalent to a denoiser from a Bayesian perspective, operating on Gaussian noise with a known level.However, existing DUNs suffer from two critical limitations: (i) their PMMs share identical architectures and denoising objectives across stages, ignoring the need for stage-specific adaptation to varying noise levels; and (ii) their chain of structurally repetitive blocks results in severe parameter redundancy and high memory consumption, hindering deployment in large-scale or resource-constrained scenarios.To address these challenges, we introduce generalized Deep Low-rank Adaptation (LoRA) Unfolding Networks for image restoration, named LoRun, harmonizing denoising objectives and adapting different denoising levels between stages with compressed memory usage for more efficient DUN.LoRun introduces a novel paradigm where a single pretrained base denoiser is shared across all stages, while lightweight, stage-specific LoRA adapters are injected into the PMMs to dynamically modulate denoising behavior according to the noise level at each unfolding step.This design decouples the core restoration capability from task-specific adaptation, enabling precise control over denoising intensity without duplicating full network parameters and achieving up to $N$ times parameter reduction for an $N$-stage DUN with on-par or better performance.Extensive experiments conducted on three IR tasks validate the efficiency of our method.
Beijing International Center for Mathematical Research, Peking University（北京国际数学研究中心）；Center for Data Science, Peking University（北京大学数据科学中心）；Beijing Institute of Big Data Research（北京大数据研究院）
In this paper, we propose a new control framework called the moving endpoint control to restore images corrupted by different degradation levels in one model. The proposed control problem contains a restoration dynamics which is modeled by an RNN. The moving endpoint, which is essentially the terminal time of the associated dynamics, is determined by a policy network. We call the proposed model the dynamically unfolding recurrent restorer (DURR). Numerical experiments show that DURR is able to achieve state-of-the-art performances on blind image denoising and JPEG image deblocking. Furthermore…
University of Science and Technology of China（中国科学技术大学）
Deep learning provides a new avenue for image restoration, which demands a delicate balance between fine-grained details and high-level contextualized information during recovering the latent clear image. In practice, however, existing methods empirically construct encapsulated end-to-end mapping networks without deepening into the rationality, and neglect the intrinsic prior knowledge of restoration task. To solve the above problems, inspired by Taylor's Approximations, we unfold Taylor's Formula to construct a novel framework for image restoration. We find the main part and the derivative pa…
LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Peking University Shenzhen Graduate School（北京大学深圳研究生院）；Peng Cheng Laboratory（鹏城实验室）
Deep neural networks (DNN) have achieved great success in image restoration. However, most DNN methods are designed as a black box, lacking transparency and interpretability. Although some methods are proposed to combine traditional optimization algorithms with DNN, they usually demand pre-defined degradation processes or handcrafted assumptions, making it difficult to deal with complex and real-world applications. In this paper, we propose a Deep Generalized Unfolding Network (DGUNet) for image restoration. Concretely, without loss of interpretability, we integrate a gradient estimation strat…
DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration
School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；Shenzhen Research Institute of Northwestern Polytechnical University（西北工业大学深圳研究院）；State Key Laboratory of Internet of Things for Smart City, University of Macau（澳门大学智慧城市物联网国家重点实验室）
Variational Deep Unfolding with Mamba-Based Nonlocal Modeling for Underwater Image Enhancement
Institute of Applied Computing and Community Code (IAC3)（应用计算与社区代码研究所 (IAC3)）；Dept. of Mathematics and Computer Science, Universitat de les Illes Balears（巴利阿里群岛大学数学与计算机科学系）
Nanyang Technological University（南洋理工大学）；University of Science and Technology of China（中国科学技术大学）；Xi’an Jiaotong University（西安交通大学）；Nankai University（南开大学）；SenseTime Research（商汤科技研究院）
In this paper, orthogonal to the existing data and model studies, we instead resort our efforts to investigate the potential of loss function in a new perspective and present our belief ``Random Weights Networks can Be Acted as Loss Prior Constraint for Image Restoration''. Inspired by Functional theory, we provide several alternative solutions to implement our belief in the strict mathematical manifolds including Taylor's Unfolding Network, Invertible Neural Network, Central Difference Convolution and Zero-order Filtering as ``random weights network prototype'' with respect of the following f…
Tsinghua University（清华大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；Peng Cheng Laboratory（鹏城实验室）；ETH Zürich（苏黎世联邦理工学院）
A Progressive Image Restoration Network for High-order Degradation Imaging in Remote Sensing
School of Mathematics and Computational Science, Xiangtan University（湘潭大学数学与计算科学学院）；Hunan Key Laboratory for Computation and Simulation in Science and Engineering（湖南省计算与模拟科学工程重点实验室）；Key Laboratory for Intelligent Computing and Information Processing of the Ministry of Education（教育部智能计算与信息处理重点实验室）；National Center for Applied Mathematics in Hunan（湖南省应用数学中心）；Hunan International Scientific and Technological Innovation Cooperation Base of Computational Science（湖南省 computational science 国际科学和技术创新合作基地）；College of Mathematical Medicine, Zhejiang Normal University（浙江师范大学数学医学院）；School of Automation and Electronic Information, Xiangtan University（湘潭大学自动化与电子信息学院）
Pengcheng Laboratory（鹏城实验室）；Dalian University of Technology（大连理工大学）
Deep learning for image super-resolution (SR) has been investigated by numerous researchers in recent years. Most of the works concentrate on effective block designs and improve the network representation but lack interpretation. There are also iterative optimization-inspired networks for image SR, which take the solution step as a whole without giving an explicit optimization step. This paper proposes an unfolding iterative shrinkage thresholding algorithm (ISTA) inspired network for interpretable image SR. Specifically, we analyze the problem of image SR and propose a solution based on the I…
Reversible Unfolding Network for Concealed Visual Perception with Generative Refinement
Department of Biomedical Engineering, Duke University（生物医学工程系，杜克大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Nankai International Advanced Research Institute (SHENZHEN FUTIAN), Nankai University（南开国际先进研究院（深圳福田），南开大学）；College of Computer Science, Nankai University（计算机科学学院，南开大学）
