Image restoration tasks have achieved tremendous performance improvements with the rapid advancement of deep neural networks. However, most prevalent deep learning models perform inference statically, ignoring that different images have varying restoration difficulties and lightly degraded images can be well restored by slimmer subnetworks. To this end, we propose a new solution pipeline dubbed ClassPruning that utilizes networks with different capabilities to process images with varying restoration difficulties. In particular, we use a lightweight classifier to identify the image restoration difficulty, and then the sparse subnetworks with different capabilities can be sampled based on predicted difficulty by performing dynamic N:M fine-grained structured pruning on base restoration networks. We further propose a novel training strategy along with two additional loss terms to stabilize training and improve performance. Experiments demonstrate that ClassPruning can help existing methods save approximately 40% FLOPs while maintaining performance.
Diverse user preferences over images have recently led to a great amount of interest in controlling the imagery effects for image restoration tasks. However, existing methods require separate inference through the entire network per each output, which hinders users from readily comparing multiple imagery effects due to long latency. To this end, we propose a novel framework based on a neural architecture search technique that enables efficient generation of multiple imagery effects via two stages of pruning: task-agnostic and task-specific pruning. Specifically, task-specific pruning learns to…
Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Life Sciences, Tsinghua University（清华大学生命科学学院）；Intelligent Science & Technology Academy of CASIC（中国科学院 CASIC 智能科学与技术学院）
Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Yunnan, China（昆明理工大学信息工程与自动化学院，云南，中国）；School of Mathematics and Statistics, Yunnan University, Yunnan, China（云南大学数学与统计学院，云南，中国）；Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, China（深圳视觉对象检测与识别重点实验室，哈尔滨工业大学，深圳，中国）
Tsinghua University（清华大学）；Shanghai AI Laboratory（上海人工智能实验室）；ETH Zürich（苏黎世联邦理工学院）；ByteDance Inc（字节跳动公司）；University of Texas at Dallas（德克萨斯大学达拉斯分校）
The high computational costs of video super-resolution (VSR) models hinder their deployment on resource-limited devices, (e.g., smartphones and drones). Existing VSR models contain considerable redundant filters, which drag down the inference efficiency. To prune these unimportant filters, we develop a structured pruning scheme called Structured Sparsity Learning (SSL) according to the properties of VSR. In SSL, we design pruning schemes for several key components in VSR models, including residual blocks, recurrent networks, and upsampling networks. Specifically, we develop a Residual Sparsity…
Deep net architectures have constantly evolved over the past few years, leading to significant advancements in a wide array of computer vision tasks. However, besides high accuracy, many applications also require a low computational load and limited memory footprint. To date, efficiency has typically been achieved either by architectural choices at the macro level (e.g. using skip connections or pruning techniques) or modifications at the level of the individual layers (e.g. using depth-wise convolutions or channel shuffle operations). Interestingly, much less attention has been devoted to the…
Motion-Aware Adaptive Pixel Pruning for Efficient Local Motion Deblurring
Harbin Institute of Technology \& City University of Hong Kong；School of Computer Science；Technology, Harbin Institute of Technology；Tianjin University \& Low-Altitude Intelligence Lab, Xiong'an National Innovation Center Technology Co., Ltd；College of Intelligence；Computing, Tianjin University
Text-guided Image Restoration and Semantic Enhancement for Text-to-Image Person Retrieval
School of Artificial Intelligence（人工智能学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Key Laboratory of Network System and Network Culture（网络系统与网络文化北京市重点实验室）
