Blind super-resolution methods based on stable diffusion showcase formidable generative capabilities in reconstructing clear high-resolution images with intricate details from low-resolution inputs. However, their practical applicability is often hampered by poor efficiency, stemming from the requirement of thousands or hundreds of sampling steps. Inspired by the efficient adversarial diffusion distillation (ADD), we design~\name~to address this issue by incorporating the ideas of both distillation and ControlNet. Specifically, we first propose a prediction-based self-refinement strategy to provide high-frequency information in the student model output with marginal additional time cost. Furthermore, we refine the training process by employing HR images, rather than LR images, to regulate the teacher model, providing a more robust constraint for distillation. Second, we introduce a timestep-adaptive ADD to address the perception-distortion imbalance problem introduced by original ADD. Extensive experiments demonstrate our~\name~generates better restoration results, while achieving faster speed than previous SD-based state-of-the-art models (e.g., $7$$\times$ faster than SeeSR).
Tsinghua University（清华大学）；ETH Zürich（苏黎世联邦理工学院）；ByteDance Inc（字节跳动公司）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Würzburg（维尔茨堡大学）
Blind image super-resolution (Blind-SR) aims to recover a high-resolution (HR) image from its corresponding low-resolution (LR) input image with unknown degradations. Most of the existing works design an explicit degradation estimator for each degradation to guide SR. However, it is infeasible to provide concrete labels of multiple degradation combinations (e.g., blur, noise, jpeg compression) to supervise the degradation estimator training. In addition, these special designs for certain degradation, such as blur, impedes the models from being generalized to handle different degradations. To t…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；University of Rochester（罗切斯特大学）
An important scenario for image quality assessment (IQA) is to evaluate image restoration (IR) algorithms. The state-of-the-art approaches adopt a full-reference paradigm that compares restored images with their corresponding pristine-quality images. However, pristine-quality images are usually unavailable in blind image restoration tasks and real-world scenarios. In this paper, we propose a practical solution named degraded-reference IQA (DR-IQA), which exploits the inputs of IR models, degraded images, as references. Specifically, we extract reference information from degraded images by dist…
University of Würzburg（维尔茨堡大学）；Institute of Computer Science（计算机科学研究所）
Generative models for image restoration, enhancement, and generation have significantly improved the quality of the generated images. Surprisingly, these models produce more pleasant images to the human eye than other methods, yet, they may get a lower perceptual quality score using traditional perceptual quality metrics such as PSNR or SSIM. Therefore, it is necessary to develop a quantitative metric to reflect the performance of new algorithms, which should be well-aligned with the person's mean opinion score (MOS). Learning-based approaches for perceptual image quality assessment (IQA) usua…
The Chinese University of Hong Kong（香港中文大学）；East China Normal University（华东师范大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Single image denoising (SID) has achieved significant breakthroughs with the development of deep learning. However, the proposed methods are often accompanied by plenty of parameters, which greatly limits their application scenarios. Different from previous works that blindly increase the depth of the network, we explore the degradation mechanism of the noisy image and propose a lightweight Multiple Degradation and Reconstruction Network (MDRN) to progressively remove noise. Meanwhile, we propose two novel Heterogeneous Knowledge Distillation Strategies (HMDS) to enable MDRN to learn richer an…
LightBSR: Towards Lightweight Blind Super-Resolution via Discriminative Implicit Degradation Representation Learning
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）；North China Electric Power University（华北电力大学）；Beijing Jiaotong University（北京交通大学）；YunQue AGI, Hangzhou, China（云阙AGI，杭州，中国）
Harbin Institute of Technology（哈尔滨工业大学）；University of Tianjin（天津大学）；Peng Cheng Lab（鹏城实验室）
We investigate the task of learning blind image denoising networks from an unpaired set of clean and noisy images. Such problem setting generally is practical and valuable considering that it is feasible to collect unpaired noisy and clean images in most real-world applications. And we further assume that the noise can be signal dependent but is spatially uncorrelated. In order to facilitate unpaired learning of denoising network, this paper presents a two-stage scheme by incorporating self-supervised learning and knowledge distillation. For self-supervised learning, we suggest a dilated blind…
Leveraging Classic Deconvolution and Feature Extraction in Zero-Shot Image Restoration
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI, Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹人工智能平台，亥姆霍兹慕尼黑中心——德国环境健康研究中心）；Georg-August-University Göttingen（哥廷根乔治-奥古斯特大学）；GEOMAR Helmholtz Centre for Ocean Research Kiel（基尔亥姆霍兹海洋研究中心）
Although diffusion model has shown great potential for generating higher quality images than GANs, slow sampling speed hinders its wide application in practice. Progressive distillation is thus proposed for fast sampling by progressively aligning output images of $N$-step teacher sampler with $N/2$-step student sampler. In this paper, we argue that this distillation-based accelerating method can be further improved, especially for few-step samplers, with our proposed \textbf{C}lassifier-based \textbf{F}eature \textbf{D}istillation (CFD). Instead of aligning output images, we distill teacher's …
Flordia International University（佛罗里达国际大学）；Northeastern University（东北大学）
Image compression-based approaches for defending against the adversarial-example attacks, which threaten the safety use of deep neural networks (DNN), have been investigated recently. However, prior works mainly rely on directly tuning parameters like compression rate, to blindly reduce image features, thereby lacking guarantee on both defense efficiency (i.e. accuracy of polluted images) and classification accuracy of benign images, after applying defense methods. To overcome these limitations, we propose a JPEG-based defensive compression framework, namely "feature distillation", to effectiv…
Catch-Up Distillation: You Only Need to Train Once for Accelerating Sampling
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Southeast University（东南大学）；Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）
We propose a method to infer a dense depth map from a single image, its calibration, and the associated sparse point cloud. In order to leverage existing models (teachers) that produce putative depth maps, we propose an adaptive knowledge distillation approach that yields a positive congruent training process, wherein a student model avoids learning the error modes of the teachers. In the absence of ground truth for model selection and training, our method, termed Monitored Distillation, allows a student to exploit a blind ensemble of teachers by selectively learning from predictions that best…
Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping
South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；Beijing Institute of Technology（北京理工大学）；The University of Sydney（悉尼大学）
University of Maryland（马里兰大学）；Georgia Institute of Technology（佐治亚理工学院）；Honda Research Institute（本田研究院）
There is an increasing interest on accelerating neural networks for real-time applications. We study the student-teacher strategy, in which a small and fast student network is trained with the auxiliary information learned from a large and accurate teacher network. We propose to use conditional adversarial networks to learn the loss function to transfer knowledge from teacher to student. The proposed method is particularly effective for relatively small student networks. Moreover, experimental results show the effect of network size when the modern networks are used as student. We empirically …
Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning
Chandra Family Department of Electrical and Computer Engineering（查克拉家族电气与计算机工程系）；Department of Imaging Physics（成像物理系）；Dell Medical School Department of Neurology（德莱尔医学院神经学系）；Dell Medical School Department of Neurosurgery（德莱尔医学院神经外科系）；Dell Medical School Department of Diagnostic Medicine（德莱尔医学院诊断医学系）；Oden Institute for Computational Engineering and Sciences（奥登计算工程与科学研究院）
NeRD: Neuro-Symbolic Rule Distillation for Efficient Ontology-Grounded Chain-of-Thought in Medical Image Diagnosis
Department of Data Science & AI, Faculty of Information Technology, Monash University（莫纳什大学信息技术学院数据科学与人工智能系）；AIM for Health Lab, Faculty of Information Technology, Monash University（莫纳什大学信息技术学院AIM健康实验室）；Faculty of Engineering, Monash University（莫纳什大学工程学院）；Faculty of Medicine, The Chinese University of Hong Kong（香港中文大学医学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）
Adversarial Concept Distillation for One-Step Diffusion Personalization
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；VCIP, CS, Nankai University（南开大学计算机学院视觉计算与智能感知实验室）；Program of Computer Science, City University of Hong Kong (Dongguan)（香港城市大学（东莞）计算机科学项目）；City University of Hong Kong（香港城市大学）
A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategies
Tsinghua University（清华大学）；Nankai University（南开大学）；Kuaishou Technology（快手科技）；Tianjin University of Finance and Economics（天津财经大学）；Imperial College London（伦敦帝国学院）
Stochastic Transition-Map Distillation for Fast Probabilistic Inference
Department of Aerospace Engineering, Georgia Institute of Technology（佐治亚理工学院航空航天工程系）；Department of Computer Science, Georgia Institute of Technology（佐治亚理工学院计算机科学系）；Department of Mathematics, Florida State University（佛罗里达州立大学数学系）
Distribution Backtracking Builds A Faster Convergence Trajectory for Diffusion Distillation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；Peking University（北京大学）；Alibaba Group（阿里巴巴集团）
