Current video deblurring methods have limitations in recovering high-frequency information since the regression losses are conservative with high-frequency details. Since Diffusion Models (DMs) have strong capabilities in generating high-frequency details, we consider introducing DMs into the video deblurring task. However, we found that directly applying DMs to the video deblurring task has the following problems: (1) DMs require many iteration steps to generate videos from Gaussian noise, which consumes many computational resources. (2) DMs are easily misled by the blurry artifacts in the video, resulting in irrational content and distortion of the deblurred video. To address the above issues, we propose a novel video deblurring framework VD-Diff that integrates the diffusion model into the Wavelet-Aware Dynamic Transformer (WADT). Specifically, we perform the diffusion model in a highly compact latent space to generate prior features containing high-frequency information that conforms to the ground truth distribution. We design the WADT to preserve and recover the low-frequency information in the video while utilizing the high-frequency information generated by the diffusion model. Extensive experiments show that our proposed VD-Diff outperforms SOTA methods on GoPro, DVD, BSD, and Real-World Video datasets.
In this paper, we examine the problem of real-world image deblurring and take into account two key factors for improving the performance of the deep image deblurring model, namely, training data synthesis and network architecture design. Deblurring models trained on existing synthetic datasets perform poorly on real blurry images due to domain shift. To reduce the domain gap between synthetic and real domains, we propose a novel realistic blur synthesis pipeline to simulate the camera imaging process. As a result of our proposed synthesis method, existing deblurring models could be made more r…
Learning Deblurring Texture Prior from Unpaired Data with Diffusion Model
Xi’an Jiaotong University（西安交通大学）；Wuhan University（武汉大学）；Insta360（影石）；Nanjing University of Science and Technology（南京理工大学）；University of California, Merced（加利福尼亚大学默塞德分校）
WaveDM: Wavelet-Based Diffusion Models for Image Restoration
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；The University of Hong Kong（香港大学）
This work deals with unsupervised image deblurring. We present a new deblurring procedure on images provided by low-resolution synthetic aperture radar (SAR) or simply by multimedia in presence of multiplicative (speckle) or additive noise, respectively. The method we propose is defined as a two-step process. First, we use an original technique for noise reduction in wavelet domain. Then, the learning of a Kohonen self-organizing map (SOM) is performed directly on the denoised image to take out it the blur. This technique has been successfully applied to real SAR images, and the simulation res…
University of Electronic Science and Technology of China（电子科技大学）；Cornell University（康奈尔大学）
The wavelet frame systems have been playing an active role in image restoration and many other image processing fields over the past decades, owing to the good capability of sparsely approximating piece-wise smooth functions such as images. In this paper, we propose a novel wavelet frame based sparse recovery model called \textit{Support Driven Sparse Regularization} (SDSR) for image deblurring, where the partial support information of frame coefficients is attained via a self-learning strategy and exploited via the proposed truncated $\ell_0$ regularization. Moreover, the state-of-the-art ima…
Coarse-to-fine strategies have been extensively used for the architecture design of single image deblurring networks. Conventional methods typically stack sub-networks with multi-scale input images and gradually improve sharpness of images from the bottom sub-network to the top sub-network, yielding inevitably high computational costs. Toward a fast and accurate deblurring network design, we revisit the coarse-to-fine strategy and present a multi-input multi-output U-net (MIMO-UNet). The MIMO-UNet has three distinct features. First, the single encoder of the MIMO-UNet takes multi-scale input i…
Fujian Normal University（福建师范大学）；JOYY AI GROUP（欢聚集团AI团队）；China Design Group Co Ltd（中设集团有限公司）
Image deblurring is a classical computer vision problem that aims to recover a sharp image from a blurred image. To solve this problem, existing methods apply the Encode-Decode architecture to design the complex networks to make a good performance. However, most of these methods use repeated up-sampling and down-sampling structures to expand the receptive field, which results in texture information loss during the sampling process and some of them design the multiple stages that lead to difficulties with convergence. Therefore, our model uses dilated convolution to enable the obtainment of the…
Multi-scale Frequency Enhancement Network for Blind Image Deblurring
The Institute of Systems Engineering, Academy of Military Science（军事科学院系统工程研究所）；School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机学院）；Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence, Jiangnan University（江南大学江苏省模式识别与计算智能工程实验室）；College of Artificial Intelligence, China University of Petroleum(beijing)（中国石油大学（北京）人工智能学院）；Beijing Key Laboratory of Petroleum Data Mining, China University of Petroleum(beijing)（中国石油大学（北京）北京市石油数据挖掘重点实验室）
Ivan-ISTD: Rethinking Cross-domain Heteroscedastic Noise Perturbations in Infrared Small Target Detection
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；TikTok, ByteDance Inc（字节跳动）；School of Data and Computer Science, Sun Yat-sen University（中山大学数据与计算机科学学院）
DocPure: Prompt-Free Unified Document Restoration via Degradation-Aware Structure-Guided Wavelet Modulation
College of Computer Science and Artificial Intelligence, Wenzhou University（温州大学计算机与人工智能学院）；AI Analytics Team, Nasdaq（纳斯达克人工智能分析团队）；vivo Mobile Communication Co., Ltd（维沃移动通信有限公司）；College of Engineering and Computer Science, The Australian National University（澳大利亚国立大学工程与计算机科学学院）
