Beyond pixels and regions: A non local patch means (NLPM) method for content-level restoration, enhancement, and reconstruction of degraded document images
Reza Farrahi Moghaddam, Mohamed Cheriet
备注：
This paper has been withdrawn by the author to avoid duplication on the DBLP bibliography
A patch-based non-local restoration and reconstruction method for preprocessing degraded document images is introduced. The method collects relative data from the whole input image, while the image data are first represented by a content-level descriptor based on patches. This patch-equivalent representation of the input image is then corrected based on similar patches identified using a modified genetic algorithm (GA) resulting in a low computational load. The corrected patch-equivalent is then converted to the output restored image. The fact that the method uses the patches at the content level allows it to incorporate high-level restoration in an objective and self-sufficient way. The method has been applied to several degraded document images, including the DIBCO'09 contest dataset with promising results.
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Learning-based methods especially with convolutional neural networks (CNN) are continuously showing superior performance in computer vision applications, ranging from image classification to restoration. For image classification, most existing works focus on very clean images such as images in Caltech-256 and ImageNet datasets. However, in most realistic scenarios, the acquired images may suffer from degradation. One important and interesting problem is to combine image classification and restoration tasks to improve the performance of CNN-based classification networks on degraded images. In t…
Tokyo Institute of Technology（东京工业大学）；Kadinche Corporation（Kadinche公司）；National Institute of Advanced Industrial Science and Technology（国立先进工业科学技术研究所）
Image restoration from a single image degradation type, such as blurring, hazing, random noise, and compression has been investigated for decades. However, image degradations in practice are often a mixture of several types of degradation. Such compositional degradations complicate restoration because they require the differentiation of different degradation types and levels. In this paper, we propose a convolutional neural network (CNN) model for estimating the degradation properties of a given degraded image. Furthermore, we introduce an image restoration CNN model that adopts the estimated …
Image restoration is the task of recovering a clean image from a degraded version. In most cases, the degradation is spatially varying, and it requires the restoration network to both localize and restore the affected regions. In this paper, we present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images affected by practically occurring artifacts such as blur, rain-streaks. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration, unlike existing methods which directly learn a …
Degraded Infrared Small Object Detection via Degradation-Adapted Physics-Guided Restoration
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）；School of Ocean Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院海洋工程学院）
Harbin Institute of Technology（哈尔滨工业大学）；Sun Yat-sen University（中山大学）；University of Kentucky（肯塔基大学）
This paper studies the problem of blind face restoration from an unconstrained blurry, noisy, low-resolution, or compressed image (i.e., degraded observation). For better recovery of fine facial details, we modify the problem setting by taking both the degraded observation and a high-quality guided image of the same identity as input to our guided face restoration network (GFRNet). However, the degraded observation and guided image generally are different in pose, illumination and expression, thereby making plain CNNs (e.g., U-Net) fail to recover fine and identity-aware facial details. To tac…
All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model
School of Computing, Australian National University（澳大利亚国立大学计算学院）；School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）
Removing adverse weather conditions like rain, fog, and snow from images is an important problem in many applications. Most methods proposed in the literature have been designed to deal with just removing one type of degradation. Recently, a CNN-based method using neural architecture search (All-in-One) was proposed to remove all the weather conditions at once. However, it has a large number of parameters as it uses multiple encoders to cater to each weather removal task and still has scope for improvement in its performance. In this work, we focus on developing an efficient solution for the a…
Infrared Image Deturbulence Restoration Using Degradation Parameter-Assisted Wide & Deep Learning
Beihang University（北京航空航天大学）；Image Processing Center（图像处理中心）；The State Key Laboratory of Virtual Reality Technology and Systems（虚拟现实技术与系统国家重点实验室）；Beijing Advanced Innovation Center for Biomedical Engineering（北京生物医学工程高精尖创新中心）
In this paper, an algorithm is proposed for Image Restoration. Such algorithm is different from the traditional approaches in this area, by utilizing priors that are learned from similar images. Original images and their degraded versions by the known degradation operators are utilized for designing the Quantization. The code vectors are designed using the blurred images. For each such vector, the high frequency information obtained from the original images is also available. During restoration, the high frequency information of a given degraded image is estimated from its low frequency inform…
Shanghai Jiao Tong University（上海交通大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Shanghai AI Laboratory（上海人工智能实验室）
Blind face restoration usually synthesizes degraded low-quality data with a pre-defined degradation model for training, while more complex cases could happen in the real world. This gap between the assumed and actual degradation hurts the restoration performance where artifacts are often observed in the output. However, it is expensive and infeasible to include every type of degradation to cover real-world cases in the training data. To tackle this robustness issue, we propose Diffusion-based Robust Degradation Remover (DR2) to first transform the degraded image to a coarse but degradation-inv…
Data61-Commonwealth Scientific Industrial Research Organization (CSIRO)（Data61-联邦科学与工业研究组织）；Australian National University（澳大利亚国立大学）
Deep convolutional neural networks perform better on images containing spatially invariant degradations, also known as synthetic degradations; however, their performance is limited on real-degraded photographs and requires multiple-stage network modeling. To advance the practicability of restoration algorithms, this paper proposes a novel single-stage blind real image restoration network (R$^2$Net) by employing a modular architecture. We use a residual on the residual structure to ease the flow of low-frequency information and apply feature attention to exploit the channel dependencies. Furthe…
Face2Scene: Using Facial Degradation as an Oracle for Diffusion-Based Scene Restoration
University of Toronto（多伦多大学）；Vector Institute（向量研究所）；AI Center–Toronto, Samsung Electronics（三星电子多伦多AI中心）；University Health Network（大学健康网络）；York University（约克大学）
MeInTime: Bridging Age Gap in Identity-Preserving Face Restoration
Beijing University of Posts and Telecommunications（北京邮电大学）；Department of Computer Science and Technology, Institute for AI, Tsinghua University（清华大学计算机科学与技术系，人工智能研究院）；Minzu University of China（民族大学）；Xiaomi Corporation（小米公司）
Jilin University（吉林大学）；University of Trento（特伦托大学）；ETH Zurich（苏黎世联邦理工学院）
Constructing high-quality character image datasets is challenging because real-world images are often affected by image degradation. There are limitations when applying current image restoration methods to such real-world character images, since (i) the categories of noise in character images are different from those in general images; (ii) real-world character images usually contain more complex image degradation, e.g., mixed noise at different noise levels. To address these problems, we propose a real-world character restoration network (RCRN) to effectively restore degraded character images…
In recent years, convolutional neural network-based single image adverse weather removal methods have achieved significant performance improvements on many benchmark datasets. However, these methods require large amounts of clean-weather degraded image pairs for training, which is often difficult to obtain in practice. Although various weather degradation synthesis methods exist in the literature, the use of synthetically generated weather degraded images often results in sub-optimal performance on the real weather degraded images due to the domain gap between synthetic and real-world images. …
Michigan State University（密歇根州立大学）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
We present a general learning-based solution for restoring images suffering from spatially-varying degradations. Prior approaches are typically degradation-specific and employ the same processing across different images and different pixels within. However, we hypothesize that such spatially rigid processing is suboptimal for simultaneously restoring the degraded pixels as well as reconstructing the clean regions of the image. To overcome this limitation, we propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts computation to difficult regio…
Under-display camera (UDC) is a novel technology that can make digital imaging experience in handheld devices seamless by providing large screen-to-body ratio. UDC images are severely degraded owing to their positioning under a display screen. This work addresses the restoration of images degraded as a result of UDC imaging. Two different networks are proposed for the restoration of images taken with two types of UDC technologies. The first method uses a pyramidal dilated convolution within a wavelet decomposed convolutional neural network for pentile-organic LED (P-OLED) based display system.…
Blind face restoration is a highly ill-posed problem that often requires auxiliary guidance to 1) improve the mapping from degraded inputs to desired outputs, or 2) complement high-quality details lost in the inputs. In this paper, we demonstrate that a learned discrete codebook prior in a small proxy space largely reduces the uncertainty and ambiguity of restoration mapping by casting blind face restoration as a code prediction task, while providing rich visual atoms for generating high-quality faces. Under this paradigm, we propose a Transformer-based prediction network, named CodeFormer, to…
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算学系）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Peng Cheng Lab, Shenzhen（鹏城实验室）
Recent reference-based face restoration methods have received considerable attention due to their great capability in recovering high-frequency details on real low-quality images. However, most of these methods require a high-quality reference image of the same identity, making them only applicable in limited scenes. To address this issue, this paper suggests a deep face dictionary network (termed as DFDNet) to guide the restoration process of degraded observations. To begin with, we use K-means to generate deep dictionaries for perceptually significant face components (\ie, left/right eyes, n…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；University of Rochester（罗切斯特大学）
An important scenario for image quality assessment (IQA) is to evaluate image restoration (IR) algorithms. The state-of-the-art approaches adopt a full-reference paradigm that compares restored images with their corresponding pristine-quality images. However, pristine-quality images are usually unavailable in blind image restoration tasks and real-world scenarios. In this paper, we propose a practical solution named degraded-reference IQA (DR-IQA), which exploits the inputs of IR models, degraded images, as references. Specifically, we extract reference information from degraded images by dist…
Existing approaches for restoring weather-degraded images follow a fully-supervised paradigm and they require paired data for training. However, collecting paired data for weather degradations is extremely challenging, and existing methods end up training on synthetic data. To overcome this issue, we describe an approach for supervising deep networks that are based on CycleGAN, thereby enabling the use of unlabeled real-world data for training. Specifically, we introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. These new …
Tsinghua University（清华大学）；Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Science（中国科学院深圳先进技术研究院）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；University College London（伦敦大学学院）
Most state-of-the-art semantic segmentation approaches only achieve high accuracy in good conditions. In practically-common but less-discussed adverse environmental conditions, their performance can decrease enormously. Existing studies usually cast the handling of segmentation in adverse conditions as a separate post-processing step after signal restoration, making the segmentation performance largely depend on the quality of restoration. In this paper, we propose a novel deep-learning framework to tackle semantic segmentation and image restoration in adverse environmental conditions in a hol…
TDiR: Transformer based Diffusion for Image Restoration Tasks
Department of Computer Vision（计算机视觉系）；FutureDataMinds；Electrical Engineering Dept（电气工程系）；King Fahd University of Petroleum and Minerals（国王法赫德石油和矿产大学）；Department of Computer Science and Software Engineering（计算机科学与软件工程系）；University of Western Australia（西澳大学）；School of Computing（计算学院）
Indian Institute of Technology, Patna（印度理工学院巴特那分校）；Indian Statistical Institute, Kolkata（印度统计研究所加尔各答分所）
Image restoration is very crucial computer vision task. This paper describes two novel methods for the restoration of old degraded handwritten documents using deep neural network. In addition to that, a small-scale dataset of 26 heritage letters images is introduced. The ground truth data to train the desired network is generated semi automatically involving a pragmatic combination of color transformation, Gaussian mixture model based segmentation and shape correction by using mathematical morphological operators. In the first approach, a deep neural network has been used for text extraction f…
Tokyo Institute of Technology（东京工业大学）；National Institute of Advanced Industrial Science and Technology（日本产业技术综合研究所）
Classification for degraded images having various levels of degradation is very important in practical applications. This paper proposes a convolutional neural network to classify degraded images by using a restoration network and an ensemble learning. The results demonstrate that the proposed network can classify degraded images over various levels of degradation well. This paper also reveals how the image-quality of training data for a classification network affects the classification performance of degraded images.
Perceptual image restoration seeks for high-fidelity images that most likely degrade to given images. For better visual quality, previous work proposed to search for solutions within the natural image manifold, by exploiting the latent space of a generative model. However, the quality of generated images are only guaranteed when latent embedding lies close to the prior distribution. In this work, we propose to restrict the feasible region within the prior manifold. This is accomplished with a non-parametric metric for two distributions: the Maximum Mean Discrepancy (MMD). Moreover, we model th…
University of Science and Technology of China（中国科学技术大学）
Image deblurring has seen a great improvement with the development of deep neural networks. In practice, however, blurry images often suffer from additional degradations such as downscaling and compression. To address these challenges, we propose an Enhanced Deep Pyramid Network (EDPN) for blurry image restoration from multiple degradations, by fully exploiting the self- and cross-scale similarities in the degraded image.Specifically, we design two pyramid-based modules, i.e., the pyramid progressive transfer (PPT) module and the pyramid self-attention (PSA) module, as the main components of t…
The Chinese University of Hong Kong（香港中文大学）；SIAT CAS（中国科学院深圳先进技术研究院）
As a popular entertainment art form, manga enriches the line drawings details with bitonal screentones. However, manga resources over the Internet usually show screentone artifacts because of inappropriate scanning/rescaling resolution. In this paper, we propose an innovative two-stage method to restore quality bitonal manga from degraded ones. Our key observation is that the aliasing induced by downsampling bitonal screentones can be utilized as informative clues to infer the original resolution and screentones. First, we predict the target resolution from the degraded manga via the Scale Est…
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；The Hong Kong Polytechnic University（香港理工大学）
Blind face restoration (BFR) from severely degraded face images in the wild is a very challenging problem. Due to the high illness of the problem and the complex unknown degradation, directly training a deep neural network (DNN) usually cannot lead to acceptable results. Existing generative adversarial network (GAN) based methods can produce better results but tend to generate over-smoothed restorations. In this work, we propose a new method by first learning a GAN for high-quality face image generation and embedding it into a U-shaped DNN as a prior decoder, then fine-tuning the GAN prior emb…
Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration
School of Computer Science and Engineering, South China University of Technology（华南理工大学计算机科学与工程学院）；Institute of Super Robotics(Huangpu)（超级机器人（黄埔）研究院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Hong Kong University of Science and Technology（香港科技大学）；State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；Samsung Research China - Beijing (SRC-B)（三星中国研究院北京分部）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳校区）；SAMSUNG Electronics（三星电子）
