Generalized Rectifier Wavelet Covariance Models For Texture Synthesis
ENS, PSL University（巴黎高等师范学院，巴黎文理研究大学）；Université de Toulouse, INP, IRIT（图卢兹大学，图卢兹国立综合理工学院，图卢兹信息学研究所）；Collège de France（法兰西公学院）；Flatiron Institute（弗莱提荣研究所）
Antoine Brochard, Sixin Zhang, Stéphane Mallat
备注：
To be published as a conference paper at the International Conference on Learning Representations (ICLR) 2022
State-of-the-art maximum entropy models for texture synthesis are built from statistics relying on image representations defined by convolutional neural networks (CNN). Such representations capture rich structures in texture images, outperforming wavelet-based representations in this regard. However, conversely to neural networks, wavelets offer meaningful representations, as they are known to detect structures at multiple scales (e.g. edges) in images. In this work, we propose a family of statistics built upon non-linear wavelet based representations, that can be viewed as a particular instance of a one-layer CNN, using a generalized rectifier non-linearity. These statistics significantly improve the visual quality of previous classical wavelet-based models, and allow one to produce syntheses of similar quality to state-of-the-art models, on both gray-scale and color textures.
Review of wavelet-based unsupervised texture segmentation, advantage of adaptive wavelets
Image Processing Center, Beihang University（北京航空航天大学图像处理中心）；Centre pour les Mathématiques et Leurs Applications, Ecole Normale Supérieure of Paris Saclay（巴黎萨克雷高等师范学院数学及其应用中心）；Department of Mathematics and Statistics, San Diego State University（圣地亚哥州立大学数学与统计系）
Southeast University（东南大学）；INSERM（法国国家健康与医学研究院）；Université de Rennes 1（雷恩第一大学）
Texture plays an important role in many image analysis applications. In this paper, we give a performance evaluation of color texture classification by performing wavelet scattering network in various color spaces. Experimental results on the KTH_TIPS_COL database show that opponent RGB based wavelet scattering network outperforms other color spaces. Therefore, when dealing with the problem of color texture classification, opponent RGB based wavelet scattering network is recommended.
The University of Tokyo（东京大学）；Digital Frontier Inc.（数字前沿公司）
Texture classification is an important and challenging problem in many image processing applications. While convolutional neural networks (CNNs) achieved significant successes for image classification, texture classification remains a difficult problem since textures usually do not contain enough information regarding the shape of object. In image processing, texture classification has been traditionally studied well with spectral analyses which exploit repeated structures in many textures. Since CNNs process images as-is in the spatial domain whereas spectral analyses process images in the fr…
The scattering transform network (STN), which has a similar structure as that of a popular convolutional neural network except its use of predefined convolution filters and a small number of layers, can generates a robust representation of an input signal relative to small deformations. We propose a novel Monogenic Wavelet Scattering Network (MWSN) for 2D texture image classification through a cascade of monogenic wavelet filtering with nonlinear modulus and averaging operators by replacing the 2D Morlet wavelet filtering in the standard STN. Our MWSN can extract useful hierarchical and direct…
Velammal College of Engg and Tech（维拉马尔工程技术学院）；Thiagarajar College of Engg（蒂亚加拉贾工程技术学院）
The research work presented in this paper is to achieve the tissue classification and automatically diagnosis the abnormal tumor region present in Computed Tomography (CT) images using the wavelet based statistical texture analysis method. Comparative studies of texture analysis method are performed for the proposed wavelet based texture analysis method and Spatial Gray Level Dependence Method (SGLDM). Our proposed system consists of four phases i) Discrete Wavelet Decomposition (ii) Feature extraction (iii) Feature selection (iv) Analysis of extracted texture features by classifier. A wavelet…
New Horizon College of Engineering（新视野工程学院）；Govt. College of Engineering（政府工程学院）；University of Mysore（迈索尔大学）
In this paper, we propose a machine vision algorithm for automatically detecting defects in textures belonging to 16 out of 17 wallpaper groups using texture-periodicity and a family of Gabor wavelets. Input defective images are subjected to Gabor wavelet transformation in multi-scales and multi-orientations and a resultant image is obtained in L2 norm. The resultant image is split into several periodic blocks and energy of each block is used as a feature space to automatically identify defective and defect-free blocks using Ward's hierarchical clustering. Experiments on defective fabric image…
Texture analysis and classification remain as one of the biggest challenges for the field of computer vision and pattern recognition. On this matter, Gabor wavelets has proven to be a useful technique to characterize distinctive texture patterns. However, most of the approaches used to extract descriptors of the Gabor magnitude space usually fail in representing adequately the richness of detail present into a unique feature vector. In this paper, we propose a new method to enhance the Gabor wavelets process extracting a fractal signature of the magnitude spaces. Each signature is reduced usin…
École de Technologie Supérieure, Université du Québec（魁北克大学高等技术学院）
Breast cancer is a health problem that affects mainly the female population. An early detection increases the chances of effective treatment, improving the prognosis of the disease. In this regard, computational tools have been proposed to assist the specialist in interpreting the breast digital image exam, providing features for detecting and diagnosing tumors and cancerous cells. Nonetheless, detecting tumors with a high sensitivity rate and reducing the false positives rate is still challenging. Texture descriptors have been quite popular in medical image analysis, particularly in histopath…
Texture is the term used to characterize the surface of a given object or phenomenon and is an important feature used in image processing and pattern recognition. Our aim is to compare various Texture analyzing methods and compare the results based on time complexity and accuracy of classification. The project describes texture classification using Wavelet Transform and Co occurrence Matrix. Comparison of features of a sample texture with database of different textures is performed. In wavelet transform we use the Haar, Symlets and Daubechies wavelets. We find that, thee Haar wavelet proves to…
New Horizon College of Engineering（新视野工程学院）；Government Engineering College（政府工程学院）；University of Mysore（迈索尔大学）
Periodicity of a texture is one of the important visual characteristics and is often used as a measure for textural discrimination at the structural level. Knowledge about periodicity of a texture is very essential in the field of texture synthesis and texture compression and also in the design of frieze and wall papers. In this paper, we propose a method of periodicity extraction from noisy images based on superposition of distance matching function (DMF) and wavelet decomposition without de-noising the test images. Overall DMFs are subjected to single-level Haar wavelet decomposition to obta…
Adaptive Wavelet Filters as Practical Texture Feature Amplifiers for Parkinson's Disease Screening in OCT
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；State Key Laboratory of Ophthalmology, Optometry and Vision Science, Wenzhou Medical University（温州医科大学眼视光学与视觉科学国家重点实验室）；Coventry University（考文垂大学）；School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机学院）；School of Ophthalmology and Optometry, Wenzhou Medical University（温州医科大学眼视光学院）；Department of Electronic and Information Engineering, Changchun University（长春大学电子与信息工程学院）
With the heterogeneous nature of tissue texture, using a single resolution approach for optimum classification might not suffice. In contrast, a multiresolution wavelet packet analysis can decompose the input signal into a set of frequency subbands giving the opportunity to characterise the texture at the appropriate frequency channel. An adaptive best bases algorithm for optimal bases selection for meningioma histopathological images is proposed, via applying the fractal dimension (FD) as the bases selection criterion in a tree-structured manner. Thereby, the most significant subband that bet…
In this paper, we consider the problem of super-resolution recons-truction. This is a hot topic because super-resolution reconstruction has a wide range of applications in the medical field, remote sensing monitoring, and criminal investigation. Compared with traditional algorithms, the current super-resolution reconstruction algorithm based on deep learning greatly improves the clarity of reconstructed pictures. Existing work like Super-Resolution Using a Generative Adversarial Network (SRGAN) can effectively restore the texture details of the image. However, experimentally verified that the …
We provide a new model for texture synthesis based on a multiscale, multilayer feature extractor. Within the model, textures are represented by a set of statistics computed from ReLU wavelet coefficients at different layers, scales and orientations. A new image is synthesized by matching the target statistics via an iterative projection algorithm. We explain the necessity of the different types of pre-defined wavelet filters used in our model and the advantages of multilayer structures for image synthesis. We demonstrate the power of our model by generating samples of high quality textures and…
The advent of large scale multimedia databases has led to great challenges in content-based image retrieval (CBIR). Even though CBIR is considered an emerging field of research, however it constitutes a strong background for new methodologies and systems implementations. Therefore, many research contributions are focusing on techniques enabling higher image retrieval accuracy while preserving low level of computational complexity. Image retrieval based on texture features is receiving special attention because of the omnipresence of this visual feature in most real-world images. This paper hig…
Stop-band Energy Constraint for Orthogonal Tunable Wavelet Units in Convolutional Neural Networks for Computer Vision problems
Jacobs School of Engineering, University of California San Diego（加州大学圣地亚哥分校 Jacobs 工程学院）；Department of Computer Engineering, Tech University of Korea（科技大学韩国分校 计算机工程系）
Inpainting is the technique of reconstructing unknown or damaged portions of an image in a visually plausible way. Inpainting algorithm automatically fills the damaged region in an image using the information available in undamaged region. Propagation of structure and texture information becomes a challenge as the size of damaged area increases. In this paper, a hierarchical inpainting algorithm using wavelets is proposed. The hierarchical method tries to keep the mask size smaller while wavelets help in handling the high pass structure information and low pass texture information separately. …
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Seoul National University Bundang Hospital（首尔国立大学盆唐医院）
Model based iterative reconstruction (MBIR) algorithms for low-dose X-ray CT are computationally expensive. To address this problem, we recently proposed a deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the texture were not fully recovered. To address this problem, here we propose a novel framelet-based denoising algorithm using wavelet residual network which synergistically combines the expressive power of deep learning and the performance guarantee from the framelet-based denoising algorithms. …
This short memo aims at explaining our approach for the challenge IEEE-ISBI on Bone Texture Characterization. In this work, we focus on the use of covariance matrices and wavelet marginals in an SVM classifier.
Model based iterative reconstruction (MBIR) algorithms for low-dose X-ray CT are computationally complex because of the repeated use of the forward and backward projection. Inspired by this success of deep learning in computer vision applications, we recently proposed a deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the texture are not fully recovered, which was unfamiliar to some radiologists. To cope with this problem, here we propose a direct residual learning approach on directional wavelet d…
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Renmin University of China（中国人民大学）；Shenzhen University（深圳大学）
Magnetic resonance imaging plays an important role in computer-aided diagnosis and brain exploration. However, limited by hardware, scanning time and cost, it's challenging to acquire high-resolution (HR) magnetic resonance (MR) image clinically. In this paper, fine perceptive generative adversarial networks (FP-GANs) is proposed to produce HR MR images from low-resolution counterparts. It can cope with the detail insensitive problem of the existing super-resolution model in a divide-and-conquer manner. Specifically, FP-GANs firstly divides an MR image into low-frequency global approximation a…
Faculty of Mathematics and Computer Science（数学与计算机科学学院）；University of Havana（哈瓦那大学）
This paper presents and validates a novel lung nodule classification algorithm that uses multifractal features found in X-ray images. The proposed method includes a pre-processing step where two enhancement techniques are applied: histogram equalization and a combination of wavelet decomposition and morphological operations. As a novelty, multifractal features using wavelet leader based formalism are used with Support Vector Machine classifier; other classical texture features were also included. Best results were obtained when using multifractal features in combination with classical texture …
The GHM multi-level discrete wavelet transform is proposed as preprocessing for image super resolution with convolutional neural networks. Previous works perform analysis with the Haar wavelet only. In this work, 37 single-level wavelets are experimentally analyzed from Haar, Daubechies, Biorthogonal, Reverse Biorthogonal, Coiflets, and Symlets wavelet families. All single-level wavelets report similar results indicating that the convolutional neural network is invariant to choice of wavelet in a single-level filter approach. However, the GHM multi-level wavelet achieves higher quality reconst…
Northwestern University（西北大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of San Francisco（旧金山大学）；Imperial College London（帝国理工学院）；Univ. Paris Diderot, Sorbonne Paris Cité（巴黎第七大学（索邦巴黎大学））；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；San Diego State University（圣地亚哥州立大学）
In the fields of nanoscience and nanotechnology, it is important to be able to functionalize surfaces chemically for a wide variety of applications. Scanning tunneling microscopes (STMs) are important instruments in this area used to measure the surface structure and chemistry with better than molecular resolution. Self-assembly is frequently used to create monolayers that redefine the surface chemistry in just a single-molecule-thick layer. Indeed, STM images reveal rich information about the structure of self-assembled monolayers since they convey chemical and physical properties of the stud…
A wavelet scattering network computes a translation invariant image representation, which is stable to deformations and preserves high frequency information for classification. It cascades wavelet transform convolutions with non-linear modulus and averaging operators. The first network layer outputs SIFT-type descriptors whereas the next layers provide complementary invariant information which improves classification. The mathematical analysis of wavelet scattering networks explains important properties of deep convolution networks for classification. A scattering representation of stationary …
In this paper, we present a novel approach for image retrieval based on extraction of low level features using techniques such as Directional Binary Code, Haar Wavelet transform and Histogram of Oriented Gradients. The DBC texture descriptor captures the spatial relationship between any pair of neighbourhood pixels in a local region along a given direction, while Local Binary Patterns descriptor considers the relationship between a given pixel and its surrounding neighbours. Therefore, DBC captures more spatial information than LBP and its variants, also it can extract more edge information th…
Perceptual quality assessment for synthesized textures is a challenging task. In this paper, we propose a training-free reduced-reference (RR) objective quality assessment method that quantifies the perceived quality of synthesized textures. The proposed reduced-reference synthesized texture quality assessment metric is based on measuring the spatial and statistical attributes of the texture image using both image- and gradient-based wavelet coefficients at multiple scales. Performance evaluations on two synthesized texture databases demonstrate that our proposed RR synthesized texture quality…
Fujian Normal University（福建师范大学）；JOYY AI GROUP（欢聚集团AI团队）；China Design Group Co Ltd（中设集团有限公司）
Image deblurring is a classical computer vision problem that aims to recover a sharp image from a blurred image. To solve this problem, existing methods apply the Encode-Decode architecture to design the complex networks to make a good performance. However, most of these methods use repeated up-sampling and down-sampling structures to expand the receptive field, which results in texture information loss during the sampling process and some of them design the multiple stages that lead to difficulties with convergence. Therefore, our model uses dilated convolution to enable the obtainment of the…
Thermal infrared (IR) images represent the heat patterns emitted from hot object and they do not consider the energies reflected from an object. Objects living or non-living emit different amounts of IR energy according to their body temperature and characteristics. Humans are homoeothermic and hence capable of maintaining constant temperature under different surrounding temperature. Face recognition from thermal (IR) images should focus on changes of temperature on facial blood vessels. These temperature changes can be regarded as texture features of images and wavelet transform is a very goo…
Noise is one of the major problems that hinder an effective texture analysis of disease in medical images, which may cause variability in the reported diagnosis. In this paper seven texture measurement methods (two wavelet, two model and three statistical based) were applied to investigate their susceptibility to subtle noise caused by acquisition and reconstruction deficiencies in computed tomography (CT) images. Features of lung tumours were extracted from two different conventional and contrast enhanced CT image data-sets under filtered and noisy conditions. When measuring the noise in the …
The University of Tokyo（东京大学）；Digital Frontier Inc.（数字前沿株式会社）
Spatial and spectral approaches are two major approaches for image processing tasks such as image classification and object recognition. Among many such algorithms, convolutional neural networks (CNNs) have recently achieved significant performance improvement in many challenging tasks. Since CNNs process images directly in the spatial domain, they are essentially spatial approaches. Given that spatial and spectral approaches are known to have different characteristics, it will be interesting to incorporate a spectral approach into CNNs. We propose a novel CNN architecture, wavelet CNNs, which…
Wavelet-based Decoupling Framework for low-light Stereo Image Enhancement
Shaanxi Key Laboratory for Network Computing and Security Technology（陕西网络计算与安全技术重点实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；Xi'an University of Technology（西安理工大学）
Biorthogonal Tunable Wavelet Unit with Lifting Scheme in Convolutional Neural Network
Electrical and Computer Engineering Department, University of California San Diego（加州大学圣地亚哥分校电气与计算机工程系）；Department of Computer Engineering, Tech University of Korea（韩国科技大学计算机工程系）
Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Chinese Academy of Sciences（中国科学院大学）；Quaid-e-Awam University Engineering Science and Technology（奎德阿瓦姆工程科技大学）；Salim Habib University（萨利姆·哈比卜大学）；Waterford Institute of Technology（沃特福德理工学院）
Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from aremotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks(GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR). However, thegenerated image still suffers from undesirable artifacts such as, the absence of texture-feature representationand high-frequency information. We propose a frequency domain-based spatio-temporal remote sensingsingle image super-resolution technique to reconstruct the HR image…
Multi-scale Vision Transformer (ViT) has emerged as a powerful backbone for computer vision tasks, while the self-attention computation in Transformer scales quadratically w.r.t. the input patch number. Thus, existing solutions commonly employ down-sampling operations (e.g., average pooling) over keys/values to dramatically reduce the computational cost. In this work, we argue that such over-aggressive down-sampling design is not invertible and inevitably causes information dropping especially for high-frequency components in objects (e.g., texture details). Motivated by the wavelet theory, we…
Thermal infra-red (IR) images focus on changes of temperature distribution on facial muscles and blood vessels. These temperature changes can be regarded as texture features of images. A comparative study of face recognition methods working in thermal spectrum is carried out in this paper. In these study two local-matching methods based on Haar wavelet transform and Local Binary Pattern (LBP) are analyzed. Wavelet transform is a good tool to analyze multi-scale, multi-direction changes of texture. Local binary patterns (LBP) are a type of feature used for classification in computer vision. Fir…
University of Science and Technology of China（中国科学技术大学）；Huawei Technologies（华为技术有限公司）；Huawei Cloud BU（华为云业务单元）
When smartphone cameras are used to take photos of digital screens, usually moire patterns result, severely degrading photo quality. In this paper, we design a wavelet-based dual-branch network (WDNet) with a spatial attention mechanism for image demoireing. Existing image restoration methods working in the RGB domain have difficulty in distinguishing moire patterns from true scene texture. Unlike these methods, our network removes moire patterns in the wavelet domain to separate the frequencies of moire patterns from the image content. The network combines dense convolution modules and dilate…
Institut Gaspard Monge（加斯帕尔·蒙日研究所）；CNRS-UMR 8049（法国国家科学研究中心-混合研究单位8049）；Université de Marne la Vallée（马恩拉瓦莱大学）；Institut Français du Pétrole（法国石油研究院）
We propose a 2D generalization to the $M$-band case of the dual-tree decomposition structure (initially proposed by N. Kingsbury and further investigated by I. Selesnick) based on a Hilbert pair of wavelets. We particularly address (\textit{i}) the construction of the dual basis and (\textit{ii}) the resulting directional analysis. We also revisit the necessary pre-processing stage in the $M$-band case. While several reconstructions are possible because of the redundancy of the representation, we propose a new optimal signal reconstruction technique, which minimizes potential estimation errors…
Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Walton Institute for Information and Communication Systems Science（沃尔顿信息与通信系统科学研究所）；Waterford Institute of Technology（沃特福德理工学院）；University of Johannesburg（约翰内斯堡大学）；University of Sindh（信德大学）；Sungkyunkwan University（成均馆大学）
Deep learning based single image super resolution (SISR) algorithms has revolutionized the overall diagnosis framework by continually improving the architectural components and training strategies associated with convolutional neural networks (CNN) on low-resolution images. However, existing work lacks in two ways: i) the SR output produced exhibits poor texture details, and often produce blurred edges, ii) most of the models have been developed for a single modality, hence, require modification to adapt to a new one. This work addresses (i) by proposing generative adversarial network (GAN) wi…
University of Electronic Science and Technology of China（电子科技大学）
The wavelet frame systems have been widely investigated and applied for image restoration and many other image processing problems over the past decades, attributing to their good capability of sparsely approximating piece-wise smooth functions such as images. Most wavelet frame based models exploit the $l_1$ norm of frame coefficients for a sparsity constraint in the past. The authors in \cite{ZhangY2013, Dong2013} proposed an $l_0$ minimization model, where the $l_0$ norm of wavelet frame coefficients is penalized instead, and have demonstrated that significant improvements can be achieved c…
A scattering transform defines a signal representation which is invariant to translations and Lipschitz continuous relatively to deformations. It is implemented with a non-linear convolution network that iterates over wavelet and modulus operators. Lipschitz continuity locally linearizes deformations. Complex classes of signals and textures can be modeled with low-dimensional affine spaces, computed with a PCA in the scattering domain. Classification is performed with a penalized model selection. State of the art results are obtained for handwritten digit recognition over small training sets, …
Instituto de Telecomunicações（电信研究所）；Universidade da Beira Interior（贝拉内格罗大学）
Segmentation of skeletal muscles in Magnetic Resonance Images (MRI) is essential for the study of muscle physiology and diagnosis of muscular pathologies. However, manual segmentation of large MRI volumes is a time-consuming task. The state-of-the-art on algorithms for muscle segmentation in MRI is still not very extensive and is somewhat database-dependent. In this paper, an automated segmentation method based on AdaBoost classification of local texture features is presented. The texture descriptor consists of the Histogram of Oriented Gradients (HOG), Wavelet-based features, and a set of sta…
University of Toulouse（图卢兹大学）；CNRS（法国国家科学研究中心）；Heriot-Watt University（赫瑞-瓦特大学）；Ecole Normale Supérieure de Lyon（里昂高等师范学院）
Texture characterization is a central element in many image processing applications. Multifractal analysis is a useful signal and image processing tool, yet, the accurate estimation of multifractal parameters for image texture remains a challenge. This is due in the main to the fact that current estimation procedures consist of performing linear regressions across frequency scales of the two-dimensional (2D) dyadic wavelet transform, for which only a few such scales are computable for images. The strongly non-Gaussian nature of multifractal processes, combined with their complicated dependence…
