In this paper, we show how to efficiently and effectively extract a class of "low-rank textures" in a 3D scene from 2D images despite significant corruptions and warping. The low-rank textures capture geometrically meaningful structures in an image, which encompass conventional local features such as edges and corners as well as all kinds of regular, symmetric patterns ubiquitous in urban environments and man-made objects. Our approach to finding these low-rank textures leverages the recent breakthroughs in convex optimization that enable robust recovery of a high-dimensional low-rank matrix despite gross sparse errors. In the case of planar regions with significant affine or projective deformation, our method can accurately recover both the intrinsic low-rank texture and the precise domain transformation, and hence the 3D geometry and appearance of the planar regions. Extensive experimental results demonstrate that this new technique works effectively for many regular and near-regular patterns or objects that are approximately low-rank, such as symmetrical patterns, building facades, printed texts, and human faces.
In this paper, we present some new features describing the handwritten document as a texture. These features are based on the Radon transform. All values can be obtained easily and suit for the coarse classification of documents.
Microsoft Research（微软研究院）；Shanghai Jiao Tong University（上海交通大学）
We study on image super-resolution (SR), which aims to recover realistic textures from a low-resolution (LR) image. Recent progress has been made by taking high-resolution images as references (Ref), so that relevant textures can be transferred to LR images. However, existing SR approaches neglect to use attention mechanisms to transfer high-resolution (HR) textures from Ref images, which limits these approaches in challenging cases. In this paper, we propose a novel Texture Transformer Network for Image Super-Resolution (TTSR), in which the LR and Ref images are formulated as queries and keys…
Over the last few years, the performance of inpainting to fill missing regions has shown significant improvements by using deep neural networks. Most of inpainting work create a visually plausible structure and texture, however, due to them often generating a blurry result, final outcomes appear unrealistic and make feel heterogeneity. In order to solve this problem, the existing methods have used a patch based solution with deep neural network, however, these methods also cannot transfer the texture properly. Motivated by these observation, we propose a patch based method. Texture Transform A…
Deep Learning (DL) based methods for magnetic resonance (MR) image reconstruction have been shown to produce superior performance in recent years. However, these methods either only leverage under-sampled data or require a paired fully-sampled auxiliary modality to perform multi-modal reconstruction. Consequently, existing approaches neglect to explore attention mechanisms that can transfer textures from reference fully-sampled data to under-sampled data within a single modality, which limits these approaches in challenging cases. In this paper, we propose a novel Texture Transformer Module (T…
CUHK - SenseTime Joint Lab（香港中文大学-商汤联合实验室）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）
Despite that convolutional neural networks (CNN) have recently demonstrated high-quality reconstruction for single-image super-resolution (SR), recovering natural and realistic texture remains a challenging problem. In this paper, we show that it is possible to recover textures faithful to semantic classes. In particular, we only need to modulate features of a few intermediate layers in a single network conditioned on semantic segmentation probability maps. This is made possible through a novel Spatial Feature Transform (SFT) layer that generates affine transformation parameters for spatial-wi…
University of São Paulo（圣保罗大学）；ENSTA-ParisTech（巴黎高等先进技术学校）；Université de Paris-Saclay（巴黎-萨克雷大学）
To recognize textures many methods have been developed along the years. However, texture datasets may be hard to be classified due to artefacts such as a variety of scale, illumination and noise. This paper proposes the application of binary distance transform on the original dataset to add information to texture representation and consequently improve recognition. Texture images, usually in grayscale, suffers a binarization prior to distance transform and one of the resulted images are combined with original texture to improve the amount of information. Four datasets are used to evaluate our …
Image textures, as a kind of local variations, provide important information for human visual system. Many image textures, especially the small-scale or stochastic textures are rich in high-frequency variations, and are difficult to be preserved. Current state-of-the-art denoising algorithms typically adopt a nonlocal approach consisting of image patch grouping and group-wise denoising filtering. To achieve a better image denoising while preserving the variations in texture, we first adaptively group high correlated image patches with the same kinds of texture elements (texels) via an adaptive…
Many stochastic processes are defined on special geometrical objects like spheres and cones. We describe how tools from harmonic analysis, i.e. Fourier analysis on groups, can be used to investigate probability density functions (pdfs) on groups and homogeneous spaces. We consider the special case of the Lorentz group SU(1,1) and the unit disk with its hyperbolic geometry, but the procedure can be generalized to a much wider class of Lie-groups. We mainly concentrate on the Mehler-Fock transform which is the radial part of the Fourier transform on the disk. Some of the characteristic features …
University of Surrey（萨里大学）；iFlyTek-Surrey Joint Research Centre on Artificial Intelligence（科大讯飞-萨里人工智能联合研究中心）；Leibniz University Hannover（汉诺威莱布尼茨大学）；University of Twente（屯特大学）
A lifespan face synthesis (LFS) model aims to generate a set of photo-realistic face images of a person's whole life, given only one snapshot as reference. The generated face image given a target age code is expected to be age-sensitive reflected by bio-plausible transformations of shape and texture, while being identity preserving. This is extremely challenging because the shape and texture characteristics of a face undergo separate and highly nonlinear transformations w.r.t. age. Most recent LFS models are based on generative adversarial networks (GANs) whereby age code conditional transform…
Beijing University of Posts and Telecommunications（北京邮电大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
To solve the issue of segmenting rich texture images, a novel detection methods based on the affine invariable principle is proposed. Considering the similarity between the texture areas, we first take the affine transform to get numerous shapes, and utilize the KLT algorithm to verify the similarity. The transforms include rotation, proportional transformation and perspective deformation to cope with a variety of situations. Then we propose an improved LBP method combining canny edge detection to handle the boundary in the segmentation process. Moreover, human-computer interaction of this met…
This work presents a new procedure to extract features of grey-level texture images based on the discrete Schroedinger transform. This is a non-linear transform where the image is mapped as the initial probability distribution of a wave function and such distribution evolves in time following the Schroedinger equation from Quantum Mechanics. The features are provided by statistical moments of the distribution measured at different times. The proposed method is applied to the classification of three databases of textures used for benchmark and compared to other well-known texture descriptors in…
SkinFormer: Learning Statistical Texture Representation with Transformer for Skin Lesion Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shandong Computer Science Center（山东省计算中心）；Qilu University of Technology（齐鲁工业大学）；Shandong Fundamental Research Center for Computer Science（山东计算机科学基础研究中心）；Beijing University of Posts and Telecommunications（北京邮电大学）
Hefei University of Technology（合肥工业大学）；Lenovo Research（联想研究院）
Image inpainting has achieved remarkable progress and inspired abundant methods, where the critical bottleneck is identified as how to fulfill the high-frequency structure and low-frequency texture information on the masked regions with semantics. To this end, deep models exhibit powerful superiority to capture them, yet constrained on the local spatial regions. In this paper, we delve globally into texture and structure information to well capture the semantics for image inpainting. As opposed to the existing arts trapped on the independent local patches, the texture information of each patch…
Reference-based image super-resolution (RefSR) aims to exploit auxiliary reference (Ref) images to super-resolve low-resolution (LR) images. Recently, RefSR has been attracting great attention as it provides an alternative way to surpass single image SR. However, addressing the RefSR problem has two critical challenges: (i) It is difficult to match the correspondence between LR and Ref images when they are significantly different; (ii) How to transfer the relevant texture from Ref images to compensate the details for LR images is very challenging. To address these issues of RefSR, this paper p…
City University of Hong Kong（香港城市大学）；Flatiron Institute of the Simons Foundation（西蒙斯基金会熨斗研究所）；New York University（纽约大学）
Objective measures of image quality generally operate by comparing pixels of a "degraded" image to those of the original. Relative to human observers, these measures are overly sensitive to resampling of texture regions (e.g., replacing one patch of grass with another). Here, we develop the first full-reference image quality model with explicit tolerance to texture resampling. Using a convolutional neural network, we construct an injective and differentiable function that transforms images to multi-scale overcomplete representations. We demonstrate empirically that the spatial averages of the …
Face forgery by deepfake is widely spread over the internet and this raises severe societal concerns. In this paper, we propose a novel video transformer with incremental learning for detecting deepfake videos. To better align the input face images, we use a 3D face reconstruction method to generate UV texture from a single input face image. The aligned face image can also provide pose, eyes blink and mouth movement information that cannot be perceived in the UV texture image, so we use both face images and their UV texture maps to extract the image features. We present an incremental learning…
Georgia Institute of Technology（佐治亚理工学院）；Brandon University（布兰登大学）；GN Jabra A/S（GN捷波朗公司）
Hand-held light field cameras suffer from low spatial resolution due to the inherent spatio-angular tradeoff. In this paper, we propose a method to improve the spatial resolution of light field images with the aid of the Texture Transformer Network (TTSR). The proposed method consists of three modules: the first module produces an all-in focus high-resolution perspective image which serves as a reference image for the second module, i.e. TTSR, which in turn produces a high-resolution light field. The last module refines the spatial resolution by imposing a light field prior. The results demons…
This work presents a novel descriptor for texture images based on fractal geometry and its application to image analysis. The descriptors are provided by estimating the triangular prism fractal dimension under different scales with a weight exponential parameter, followed by dimensionality reduction using Karhunen-Loève transform. The efficiency of the proposed descriptors is tested on two well-known texture data sets, that is, Brodatz and Vistex, both for classification and image retrieval. The novel method is also tested concerning invariances in situations when the textures are rotated or a…
Najafabad Branch, Islamic Azad University（伊斯兰阿扎德大学纳杰法巴德分校）
Texture classification is an active topic in image processing which plays an important role in many applications such as image retrieval, inspection systems, face recognition, medical image processing, etc. There are many approaches extracting texture features in gray-level images such as local binary patterns, gray level co-occurrence matrices, statistical features, skeleton, scale invariant feature transform, etc. The texture analysis methods can be categorized in 4 groups titles: statistical methods, structural methods, filter-based and model based approaches. In many related researches, au…
We propose a Transformer-based framework for 3D human texture estimation from a single image. The proposed Transformer is able to effectively exploit the global information of the input image, overcoming the limitations of existing methods that are solely based on convolutional neural networks. In addition, we also propose a mask-fusion strategy to combine the advantages of the RGB-based and texture-flow-based models. We further introduce a part-style loss to help reconstruct high-fidelity colors without introducing unpleasant artifacts. Extensive experiments demonstrate the effectiveness of t…
A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis
São Carlos Institute of Physics, University of São Paulo（圣保罗大学圣卡洛斯物理学院）；Institute of Mathematics and Computer Sciences, University of São Paulo（圣保罗大学数学与计算机科学研究所）；Institute of Biosciences, Humanities and Exact Sciences, São Paulo State University（圣保罗州立大学生物科学、人文与精确科学研究所）；Ghent University（根特大学）
本文系统比较了21种预训练视觉Transformer在纹理特征提取上的表现，发现ViTs在鲁棒性、颜色/材料纹理和属性识别等任务中普遍优于CNNs和手工模型，并指出ViT-B with DINO、BeiTv2、Swin及EfficientFormer为有前景的模型。
City University of Hong Kong（香港城市大学）；Microsoft Cloud + AI（微软云与人工智能部门）
Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity. However, due to some inherent properties (e.g., local inductive prior, spatial-invariant kernels), CNNs do not perform well in understanding global structures or naturally support pluralistic completion. Recently, transformers demonstrate their power in modeling the long-term relationship and generating diverse results, but their computation complexity is quadratic to input length, thus hampering the application in processing high-resolution images. This…
École de Technologie Supérieure, Université du Québec（魁北克大学高等技术学院）
Breast cancer is a health problem that affects mainly the female population. An early detection increases the chances of effective treatment, improving the prognosis of the disease. In this regard, computational tools have been proposed to assist the specialist in interpreting the breast digital image exam, providing features for detecting and diagnosing tumors and cancerous cells. Nonetheless, detecting tumors with a high sensitivity rate and reducing the false positives rate is still challenging. Texture descriptors have been quite popular in medical image analysis, particularly in histopath…
Faculty of Sciences, Ibn Zohr University（伊本·佐赫尔大学理学院）
With the large uses of the intelligent systems in different domains, and in order to increase the drivers and pedestrians safety, the road and traffic sign recognition system has been a challenging issue and an important task for many years. But studies, done in this field of detection and recognition of traffic signs in an image, which are interested in the Arab context, are still insufficient. Detection of the road signs present in the scene is the one of the main stages of the traffic sign detection and recognition. In this paper, an efficient solution to enhance road signs detection, inclu…
Numerous methods have been proposed to transform color and grayscale images to their single bit-per-pixel binary counterparts. Commonly, the goal is to enhance specific attributes of the original image to make it more amenable for analysis. However, when the resulting binarized image is intended for human viewing, aesthetics must also be considered. Binarization techniques, such as half-toning, stippling, and hatching, have been widely used for modeling the original image's intensity profile. We present an automated method to transform an image to a set of binary textures that represent not on…
This paper presents a novel framework for generating texture mosaics with convolutional neural networks. Our method is called GANosaic and performs optimization in the latent noise space of a generative texture model, which allows the transformation of a content image into a mosaic exhibiting the visual properties of the underlying texture manifold. To represent that manifold, we use a state-of-the-art generative adversarial method for texture synthesis, which can learn expressive texture representations from data and produce mosaic images with very high resolution. This fully convolutional mo…
