Text extraction is an important problem in image processing with applications from optical character recognition to autonomous driving. Most of the traditional text segmentation algorithms consider separating text from a simple background (which usually has a different color from texts). In this work we consider separating texts from a textured background, that has similar color to texts. We look at this problem from a signal decomposition perspective, and consider a more realistic scenario where signal components are overlaid on top of each other (instead of adding together). When the signals are overlaid, to separate signal components, we need to find a binary mask which shows the support of each component. Because directly solving the binary mask is intractable, we relax this problem to the approximated continuous problem, and solve it by alternating optimization method. We show that the proposed algorithm achieves significantly better results than other recent works on several challenging images.
In this paper, we present StrucTexTv2, an effective document image pre-training framework, by performing masked visual-textual prediction. It consists of two self-supervised pre-training tasks: masked image modeling and masked language modeling, based on text region-level image masking. The proposed method randomly masks some image regions according to the bounding box coordinates of text words. The objectives of our pre-training tasks are reconstructing the pixels of masked image regions and the corresponding masked tokens simultaneously. Hence the pre-trained encoder can capture more textual…
Korea Institute of Science and Technology（韩国科学技术研究院）；Seoul National University（首尔大学）；University of Science and Technology（科学技术联合大学院大学）
Face recognition now requires a large number of labelled masked face images in the era of this unprecedented COVID-19 pandemic. Unfortunately, the rapid spread of the virus has left us little time to prepare for such dataset in the wild. To circumvent this issue, we present a 3D model-based approach called WearMask3D for augmenting face images of various poses to the masked face counterparts. Our method proceeds by first fitting a 3D morphable model on the input image, second overlaying the mask surface onto the face model and warping the respective mask texture, and last projecting the 3D mas…
Hefei University of Technology（合肥工业大学）；Lenovo Research（联想研究院）
Image inpainting has achieved remarkable progress and inspired abundant methods, where the critical bottleneck is identified as how to fulfill the high-frequency structure and low-frequency texture information on the masked regions with semantics. To this end, deep models exhibit powerful superiority to capture them, yet constrained on the local spatial regions. In this paper, we delve globally into texture and structure information to well capture the semantics for image inpainting. As opposed to the existing arts trapped on the independent local patches, the texture information of each patch…
Research into the visual cortex and general neural information processing has led to various attempts to integrate pulse computation schemes in image analysis systems. Of interest is especially the robustness of representing an analogue signal in the phase or duration of a pulsed, quasi-digital signal, as well as the possibility of direct digital interaction, i.e. computation, among these signals. Such a computation can also achieve information compaction for subsequent processing stages. By using a pulse order encoding scheme motivated by dendritic pulse interaction, we will show that a power…
Texture analysis and classification are some of the problems which have been paid much attention by image processing scientists since late 80s. If texture analysis is done accurately, it can be used in many cases such as object tracking, visual pattern recognition, and face recognition.Since now, so many methods are offered to solve this problem. Against their technical differences, all of them used same popular databases to evaluate their performance such asBrodatz or Outex, which may be made their performance biased on these databases. In this paper, an approach is proposed to collect more e…
New Horizon College of Engineering（新视野工程学院）；Government Engineering College（政府工程学院）；University of Mysore（迈索尔大学）
Periodicity of a texture is one of the important visual characteristics and is often used as a measure for textural discrimination at the structural level. Knowledge about periodicity of a texture is very essential in the field of texture synthesis and texture compression and also in the design of frieze and wall papers. In this paper, we propose a method of periodicity extraction from noisy images based on superposition of distance matching function (DMF) and wavelet decomposition without de-noising the test images. Overall DMFs are subjected to single-level Haar wavelet decomposition to obta…
University of São Paulo（圣保罗大学）；University of Campinas（坎皮纳斯大学）
Convolutional neural networks have shown successful results in image classification achieving real-time results superior to the human level. However, texture images still pose some challenge to these models due, for example, to the limited availability of data for training in several problems where these images appear, high inter-class similarity, the absence of a global viewpoint of the object represented, and others. In this context, the present paper is focused on improving the accuracy of convolutional neural networks in texture classification. This is done by extracting features from mult…
Researches of analysis and parsing around façades to enrich the 3D feature of façade models by semantic information raised some attention in the community, whose main idea is to generate higher resolution components with similar shapes and textures to increase the overall resolution at the expense of reconstruction accuracy. While this approach works well for components like windows and doors, there is no solution for façade background at present. In this paper, we introduce the concept of representative region texture, which can be used in the above modeling approach by tiling the representat…
Graduate School of Information Sciences, Tohoku University（东北大学信息科学研究科）；Biometrics Research Laboratories, NEC Corporation（NEC公司生物识别研究所）
Although most fingerprint matching methods utilize minutia points and/or texture of fingerprint images as fingerprint features, the frequency spectrum is also a useful feature since a fingerprint is composed of ridge patterns with its inherent frequency band. We propose a novel CNN-based method for extracting fingerprint features from texture, minutiae, and frequency spectrum. In order to extract effective texture features from local regions around the minutiae, the minutia attention module is introduced to the proposed method. We also propose new data augmentation methods, which takes into ac…
In this paper we use machine learning to study the application of Local Tchebichef Moments (LTM) to the problem of texture classification. The original LTM method was proposed by Mukundan (2014). The LTM method can be used for texture analysis in many different ways, either using the moment values directly, or more simply creating a relationship between the moment values of different orders, producing a histogram similar to those of Local Binary Pattern (LBP) based methods. The original method was not fully tested with large datasets, and there are several parameters that should be characteris…
This paper aims to compare between four different types of feature extraction approaches in terms of texture segmentation. The feature extraction methods that were used for segmentation are Gabor filters (GF), Gaussian Markov random fields (GMRF), run-length matrix (RLM) and co-occurrence matrix (GLCM). It was shown that the GF performed best in terms of quality of segmentation while the GLCM localises the texture boundaries better as compared to the other methods.
SOM is a type of unsupervised learning where the goal is to discover some underlying structure of the data. In this paper, a new extraction method based on the main idea of Concurrent Self-Organizing Maps (CSOM), representing a winner-takes-all collection of small SOM networks is proposed. Each SOM of the system is trained individually to provide best results for one class only. The experiments confirm that the proposed features based CSOM is capable to represent image content better than extracted features based on a single big SOM and these proposed features improve the final decision of the…
University of São Paulo（圣保罗大学）；ENSTA-ParisTech（巴黎高等先进技术学校）；Université de Paris-Saclay（巴黎-萨克雷大学）
To recognize textures many methods have been developed along the years. However, texture datasets may be hard to be classified due to artefacts such as a variety of scale, illumination and noise. This paper proposes the application of binary distance transform on the original dataset to add information to texture representation and consequently improve recognition. Texture images, usually in grayscale, suffers a binarization prior to distance transform and one of the resulted images are combined with original texture to improve the amount of information. Four datasets are used to evaluate our …
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Tencent America（腾讯美国）；Advanced Institute of Information Technology, Peking University（北京大学高等信息科学技术研究院）
We deal with the controllable person image synthesis task which aims to re-render a human from a reference image with explicit control over body pose and appearance. Observing that person images are highly structured, we propose to generate desired images by extracting and distributing semantic entities of reference images. To achieve this goal, a neural texture extraction and distribution operation based on double attention is described. This operation first extracts semantic neural textures from reference feature maps. Then, it distributes the extracted neural textures according to the spati…
Avinashilingam Institute for Home Science and Higher Education for Women（阿维纳希林加姆家政与高等教育女子学院）
Texture segmentation is the process of partitioning an image into regions with different textures containing a similar group of pixels. Detecting the discontinuity of the filter's output and their statistical properties help in segmenting and classifying a given image with different texture regions. In this proposed paper, chili x-ray image texture segmentation is performed by using Gabor filter. The texture segmented result obtained from Gabor filter fed into three texture filters, namely Entropy, Standard Deviation and Range filter. After performing texture analysis, features can be extracte…
Texture is one of the most important properties of visual surface that helps in discriminating one object from another or an object from background. The self-organizing map (SOM) is an excellent tool in exploratory phase of data mining. It projects its input space on prototypes of a low-dimensional regular grid that can be effectively utilized to visualize and explore properties of the data. This paper proposes an enhancement extraction method for accurate extracting features for efficient image representation it based on SOM neural network. In this approach, we apply three different partition…
HMSViT: A Hierarchical Masked Self-Supervised Vision Transformer for Corneal Nerve Segmentation and Diabetic Neuropathy Diagnosis
Department of Computing and Mathematics, Manchester Metropolitan University（计算与数学系，曼彻斯特 Metropolitan 大学）；Department of Eye and Vision Sciences, University of Liverpool（眼科学与视觉科学系，利物浦大学）；Faculty of Biology, Medicine and Health, University of Manchester（生物、医学与健康学院，曼彻斯特大学）；Department of Medicine, Weill Cornell Medicine-Qatar（医学系，Weill Cornell Medicine-Qatar）
The advent of large scale multimedia databases has led to great challenges in content-based image retrieval (CBIR). Even though CBIR is considered an emerging field of research, however it constitutes a strong background for new methodologies and systems implementations. Therefore, many research contributions are focusing on techniques enabling higher image retrieval accuracy while preserving low level of computational complexity. Image retrieval based on texture features is receiving special attention because of the omnipresence of this visual feature in most real-world images. This paper hig…
Southwest Jiaotong University（西南交通大学）；Capital Normal University（首都师范大学）
Photogrammetric mesh models obtained from aerial oblique images have been widely used for urban reconstruction. However, the photogrammetric meshes also suffer from severe texture problems, especially on the road areas due to occlusion. This paper proposes a structure-aware completion approach to improve the quality of meshes by removing undesired vehicles on the road seamlessly. Specifically, the discontinuous texture atlas is first integrated to a continuous screen space through rendering by the graphics pipeline; the rendering also records necessary mapping for deintegration to the original…
Jawaharlal Nehru University（贾瓦哈拉尔·尼赫鲁大学）；Indian Institute of Technology Delhi（印度理工学院德里分校）
Predicting attributes in the landmark free facial images is itself a challenging task which gets further complicated when the face gets occluded due to the usage of masks. Smart access control gates which utilize identity verification or the secure login to personal electronic gadgets may utilize face as a biometric trait. Particularly, the Covid-19 pandemic increasingly validates the essentiality of hygienic and contactless identity verification. In such cases, the usage of masks become more inevitable and performing attribute prediction helps in segregating the target vulnerable groups from …
A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis
São Carlos Institute of Physics, University of São Paulo（圣保罗大学圣卡洛斯物理学院）；Institute of Mathematics and Computer Sciences, University of São Paulo（圣保罗大学数学与计算机科学研究所）；Institute of Biosciences, Humanities and Exact Sciences, São Paulo State University（圣保罗州立大学生物科学、人文与精确科学研究所）；Ghent University（根特大学）
本文系统比较了21种预训练视觉Transformer在纹理特征提取上的表现，发现ViTs在鲁棒性、颜色/材料纹理和属性识别等任务中普遍优于CNNs和手工模型，并指出ViT-B with DINO、BeiTv2、Swin及EfficientFormer为有前景的模型。
Texture-based classification solutions have proven their significance in many domains, from industrial inspections to health-related applications. New methods have been developed based on texture feature learning and CNN-based architectures to address computer vision use cases for images with rich texture-based features. In recent years, architectures solving texture-based classification problems and demonstrating state-of-the-art results have emerged. Yet, one limitation of these approaches is that they cannot claim to be suitable for all types of image texture patterns. Each technique has an…
MixSA: Training-free Reference-based Sketch Extraction via Mixture-of-Self-Attention
School of Computer Sciences, Shaanxi Normal University（陕西师范大学计算机科学学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）
Shanghai AI Laboratory（上海人工智能实验室）；Simon Fraser University（西蒙弗雷泽大学）；The Chinese University of Hong Kong（香港中文大学）
Masked Image Modeling (MIM) has achieved promising progress with the advent of Masked Autoencoders (MAE) and BEiT. However, subsequent works have complicated the framework with new auxiliary tasks or extra pre-trained models, inevitably increasing computational overhead. This paper undertakes a fundamental analysis of MIM from the perspective of pixel reconstruction, which examines the input image patches and reconstruction target, and highlights two critical but previously overlooked bottlenecks. Based on this analysis, we propose a remarkably simple and effective method, {\ourmethod}, that e…
In this paper, we present a novel approach for image retrieval based on extraction of low level features using techniques such as Directional Binary Code, Haar Wavelet transform and Histogram of Oriented Gradients. The DBC texture descriptor captures the spatial relationship between any pair of neighbourhood pixels in a local region along a given direction, while Local Binary Patterns descriptor considers the relationship between a given pixel and its surrounding neighbours. Therefore, DBC captures more spatial information than LBP and its variants, also it can extract more edge information th…
Feature extraction is a key step in image processing for pattern recognition and machine learning processes. Its purpose lies in reducing the dimensionality of the input data through the computing of features which accurately describe the original information. In this article, a new feature extraction method based on Discrete Modal Decomposition (DMD) is introduced, to extend the group of space and frequency based features. These new features are called modal features. Initially aiming to decompose a signal into a modal basis built from a vibration mechanics problem, the DMD projection is appl…
University of Electronic Science and Technology of China（电子科技大学）；Yibin University（宜宾学院）
In this paper, we proposed an effective face feature extraction method by Learning Gabor Log-Euclidean Gaussian with Whitening Principal Component Analysis (WPCA), called LGLG-WPCA. The proposed method learns face features from the embedded multivariate Gaussian in Gabor wavelet domain; it has the robust performance to adverse conditions such as varying poses, skin aging and uneven illumination. Because the space of Gaussian is a Riemannian manifold and it is difficult to incorporate learning mechanism in the model. To address this issue, we use L2EMG to map the multidimensional Gaussian model…
This work studies the problem of content-based image retrieval, specifically, texture retrieval. It focuses on feature extraction and similarity measure for texture images. Our approach employs a recently developed method, the so-called Scattering transform, for the process of feature extraction in texture retrieval. It shares a distinctive property of providing a robust representation, which is stable with respect to spatial deformations. Recent work has demonstrated its capability for texture classification, and hence as a promising candidate for the problem of texture retrieval. Moreover, w…
Thinking inside the Convolution for Image Inpainting: Reconstructing Texture via Structure under Global and Local Side
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）；Lenovo Research（联想研究院）
Al-Hussein Bin Talal University（侯赛因·本·塔拉勒大学）；University of Mysore（迈索尔大学）
In this paper, we have developed Biometric recognition system adopting hand based modality Handvein, which has the unique pattern for each individual and it is impossible to counterfeit and fabricate as it is an internal feature. We have opted in choosing feature extraction algorithms such as LBP-visual descriptor ,LPQ-blur insensitive texture operator, Log-Gabor-Texture descriptor. We have chosen well known classifiers such as KNN and SVM for classification. We have experimented and tabulated results of single algorithm recognition rate for Handvein under different distance measures and kerne…
Nanjing University of Science and Technology（南京理工大学）；Tencent AI Lab（腾讯人工智能实验室）
We present a large-scale facial UV-texture dataset that contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions, which are desired characteristics for rendering realistic 3D face models under different lighting conditions. The dataset is derived from a large-scale face image dataset namely FFHQ, with the help of our fully automatic and robust UV-texture production pipeline. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image …
University of Chile（智利大学）；Snap Research（Snap 研究院）；Impresee Inc.（Impresee 公司）
Visual attributes play an essential role in real applications based on image retrieval. For instance, the extraction of attributes from images allows an eCommerce search engine to produce retrieval results with higher precision. The traditional manner to build an attribute extractor is by training a convnet-based classifier with a fixed number of classes. However, this approach does not scale for real applications where the number of attributes changes frequently. Therefore in this work, we propose an approach for extracting visual attributes from images, leveraging the learned capability of t…
Within the domain of texture classification, a lot of effort has been spent on local descriptors, leading to many powerful algorithms. However, preprocessing techniques have received much less attention despite their important potential for improving the overall classification performance. We address this question by proposing a novel, simple, yet very powerful biologically-inspired filtering (BF) which simulates the performance of human retina. In the proposed approach, given a texture image, after applying a DoG filter to detect the "edges", we first split the filtered image into two "maps" …
Velammal College of Engg and Tech（维拉马尔工程技术学院）；Thiagarajar College of Engg（蒂亚加拉贾工程技术学院）
The research work presented in this paper is to achieve the tissue classification and automatically diagnosis the abnormal tumor region present in Computed Tomography (CT) images using the wavelet based statistical texture analysis method. Comparative studies of texture analysis method are performed for the proposed wavelet based texture analysis method and Spatial Gray Level Dependence Method (SGLDM). Our proposed system consists of four phases i) Discrete Wavelet Decomposition (ii) Feature extraction (iii) Feature selection (iv) Analysis of extracted texture features by classifier. A wavelet…
Multi-scale Restoration of Missing Data in Optical Time-series Images with Masked Spatial-Temporal Attention Network
School of Computer Science, China University of Geosciences（中国地质大学（武汉）计算机学院）；Engineering Research Center of Natural Resource Information Management and Digital Twin Engineering Software, Ministry of Education（教育部自然资源信息管理与数字孪生工程软件工程研究中心）；Second Surveying and Mapping Institute of Hunan Province（湖南省第二测绘院）；Key Laboratory of Natural Resources Monitoring and Supervision in Southern Hilly Region, Ministry of Natural Resources（自然资源部南方山地自然资源监测与监管重点实验室）
Wavelet-Driven Masked Image Modeling: A Path to Efficient Visual Representation
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Peng Cheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Shanghai Jiao Tong University（上海交通大学）
NeRF-MIR: Towards High-Quality Restoration of Masked Images with Neural Radiance Fields
Shanghai Key Lab of Intelligent Information Processing（上海智能信息处理重点实验室）；College of Computer Science and Artificial Intelligence, Fudan University（复旦大学计算机科学与人工智能学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Hefei University of Technology（合肥工业大学）；University of Science and Technology of China（中国科学技术大学）；Institute of Advanced Technology, University of Science and Technology of China（中国科学技术大学先进技术研究院）
This paper presents an efficient method for texture retrieval using multiscale feature extraction and embedding based on the local extrema keypoints. The idea is to first represent each texture image by its local maximum and local minimum pixels. The image is then divided into regular overlapping blocks and each one is characterized by a feature vector constructed from the radiometric, geometric and structural information of its local extrema. All feature vectors are finally embedded into a covariance matrix which will be exploited for dissimilarity measurement within retrieval task. Thanks to…
East China Normal University（华东师范大学）；Dickinson College（狄金森学院）
Recently, we have witnessed the explosive growth of images with complex information and content. In order to effectively and precisely retrieve desired images from a large-scale image database with low time-consuming, we propose the multiple feature fusion image retrieval algorithm based on the texture feature and rough set theory in this paper. In contrast to the conventional approaches that only use the single feature or standard, we fuse the different features with operation of normalization. The rough set theory will assist us to enhance the robustness of retrieval system when facing with …
