Deep learning algorithm display powerful ability in Computer Vision area, in recent year, the CNN has been applied to solve problems in the subarea of Image-generating, which has been widely applied in areas such as photo editing, image design, computer animation, real-time rendering for large scale of scenes and for visual effects in movies. However in the texture synthesize procedure. The state-of-art CNN can not capture the spatial location of texture in image, lead to significant distortion after texture synthesize, we propose a new way to generating-image by adding the semantic segment step with deep learning algorithm as Pre-Processing and analyze the outcome.
Perceptual quality assessment for synthesized textures is a challenging task. In this paper, we propose a training-free reduced-reference (RR) objective quality assessment method that quantifies the perceived quality of synthesized textures. The proposed reduced-reference synthesized texture quality assessment metric is based on measuring the spatial and statistical attributes of the texture image using both image- and gradient-based wavelet coefficients at multiple scales. Performance evaluations on two synthesized texture databases demonstrate that our proposed RR synthesized texture quality…
STS-GAN: Can We Synthesize Solid Texture with High Fidelity from Arbitrary 2D Exemplar?
Shandong Provincial Key Laboratory of Preparation and Measurement of Building Materials（山东省建筑材料制备与测试技术重点实验室）；South China University of Technology（华南理工大学）；Shandong Provincial Key Laboratory of Network Based Intelligent Computing（山东省网络环境智能计算技术重点实验室）；Quan Cheng Laboratory（泉城实验室）；Shandong Qiuqi Analysis Instrument Co., Ltd.（山东求奇分析仪器有限公司）；University of Waterloo（滑铁卢大学）；University of Jinan（济南大学）
Avinashilingam Institute for Home Science and Higher Education for Women（阿维纳希林加姆家政与高等教育女子学院）
Texture segmentation is the process of partitioning an image into regions with different textures containing a similar group of pixels. Detecting the discontinuity of the filter's output and their statistical properties help in segmenting and classifying a given image with different texture regions. In this proposed paper, chili x-ray image texture segmentation is performed by using Gabor filter. The texture segmented result obtained from Gabor filter fed into three texture filters, namely Entropy, Standard Deviation and Range filter. After performing texture analysis, features can be extracte…
The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）
Existing unconditional generative models mainly focus on modeling general objects, such as faces and indoor scenes. Fashion textures, another important type of visual elements around us, have not been extensively studied. In this work, we propose an effective generative model for fashion textures and also comprehensively investigate the key components involved: internal representation, latent space sampling and the generator architecture. We use Gram matrix as a suitable internal representation for modeling realistic fashion textures, and further design two dedicated modules for modulating Gra…
Instituto de Telecomunicações（电信研究所）；Universidade da Beira Interior（贝拉内格罗大学）
Segmentation of skeletal muscles in Magnetic Resonance Images (MRI) is essential for the study of muscle physiology and diagnosis of muscular pathologies. However, manual segmentation of large MRI volumes is a time-consuming task. The state-of-the-art on algorithms for muscle segmentation in MRI is still not very extensive and is somewhat database-dependent. In this paper, an automated segmentation method based on AdaBoost classification of local texture features is presented. The texture descriptor consists of the Histogram of Oriented Gradients (HOG), Wavelet-based features, and a set of sta…
In the last decade, deep learning has contributed to advances in a wide range computer vision tasks including texture analysis. This paper explores a new approach for texture segmentation using deep convolutional neural networks, sharing important ideas with classic filter bank based texture segmentation methods. Several methods are developed to train Fully Convolutional Networks to segment textures in various applications. We show in particular that these networks can learn to recognize and segment a type of texture, e.g. wood and grass from texture recognition datasets (no training segmentat…
NeRF-Texture: Synthesizing Neural Radiance Field Textures
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；VAST；ARC Lab, Tencent PCG（腾讯PCG ARC实验室）；School of Computer Science & Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）
In this work we present a method for lung nodules segmentation, their texture classification and subsequent follow-up recommendation from the CT image of lung. Our method consists of neural network model based on popular U-Net architecture family but modified for the joint nodule segmentation and its texture classification tasks and an ensemble-based model for the follow-up recommendation. This solution was evaluated within the LNDb medical imaging challenge and produced the best nodule segmentation result on the final leaderboard.
Federal University at Rio Grande do Norte（北里奥格兰德联邦大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Brazilian Federal Highway Police Station（巴西联邦公路警察局）；Universidad Nacional Autónoma de México（墨西哥国立自治大学）
Digital image segmentation is the process of assigning distinct labels to different objects in a digital image, and the fuzzy segmentation algorithm has been successfully used in the segmentation of images from a wide variety of sources. However, the traditional fuzzy segmentation algorithm fails to segment objects that are characterized by textures whose patterns cannot be successfully described by simple statistics computed over a very restricted area. In this paper, we propose an extension of the fuzzy segmentation algorithm that uses adaptive textural affinity functions to perform the segm…
Texture is intuitively defined as a repeated arrangement of a basic pattern or object in an image. There is no mathematical definition of a texture though. The human visual system is able to identify and segment different textures in a given image. Automating this task for a computer is far from trivial. There are three major components of any texture segmentation algorithm: (a) The features used to represent a texture, (b) the metric induced on this representation space and (c) the clustering algorithm that runs over these features in order to segment a given image into different textures. In…
We introduce one-shot texture segmentation: the task of segmenting an input image containing multiple textures given a patch of a reference texture. This task is designed to turn the problem of texture-based perceptual grouping into an objective benchmark. We show that it is straight-forward to generate large synthetic data sets for this task from a relatively small number of natural textures. In particular, this task can be cast as a self-supervised problem thereby alleviating the need for massive amounts of manually annotated data necessary for traditional segmentation tasks. In this paper w…
University for Health Sciences, Medical Informatics and Technology (UMIT)（健康科学、医学信息与技术大学（UMIT））
We study the applicability of a set of texture descriptors introduced in recent work by the author to texture-based segmentation of images. The texture descriptors under investigation result from applying graph indices from quantitative graph theory to graphs encoding the local structure of images. The underlying graphs arise from the computation of morphological amoebas as structuring elements for adaptive morphology, either as weighted or unweighted Dijkstra search trees or as edge-weighted pixel graphs within structuring elements. In the present paper we focus on texture descriptors in whic…
Existing semantic segmentation works mainly focus on learning the contextual information in high-level semantic features with CNNs. In order to maintain a precise boundary, low-level texture features are directly skip-connected into the deeper layers. Nevertheless, texture features are not only about local structure, but also include global statistical knowledge of the input image. In this paper, we fully take advantages of the low-level texture features and propose a novel Statistical Texture Learning Network (STLNet) for semantic segmentation. For the first time, STLNet analyzes the distribu…
South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Cixi Institute of Biomedical Engineering, Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences（中国科学院宁波材料技术与工程研究所慈溪生物医学工程研究所）
Retinal image segmentation plays an important role in automatic disease diagnosis. This task is very challenging because the complex structure and texture information are mixed in a retinal image, and distinguishing the information is difficult. Existing methods handle texture and structure jointly, which may lead biased models toward recognizing textures and thus results in inferior segmentation performance. To address it, we propose a segmentation strategy that seeks to separate structure and texture components and significantly improve the performance. To this end, we design a structure-tex…
Beijing University of Posts and Telecommunications（北京邮电大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
To solve the issue of segmenting rich texture images, a novel detection methods based on the affine invariable principle is proposed. Considering the similarity between the texture areas, we first take the affine transform to get numerous shapes, and utilize the KLT algorithm to verify the similarity. The transforms include rotation, proportional transformation and perspective deformation to cope with a variety of situations. Then we propose an improved LBP method combining canny edge detection to handle the boundary in the segmentation process. Moreover, human-computer interaction of this met…
Since annotating pixel-level labels for semantic segmentation is laborious, leveraging synthetic data is an attractive solution. However, due to the domain gap between synthetic domain and real domain, it is challenging for a model trained with synthetic data to generalize to real data. In this paper, considering the fundamental difference between the two domains as the texture, we propose a method to adapt to the texture of the target domain. First, we diversity the texture of synthetic images using a style transfer algorithm. The various textures of generated images prevent a segmentation mo…
Coordinated Science Lab, University of Illinois（伊利诺伊大学协调科学实验室）；HRL Laboratories, LLC（HRL 有限责任公司实验室）；University of California, Berkeley（加州大学伯克利分校）；Microsoft Research Asia（微软亚洲研究院）
We present a novel algorithm for segmentation of natural images that harnesses the principle of minimum description length (MDL). Our method is based on observations that a homogeneously textured region of a natural image can be well modeled by a Gaussian distribution and the region boundary can be effectively coded by an adaptive chain code. The optimal segmentation of an image is the one that gives the shortest coding length for encoding all textures and boundaries in the image, and is obtained via an agglomerative clustering process applied to a hierarchy of decreasing window sizes as multi…
In image processing, a segmentation is a process of partitioning an image into multiple sets of pixels, that are defined as super-pixels. Each super-pixel is characterized by a label or parameter. Here, we are proposing a method for determining the super-pixels based on the thresholding of the image. This approach is quite useful for studying the images showing vesicular textures.
Texture synthesis has proven successful at imitating a wide variety of textures. Adding additional constraints (in the form of a low-resolution version of the texture to be synthesized) makes it possible to use texture synthesis methods for texture superresolution.
Technical University of Munich（慕尼黑工业大学）；Universität Heidelberg（海德堡大学）；Darmstadt University of Applied Sciences（达姆施塔特应用科学大学）；Helmholtz Center Munich（慕尼黑亥姆霍兹中心）
Features that capture well the textural patterns of a certain class of images are crucial for the performance of texture segmentation methods. The manual selection of features or designing new ones can be a tedious task. Therefore, it is desirable to automatically adapt the features to a certain image or class of images. Typically, this requires a large set of training images with similar textures and ground truth segmentation. In this work, we propose a framework to learn features for texture segmentation when no such training data is available. The cost function for our learning process is c…
Amity University（阿米提大学）；Otto von Guericke University Magdeburg（马格德堡奥托·冯·格里克大学）
Colon Cancer is one of the most common types of cancer. The treatment is planned to depend on the grade or stage of cancer. One of the preconditions for grading of colon cancer is to segment the glandular structures of tissues. Manual segmentation method is very time-consuming, and it leads to life risk for the patients. The principal objective of this project is to assist the pathologist to accurate detection of colon cancer. In this paper, the authors have proposed an algorithm for an automatic segmentation of glands in colon histology using local intensity and texture features. Here the dat…
Technical University of Munich（慕尼黑工业大学）；Stanford University（斯坦福大学）
The modern computer graphics pipeline can synthesize images at remarkable visual quality; however, it requires well-defined, high-quality 3D content as input. In this work, we explore the use of imperfect 3D content, for instance, obtained from photo-metric reconstructions with noisy and incomplete surface geometry, while still aiming to produce photo-realistic (re-)renderings. To address this challenging problem, we introduce Deferred Neural Rendering, a new paradigm for image synthesis that combines the traditional graphics pipeline with learnable components. Specifically, we propose Neural …
Dynamic texture (DT) segmentation, and video processing in general, is currently widely dominated by methods based on deep neural networks that require the deployment of a large number of layers. Although this parametric approach has shown superior performances for the dynamic texture segmentation, all current deep learning methods suffer from a significant main weakness related to the lack of a sufficient reference annotation to train models and to make them functional. This study explores the unsupervised segmentation approach that can be used in the absence of training data to segment new v…
SkinFormer: Learning Statistical Texture Representation with Transformer for Skin Lesion Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shandong Computer Science Center（山东省计算中心）；Qilu University of Technology（齐鲁工业大学）；Shandong Fundamental Research Center for Computer Science（山东计算机科学基础研究中心）；Beijing University of Posts and Telecommunications（北京邮电大学）
Pinscreen；University of Southern California（南加州大学）；USC Institute for Creative Technologies（南加州大学创意技术研究所）
We present a data-driven inference method that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild. After an initial estimation of shape and low-frequency albedo, we compute a high-frequency partial texture map, without the shading component, of the visible face area. To extract the fine appearance details from this incomplete input, we introduce a multi-scale detail analysis technique based on mid-layer feature correlations extracted from a deep convolutional neural network. We demonstrate that fitting a convex combination of…
Domain generalization in semantic segmentation aims to alleviate the performance degradation on unseen domains through learning domain-invariant features. Existing methods diversify images in the source domain by adding complex or even abnormal textures to reduce the sensitivity to domain specific features. However, these approaches depend heavily on the richness of the texture bank, and training them can be time-consuming. In contrast to importing textures arbitrarily or augmenting styles randomly, we focus on the single source domain itself to achieve generalization. In this paper, we presen…
Univ Gustave Eiffel（古斯塔夫·埃菲尔大学）；IGN（法国国家地理与林业信息研究所）；ENSG（法国国立地理科学学院）；Direction Générale de l’Armement（法国武器装备总署）
Textured meshes are becoming an increasingly popular representation combining the 3D geometry and radiometry of real scenes. However, semantic segmentation algorithms for urban mesh have been little investigated and do not exploit all radiometric information. To address this problem, we adopt an approach consisting in sampling a point cloud from the textured mesh, then using a point cloud semantic segmentation algorithm on this cloud, and finally using the obtained semantic to segment the initial mesh. In this paper, we study the influence of different parameters such as the sampling method, t…
University of Science and Technology of China（中国科学技术大学）
In this paper, we tackle one-shot texture retrieval: given an example of a new reference texture, detect and segment all the pixels of the same texture category within an arbitrary image. To address this problem, we present an OS-TR network to encode both reference and query image, leading to achieve texture segmentation towards the reference category. Unlike the existing texture encoding methods that integrate CNN with orderless pooling, we propose a directionality-aware module to capture the texture variations at each direction, resulting in spatially invariant representation. To segment new…
University of Oxford（牛津大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；CentraleSupélec（巴黎中央理工-高等电力学院）；INRIA-Saclay（法国国家信息与自动化研究所萨克莱分院）
Visual textures have played a key role in image understanding because they convey important semantics of images, and because texture representations that pool local image descriptors in an orderless manner have had a tremendous impact in diverse applications. In this paper we make several contributions to texture understanding. First, instead of focusing on texture instance and material category recognition, we propose a human-interpretable vocabulary of texture attributes to describe common texture patterns, complemented by a new describable texture dataset for benchmarking. Second, we look a…
Sichuan University（四川大学）；University of Adelaide（阿德莱德大学）；Dalian University of Technology（大连理工大学）；Singapore University of Technology and Design（新加坡科技设计大学）
Semantic segmentation is a crucial image understanding task, where each pixel of image is categorized into a corresponding label. Since the pixel-wise labeling for ground-truth is tedious and labor intensive, in practical applications, many works exploit the synthetic images to train the model for real-word image semantic segmentation, i.e., Synthetic-to-Real Semantic Segmentation (SRSS). However, Deep Convolutional Neural Networks (CNNs) trained on the source synthetic data may not generalize well to the target real-world data. In this work, we propose two simple yet effective texture randomi…
NVIDIA（英伟达）；Simon Fraser University（西蒙弗雷泽大学）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
In this paper, we address the problem of texture representation for 3D shapes for the challenging and underexplored tasks of texture transfer and synthesis. Previous works either apply spherical texture maps which may lead to large distortions, or use continuous texture fields that yield smooth outputs lacking details. We argue that the traditional way of representing textures with images and linking them to a 3D mesh via UV mapping is more desirable, since synthesizing 2D images is a well-studied problem. We propose AUV-Net which learns to embed 3D surfaces into a 2D aligned UV space, by mapp…
Models for semantic segmentation require a large amount of hand-labeled training data which is costly and time-consuming to produce. For this purpose, we present a label fusion framework that is capable of improving semantic pixel labels of video sequences in an unsupervised manner. We make use of a 3D mesh representation of the environment and fuse the predictions of different frames into a consistent representation using semantic mesh textures. Rendering the semantic mesh using the original intrinsic and extrinsic camera parameters yields a set of improved semantic segmentation images. Due t…
University of Oxford（牛津大学）；University of Massachusetts, Amherst（马萨诸塞大学阿默斯特分校）
Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture at- tributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN substantially improves the state-of-the-art in texture, mate- rial and sce…
The goal of exemplar-based texture synthesis is to generate texture images that are visually similar to a given exemplar. Recently, promising results have been reported by methods relying on convolutional neural networks (ConvNets) pretrained on large-scale image datasets. However, these methods have difficulties in synthesizing image textures with non-local structures and extending to dynamic or sound textures. In this paper, we present a conditional generative ConvNet (cgCNN) model which combines deep statistics and the probabilistic framework of generative ConvNet (gCNN) model. Given a text…
Transferring Styles for Reduced Texture Bias and Improved Robustness in Semantic Segmentation Networks
Department of Computer Science Heinrich-Heine-University Düsseldorf（计算机科学系 汉诺威-海因希-恩斯特大学）；Department of Mathematics University of Wuppertal（数学系 威斯巴登大学）
Institute for High Performance Computing and Networking (ICAR), National Research Council (CNR)（国家研究委员会（CNR）高性能计算与网络研究所（ICAR））；University of Campania “Luigi Vanvitelli”（坎帕尼亚“路易吉·万维泰利”大学）
Two-region image segmentation is the process of dividing an image into two regions of interest, i.e., the foreground and the background. To this aim, Chan et al. [Chan, Esedoglu, Nikolova, SIAM Journal on Applied Mathematics 66(5), 1632-1648, 2006] designed a model well suited for smooth images. One drawback of this model is that it may produce a bad segmentation when the image contains oscillatory components. Based on a cartoon-texture decomposition of the image to be segmented, we propose a new model that is able to produce an accurate segmentation of images also containing noise or oscillat…
DexForce Technology Co. Ltd.（德克斯福科技有限公司）；Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；SmartMore Corporation（思谋科技有限公司）；South China University of Technology（华南理工大学）；Pazhou Lab（琶洲实验室）；Peng Cheng Lab（鹏程实验室）
In this work, we propose an interactive system to design diverse high-quality garment images from fashion sketches and the texture information. The major challenge behind this system is to generate high-quality and detailed texture according to the user-provided texture information. Prior works mainly use the texture patch representation and try to map a small texture patch to a whole garment image, hence unable to generate high-quality details. In contrast, inspired by intrinsic image decomposition, we decompose this task into texture synthesis and shading enhancement. In particular, we propo…
Georgia Institute of Technology（佐治亚理工学院）；Adobe Research（奥多比研究院）；UC Berkeley（加州大学伯克利分校）；Argo AI（阿尔戈人工智能公司）
In this paper, we investigate deep image synthesis guided by sketch, color, and texture. Previous image synthesis methods can be controlled by sketch and color strokes but we are the first to examine texture control. We allow a user to place a texture patch on a sketch at arbitrary locations and scales to control the desired output texture. Our generative network learns to synthesize objects consistent with these texture suggestions. To achieve this, we develop a local texture loss in addition to adversarial and content loss to train the generative network. We conduct experiments using sketche…
Manipal Institute of Technology（马尼帕尔理工学院）；Manipal Academy of Higher Education（马尼帕尔高等教育学院）；Manipal Institute of Technology Bengaluru（马尼帕尔理工学院（班加罗尔））
Forest plays a vital role in reducing greenhouse gas emissions and mitigating climate change besides maintaining the world's biodiversity. The existing satellite-based forest monitoring system utilizes supervised learning approaches that are limited to a particular region and depend on manually annotated data to identify forest. This work envisages forest identification as a few-shot semantic segmentation task to achieve generalization across different geographical regions. The proposed few-shot segmentation approach incorporates a texture attention module in the prototypical network to highli…
The ability to detect failures and anomalies are fundamental requirements for building reliable systems for computer vision applications, especially safety-critical applications of semantic segmentation, such as autonomous driving and medical image analysis. In this paper, we systematically study failure and anomaly detection for semantic segmentation and propose a unified framework, consisting of two modules, to address these two related problems. The first module is an image synthesis module, which generates a synthesized image from a segmentation layout map, and the second is a comparison m…
Zero-shot learning has been actively studied for image classification task to relieve the burden of annotating image labels. Interestingly, semantic segmentation task requires more labor-intensive pixel-wise annotation, but zero-shot semantic segmentation has only attracted limited research interest. Thus, we focus on zero-shot semantic segmentation, which aims to segment unseen objects with only category-level semantic representations provided for unseen categories. In this paper, we propose a novel Context-aware feature Generation Network (CaGNet), which can synthesize context-aware pixel-wi…
Texturing is a fundamental process in computer graphics. Texture is leveraged to enhance the visualization outcome for a 3D scene. In many cases a texture image cannot cover a large 3D model surface because of its small resolution. Conventional techniques like repeating, mirror repeating or clamp to edge do not yield visually acceptable results. Deep learning based texture synthesis has proven to be very effective in such cases. All deep texture synthesis methods trying to create larger resolution textures are limited in terms of GPU memory resources. In this paper, we propose a novel approach…
An easy zero-shot learning combination: Texture Sensitive Semantic Segmentation IceHrNet and Advanced Style Transfer Learning Strategy
Hohai University（河海大学）；Nanjing Research Institute of Hydrology and Water Conservation Automation, Ministry of Water Resources（水利部南京水利水文自动化研究所）；Nanjing Zhongyu Intelligent Water Conservancy Research Institute Co., Ltd（南京中禹智慧水利研究院有限公司）
University of Cambridge（剑桥大学）；National University of Singapore（新加坡国立大学）；National Institute of Technology（国家理工学院）
Automation of brain matter segmentation from MR images is a challenging task due to the irregular boundaries between the grey and white matter regions. In addition, the presence of intensity inhomogeneity in the MR images further complicates the problem. In this paper, we propose a texture and vesselness incorporated version of the ScatterNet Hybrid Deep Learning Network (TS-SHDL) that extracts hierarchical invariant mid-level features, used by fisher vector encoding and a conditional random field (CRF) to perform the desired segmentation. The performance of the proposed network is evaluated b…
