Mathematical morphological methods have successfully been applied to filter out (emphasize or remove) different structures of an image. However, it is argued that these methods could be suitable for the task only if the type and order of the filter(s) as well as the shape and size of operator kernel are designed properly. Thus the existing filtering operators are problem (instance) specific and are designed by the domain experts. In this work we propose a morphological network that emulates classical morphological filtering consisting of a series of erosion and dilation operators with trainable structuring elements. We evaluate the proposed network for image de-raining task where the SSIM and mean absolute error (MAE) loss corresponding to predicted and ground-truth clean image is back-propagated through the network to train the structuring elements. We observe that a single morphological network can de-rain an image with any arbitrary shaped rain-droplets and achieves similar performance with the contemporary CNNs for this task with a fraction of trainable parameters (network size). The proposed morphological network(MorphoN) is not designed specifically for de-raining and can readily be applied to similar filtering / noise cleaning tasks. The source code can be found here https://github.com/ranjanZ/2D-Morphological-Network
Severe weather conditions such as rain and snow adversely affect the visual quality of images captured under such conditions thus rendering them useless for further usage and sharing. In addition, such degraded images drastically affect performance of vision systems. Hence, it is important to solve the problem of single image de-raining/de-snowing. However, this is a difficult problem to solve due to its inherent ill-posed nature. Existing approaches attempt to introduce prior information to convert it into a well-posed problem. In this paper, we investigate a new point of view in addressing t…
Single image rain streak removal is an extremely challenging problem due to the presence of non-uniform rain densities in images. We present a novel density-aware multi-stream densely connected convolutional neural network-based algorithm, called DID-MDN, for joint rain density estimation and de-raining. The proposed method enables the network itself to automatically determine the rain-density information and then efficiently remove the corresponding rain-streaks guided by the estimated rain-density label. To better characterize rain-streaks with different scales and shapes, a multi-stream den…
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）
We present a method for improving segmentation tasks on images affected by adherent rain drops and streaks. We introduce a novel stereo dataset recorded using a system that allows one lens to be affected by real water droplets while keeping the other lens clear. We train a denoising generator using this dataset and show that it is effective at removing the effect of real water droplets, in the context of image reconstruction and road marking segmentation. To further test our de-noising approach, we describe a method of adding computer-generated adherent water droplets and streaks to any images…
University of Electronic Science and Technology of China（电子科技大学）
Images acquired by outdoor vision systems easily suffer poor visibility and annoying interference due to the rainy weather, which brings great challenge for accurately understanding and describing the visual contents. Recent researches have devoted great efforts on the task of rain removal for improving the image visibility. However, there is very few exploration about the quality assessment of de-rained image, even it is crucial for accurately measuring the performance of various de-raining algorithms. In this paper, we first create a de-raining quality assessment (DQA) database that collects…
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Sun Yat-sen University（中山大学）；SenseTime Research（商汤科技研究院）
Most advances in single image de-raining meet a key challenge, which is removing rain streaks with different scales and shapes while preserving image details. Existing single image de-raining approaches treat rain-streak removal as a process of pixel-wise regression directly. However, they are lacking in mining the balance between over-de-raining (e.g. removing texture details in rain-free regions) and under-de-raining (e.g. leaving rain streaks). In this paper, we firstly propose a coarse-to-fine network called Gradual Network (GraNet) consisting of coarse stage and fine stage for delving int…
Whiting School of Engineering, Johns Hopkins University（约翰斯·霍普金斯大学怀廷工程学院）
Single image de-raining is an extremely challenging problem since the rainy images contain rain streaks which often vary in size, direction and density. This varying characteristic of rain streaks affect different parts of the image differently. Previous approaches have attempted to address this problem by leveraging some prior information to remove rain streaks from a single image. One of the major limitations of these approaches is that they do not consider the location information of rain drops in the image. The proposed Image Quality-based single image Deraining using Confidence measure (Q…
Sun Yat-sen University（中山大学）；University of Electronic Science and Technology of China（电子科技大学）
Single image rain streaks removal has recently witnessed substantial progress due to the development of deep convolutional neural networks. However, existing deep learning based methods either focus on the entrance and exit of the network by decomposing the input image into high and low frequency information and employing residual learning to reduce the mapping range, or focus on the introduction of cascaded learning scheme to decompose the task of rain streaks removal into multi-stages. These methods treat the convolutional neural network as an encapsulated end-to-end mapping module without d…
Single image de-raining is an extremely challenging problem since the rainy image may contain rain streaks which may vary in size, direction and density. Previous approaches have attempted to address this problem by leveraging some prior information to remove rain streaks from a single image. One of the major limitations of these approaches is that they do not consider the location information of rain drops in the image. The proposed Uncertainty guided Multi-scale Residual Learning (UMRL) network attempts to address this issue by learning the rain content at different scales and using them to …
Beijing Institute of Technology（北京理工大学）；Beijing Key Laboratory of Intelligent Information Technology（北京智能信息技术重点实验室）
Removing rain streaks from rainy images is necessary for many tasks in computer vision, such as object detection and recognition. It needs to address two mutually exclusive objectives: removing rain streaks and reserving realistic details. Balancing them is critical for de-raining methods. We propose an end-to-end network, called dual-task de-raining network (DTDN), consisting of two sub-networks: generative adversarial network (GAN) and convolutional neural network (CNN), to remove rain streaks via coordinating the two mutually exclusive objectives self-adaptively. DTDN-GAN is mainly used to …
This paper addresses the problem of single image de-raining, that is, the task of recovering clean and rain-free background scenes from a single image obscured by a rainy artifact. Although recent advances adopt real-world time-lapse data to overcome the need for paired rain-clean images, they are limited to fully exploit the time-lapse data. The main cause is that, in terms of network architectures, they could not capture long-term rain streak information in the time-lapse data during training owing to the lack of memory components. To address this problem, we propose a novel network architec…
Rain removal in images is an important task in computer vision filed and attracting attentions of more and more people. In this paper, we address a non-trivial issue of removing visual effect of rain streak from a single image. Differing from existing work, our method combines various semantic constraint task in a proposed multi-task regression model for rain removal. These tasks reinforce the model's capabilities from the content, edge-aware, and local texture similarity respectively. To further improve the performance of multi-task learning, we also present two simple but powerful dynamic we…
Zhejiang University（浙江大学）；Henan University of Science and Technology（河南科技大学）
Few researches have been proposed specifically for real-time semantic segmentation in rainy environments. However, the demand in this area is huge and it is challenging for lightweight networks. Therefore, this paper proposes a lightweight network which is specially designed for the foreground segmentation in rainy environments, named De-raining Semantic Segmentation Network (DRSNet). By analyzing the characteristics of raindrops, the MultiScaleSE Block is targetedly designed to encode the input image, it uses multi-scale dilated convolutions to increase the receptive field, and SE attention m…
University of Electronic Science and Technology of China（电子科技大学）；The University of Adelaide（阿德莱德大学）；Shanghai Jiao Tong University（上海交通大学）；Beijing Jiao Tong University（北京交通大学）
Rain removal in images/videos is still an important task in computer vision field and attracting attentions of more and more people. Traditional methods always utilize some incomplete priors or filters (e.g. guided filter) to remove rain effect. Deep learning gives more probabilities to better solve this task. However, they remove rain either by evaluating background from rainy image directly or learning a rain residual first then subtracting the residual to obtain a clear background. No other models are used in deep learning based de-raining methods to remove rain and obtain other information…
We presented a method for improving computer vision tasks on images affected by adverse weather conditions, including distortions caused by adherent raindrops. Overcoming the challenge of applying computer vision to images affected by adverse weather conditions is essential for autonomous vehicles utilizing RGB cameras. For this purpose, we trained an appropriate generative adversarial network and showed that it was effective at removing the effect of the distortions, in the context of image reconstruction and computer vision tasks. We showed that object recognition, a vital task for autonomou…
Wuhan University of Science and Technology（武汉科技大学）
It is challenging to remove rain-steaks from a single rainy image because the rain steaks are spatially varying in the rainy image. This problem is studied in this paper by combining conventional image processing techniques and deep learning based techniques. An improved weighted guided image filter (iWGIF) is proposed to extract high frequency information from a rainy image. The high frequency information mainly includes rain steaks and noise, and it can guide the rain steaks aware deep convolutional neural network (RSADCNN) to pay more attention to rain steaks. The efficiency and explain-abi…
This paper deals with impulse noise removal from color images. The proposed noise removal algorithm employs a novel approach with morphological filtering for color image denoising; that is, detection of corrupted pixels and removal of the detected noise by means of morphological filtering. With the help of computer simulation we show that the proposed algorithm can effectively remove impulse noise. The performance of the proposed algorithm is compared in terms of image restoration metrics and processing speed with that of common successful algorithms.
Detecting the edges of objects within images is critical for quality image processing. We present an edge-detecting technique that uses morphological amoebas that adjust their shape based on variation in image contours. We evaluate the method both quantitatively and qualitatively for edge detection of images, and compare it to classic morphological methods. Our amoeba-based edge-detection system performed better than the classic edge detectors.
The object recognition is a complex problem in the image processing. Mathematical morphology is Shape oriented operations, that simplify image data, preserving their essential shape characteristics and eliminating irrelevancies. This paper briefly describes morphological operators using hypergraph and its applications for thinning algorithms. The morphological operators using hypergraph method is used to preventing errors and irregularities in skeleton, and is an important step recognizing line objects. The morphological operators using hypergraph such as dilation, erosion, opening, closing is…
Mathematical morphology (MM) helps to describe and analyze shapes using set theory. MM can be effectively applied to binary images which are treated as sets. Basic morphological operators defined can be used as an effective tool in image processing. Morphological operators are also developed based on graph and hypergraph. These operators have found better performance and applications in image processing. Bino et al. [8], [9] developed the theory of morphological operators on hypergraph. A hypergraph structure is considered and basic morphological operation erosion/dilation is defined. Several …
This paper deals with enhancement of images with poor contrast and detection of background. Proposes a frame work which is used to detect the background in images characterized by poor contrast. Image enhancement has been carried out by the two methods based on the Weber's law notion. The first method employs information from image background analysis by blocks, while the second transformation method utilizes the opening operation, closing operation, which is employed to define the multi-background gray scale images. The complete image processing is done using MATLAB simulation model. Finally,…
Fujian Key Laboratory of Sensing and Computing for Smart City, School of Information Science and Engineering, Xiamen University（厦门大学信息科学与技术学院福建省智慧城市感知与计算重点实验室）；Columbia University（哥伦比亚大学）
We introduce a deep network architecture called DerainNet for removing rain streaks from an image. Based on the deep convolutional neural network (CNN), we directly learn the mapping relationship between rainy and clean image detail layers from data. Because we do not possess the ground truth corresponding to real-world rainy images, we synthesize images with rain for training. In contrast to other common strategies that increase depth or breadth of the network, we use image processing domain knowledge to modify the objective function and improve deraining with a modestly-sized CNN. Specifical…
Laval University（拉瓦尔大学）；University of Pennsylvania（宾夕法尼亚大学）
Multiscale shape skeletonization on pixel adjacency graphs is an advanced intriguing research subject in the field of image processing, computer vision and data mining. The previous works in this area almost focused on the graph vertices. We proposed novel structured based graph morphological transformations based on edges opposite to the current node based transformations and used them for deploying skeletonization and reconstruction of infrared thermal images represented by graphs. The advantage of this method is that many widely used path based approaches become available within this defini…
University of Technology Sydney（悉尼科技大学）；University of Sydney（悉尼大学）；Université Paris-Est（巴黎东部大学）
In this paper, we propose a principled Perceptual Adversarial Networks (PAN) for image-to-image transformation tasks. Unlike existing application-specific algorithms, PAN provides a generic framework of learning mapping relationship between paired images (Fig. 1), such as mapping a rainy image to its de-rained counterpart, object edges to its photo, semantic labels to a scenes image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the image transformation network T and the discriminative network D. Through combining the generative adversarial loss and t…
Slender Object Scene Segmentation in Remote Sensing Image Based on Learnable Morphological Skeleton with Segment Anything Model
School of Mathematical Sciences, Beijing Normal University（北京师范大学数学科学学院）；State Key Laboratory of Remote Sensing Science, Faculty of Geographical Science, Beijing Normal University（北京师范大学地理科学学部遥感科学国家重点实验室）
Instead of directly utilizing an observed image including some outliers, noise or intensity inhomogeneity, the use of its ideal value (e.g. noise-free image) has a favorable impact on clustering. Hence, the accurate estimation of the residual (e.g. unknown noise) between the observed image and its ideal value is an important task. To do so, we propose an $\ell_0$ regularization-based Fuzzy $C$-Means (FCM) algorithm incorporating a morphological reconstruction operation and a tight wavelet frame transform. To achieve a sound trade-off between detail preservation and noise suppression, morpholog…
Shaanxi University of Science and Technology（陕西科技大学）；Northwestern Polytechnical University（西北工业大学）；The University of Sydney（悉尼大学）；Shaanxi Normal University（陕西师范大学）；Brunel University London（伦敦布鲁内尔大学）；Tongji University（同济大学）
Morphological reconstruction (MR) is often employed by seeded image segmentation algorithms such as watershed transform and power watershed as it is able to filter seeds (regional minima) to reduce over-segmentation. However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed. In this paper, a novel adaptive morphological reconstruction (AMR) operation is proposed that has three advantages. Firstly, AMR can adaptively filter useless seeds while preserving…
WBCAtt+: Fine-Grained Pixel-Level Morphological Annotations for White Blood Cell Images
Rapid-Rich Object Search (ROSE) Lab, School of Electrical and Electronic Engineering, Nanyang Technological University（快速丰富目标搜索（ROSE）实验室，电气与电子工程学院，南洋理工大学）；Shanghai University of Finance and Economics（上海财经大学）
Indian Statistical Institute（印度统计研究所）；Jadavpur University（贾达普大学）
This paper proposes a technique for automatic face recognition using integrated multiple feature sets extracted from the significant blocks of a gradient image. We discuss about the use of novel morphological, local directional pattern (LDP) and gray-level co-occurrence matrix GLCM based feature extraction technique to recognize human faces. Firstly, the new morphological features i.e., features based on number of runs of pixels in four directions (N,NE,E,NW) are extracted, together with the GLCM based statistical features and LDP features that are less sensitive to the noise and non-monotonic…
A general framework of spatio-spectral segmentation for multi-spectral images is introduced in this paper. The method is based on classification-driven stochastic watershed (WS) by Monte Carlo simulations, and it gives more regular and reliable contours than standard WS. The present approach is decomposed into several sequential steps. First, a dimensionality-reduction stage is performed using the factor-correspondence analysis method. In this context, a new way to select the factor axes (eigenvectors) according to their spatial information is introduced. Then, a spectral classification produc…
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Korea University（高丽大学）；Shanghai United Imaging Intelligence Co., Ltd.（上海联影智能科技有限公司）
Objective: Deformable brain MR image registration is challenging due to large inter-subject anatomical variation. For example, the highly complex cortical folding pattern makes it hard to accurately align corresponding cortical structures of individual images. In this paper, we propose a novel deep learning way to simplify the difficult registration problem of brain MR images. Methods: We train a morphological simplification network (MS-Net), which can generate a "simple" image with less anatomical details based on the "complex" input. With MS-Net, the complexity of the fixed image or the movi…
Although spatial information of images usually enhance the robustness of the Fuzzy C-Means (FCM) algorithm, it greatly increases the computational costs for image segmentation. To achieve a sound trade-off between the segmentation performance and the speed of clustering, we come up with a Kullback-Leibler (KL) divergence-based FCM algorithm by incorporating a tight wavelet frame transform and a morphological reconstruction operation. To enhance FCM's robustness, an observed image is first filtered by using the morphological reconstruction. A tight wavelet frame system is employed to decompose …
International Prevention Research Institute（国际预防研究所）
We introduce a simple expression for the map of Asplund's distances with the multiplicative Logarithmic Image Processing (LIP) law. It is a difference between a morphological dilation and a morphological erosion with an additive structuring function which corresponds to a morphological gradient.
MorphXAI: An Explainable Framework for Morphological Analysis of Parasites in Blood Smear Images
Department of Computer Science and Engineering, University of Texas at Arlington（德克萨斯大学阿灵顿分校计算机科学与工程系）；Department of Medicine, Division of Infectious Diseases, NYU Grossman School of Medicine（纽约大学格罗斯曼医学院医学系感染病科）
De Montfort University（德蒙福特大学）；Instituto de Investigación Biomédica de Málaga – IBIMA（马拉加生物医学研究所）；Costa Rica Institute of Technology（哥斯达黎加理工学院）；University of Costa Rica（哥斯达黎加大学）；Rovira i Virgili University（罗维拉-威尔吉利大学）；University of Malaga（马拉加大学）
Cell instance segmentation in fluorescence microscopy images is becoming essential for cancer dynamics and prognosis. Data extracted from cancer dynamics allows to understand and accurately model different metabolic processes such as proliferation. This enables customized and more precise cancer treatments. However, accurate cell instance segmentation, necessary for further cell tracking and behavior analysis, is still challenging in scenarios with high cell concentration and overlapping edges. Within this framework, we propose a novel cell instance segmentation approach based on the well-know…
University of Electronic Science and Technology of China（电子科技大学）；The University of Adelaide（阿德莱德大学）
Removing rain effects from an image is of importance for various applications such as autonomous driving, drone piloting, and photo editing. Conventional methods rely on some heuristics to handcraft various priors to remove or separate the rain effects from an image. Recent deep learning models are proposed to learn end-to-end methods to complete this task. However, they often fail to obtain satisfactory results in many realistic scenarios, especially when the observed images suffer from heavy rain. Heavy rain brings not only rain streaks but also haze-like effect caused by the accumulation of…
University for Health Sciences, Medical Informatics and Technology (UMIT)（健康科学、医学信息与技术大学（UMIT））
Morphological amoebas are image-adaptive structuring elements for morphological and other local image filters introduced by Lerallut et al. Their construction is based on combining spatial distance with contrast information into an image-dependent metric. Amoeba filters show interesting parallels to image filtering methods based on partial differential equations (PDEs), which can be confirmed by asymptotic equivalence results. In computing amoebas, graph structures are generated that hold information about local image texture. This paper reviews and summarises the work of the author and his co…
A variety of computer vision applications depend on the efficiency of image matching algorithms used. Various descriptors are designed to detect and match features in images. Deployment of this algorithms in mobile applications creates a need for low computation time. Binary descriptors requires less computation time than float-point based descriptors because of the intensity comparison between pairs of sample points and comparing after creating a binary string. In order to decrease time complexity, quality of keypoints matched is often compromised. We propose a keypoint descriptor named Morph…
Universidad Politécnica de Puebla（普埃布拉理工大学）；Instituto de Investigaciones en Materiales, Universidad Nacional Autónoma de México（墨西哥国立自治大学材料研究所）；Instituto Nacional de Astrofísica, Óptica, Electrónica（国家天体物理、光学和电子学研究所）
In this report we present experimental results using \emph{Haussdorf-Besicovich} fractal dimension for performing morphological galaxy classification. The fractal dimension is a topological, structural and spatial property that give us information about the space were an object lives. We have calculated the fractal dimension value of the main types of galaxies: ellipticals, spirals and irregulars; and we use it as a feature for classifying them. Also, we have performed an image analysis process in order to standardize the galaxy images, and we have used principal component analysis to obtain t…
Shenzhen University（深圳大学）；Tencent Holdings Limited（腾讯控股有限公司）；Nvidia Corporation（英伟达公司）；National Institutes of Health（美国国立卫生研究院）
Fine-grained classification of cervical cells into different abnormality levels is of great clinical importance but remains very challenging. Contrary to traditional classification methods that rely on hand-crafted or engineered features, convolution neural network (CNN) can classify cervical cells based on automatically learned deep features. However, CNN in previous studies do not involve cell morphological information, and it is unknown whether morphological features can be directly modeled by CNN to classify cervical cells. This paper presents a CNN-based method that combines cell image ap…
Centre de Morphologie Mathématique（数学形态学中心）；Ecole des Mines de Paris（巴黎高等矿业学院）
The present paper develops a general methodology for the morphological segmentation of hyperspectral images, i.e., with an important number of channels. This approach, based on watershed, is composed of a spectral classification to obtain the markers and a vectorial gradient which gives the spatial information. Several alternative gradients are adapted to the different hyperspectral functions. Data reduction is performed either by Factor Analysis or by model fitting. Image segmentation is done on different spaces: factor space, parameters space, etc. On all these spaces the spatial/spectral se…
The shapes and morphology of the organs and tissues are important prior knowledge in medical imaging recognition and segmentation. The morphological operation is a well-known method for morphological feature extraction. As the morphological operation is performed well in hand-crafted image segmentation techniques, it is also promising to design an approach to approximate morphological operation in the convolutional networks. However, using the traditional convolutional neural network as a black-box is usually hard to specify the morphological operation action. Here, we introduced a 3D morpholo…
An Approach to Colour Morphological Supremum Formation using the LogSumExp Approximation
Brandenburg University of Technology Cottbus-Senftenberg（科特布斯-森夫滕贝格勃兰登堡工业大学）；Forschungszentrum Jülich GmbH（于利希研究中心）；University of Applied Sciences Aachen（亚琛应用科学大学）；UMIT TIROL – Private University for Health Sciences and Health Technology（UMIT蒂罗尔——私立健康科学与健康技术大学）
Islamic University of Technology（伊斯兰科技大学）；University of South Florida（南佛罗里达大学）
To guide surgical and medical treatment X-ray images have been used by physicians in every modern healthcare organization and hospitals. Doctor's evaluation process and disease identification in the area of skeletal system can be performed in a faster and efficient way with the help of X-ray imaging technique as they can depict bone structure painlessly. This paper presents an efficient contrast enhancement technique using morphological operators which will help to visualize important bone segments and soft tissues more clearly. Top-hat and Bottom-hat transform are utilized to enhance the imag…
