Despite advances in deep learning, robustness under domain shift remains a major bottleneck in medical imaging settings. Findings on natural images suggest that deep neural models can show a strong textural bias when carrying out image classification tasks. In this thorough empirical study, we draw inspiration from findings on natural images and investigate ways in which addressing the textural bias phenomenon could bring up the robustness of deep segmentation models when applied to three-dimensional (3D) medical data. To achieve this, publicly available MRI scans from the Developing Human Connectome Project are used to study ways in which simulating textural noise can help train robust models in a complex semantic segmentation task. We contribute an extensive empirical investigation consisting of 176 experiments and illustrate how applying specific types of simulated textural noise prior to training can lead to texture invariant models, resulting in improved robustness when segmenting scans corrupted by previously unseen noise types and levels.
University of Isfahan（伊斯法罕大学）；Duke University（杜克大学）；Duke University Medical Center（杜克大学医学中心）
Optical coherence tomography (OCT) helps ophthalmologists assess macular edema, accumulation of fluids, and lesions at microscopic resolution. Quantification of retinal fluids is necessary for OCT-guided treatment management, which relies on a precise image segmentation step. As manual analysis of retinal fluids is a time-consuming, subjective, and error-prone task, there is increasing demand for fast and robust automatic solutions. In this study, a new convolutional neural architecture named RetiFluidNet is proposed for multi-class retinal fluid segmentation. The model benefits from hierarchi…
Multi-modal Liver Segmentation and Fibrosis Staging Using Real-world MRI Images
Multiscale X-ray Imaging (MXI) Lab, Department of Mechanical Engineering, University College London（多尺度X射线成像实验室，机械工程系，伦敦大学学院）；Centre for Clinical Brain Sciences, University of Edinburgh（临床脑科学中心，爱丁堡大学）；MRC Weatherall Institute of Molecular Medicine, University of Oxford（MRC韦伯尔分子医学研究所，牛津大学）
Indian Institute of Science Education and Research, Pune（印度科学教育与研究学院浦那分校）；DeepTek Inc（DeepTek公司）
Deep learning semantic segmentation algorithms can localise abnormalities or opacities from chest radiographs. However, the task of collecting and annotating training data is expensive and requires expertise which remains a bottleneck for algorithm performance. We investigate the effect of image augmentations on reducing the requirement of labelled data in the semantic segmentation of chest X-rays for pneumonia detection. We train fully convolutional network models on subsets of different sizes from the total training data. We apply a different image augmentation while training each model and …
Phi-SegNet: Phase-Integrated Supervision for Medical Image Segmentation
mHealth Lab, Department of Biomedical Engineering, Bangladesh University of Engineering and Technology（孟加拉工程与技术大学生物医学工程系mHealth实验室）；Department of Electrical and Computer Engineering, Johns Hopkins University（约翰霍普金斯大学电气与计算机工程系）；Center for Bioengineering Innovation and Design, Department of Biomedical Engineering, Johns Hopkins University（约翰霍普金斯大学生物工程创新与设计中心，生物医学工程系）
Optical coherence tomography (OCT) is a powerful and noninvasive method for retinal imaging. In this paper, we introduce a fast segmentation method based on a new variant of spectral graph theory named diffusion maps. The research is performed on spectral domain (SD) OCT images depicting macular and optic nerve head appearance. The presented approach does not require edge-based image information and relies on regional image texture. Consequently, the proposed method demonstrates robustness in situations of low image contrast or poor layer-to-layer image gradients. Diffusion mapping is applied …
Medical University of Vienna（维也纳医科大学）；Medical University Vienna（维也纳医科大学）
Optical coherence tomography (OCT) has become the most important imaging modality in ophthalmology. A substantial amount of research has recently been devoted to the development of machine learning (ML) models for the identification and quantification of pathological features in OCT images. Among the several sources of variability the ML models have to deal with, a major factor is the acquisition device, which can limit the ML model's generalizability. In this paper, we propose to reduce the image variability across different OCT devices (Spectralis and Cirrus) by using CycleGAN, an unsupervis…
Northwestern University（西北大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of San Francisco（旧金山大学）；Imperial College London（帝国理工学院）；Univ. Paris Diderot, Sorbonne Paris Cité（巴黎第七大学（索邦巴黎大学））；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；San Diego State University（圣地亚哥州立大学）
In the fields of nanoscience and nanotechnology, it is important to be able to functionalize surfaces chemically for a wide variety of applications. Scanning tunneling microscopes (STMs) are important instruments in this area used to measure the surface structure and chemistry with better than molecular resolution. Self-assembly is frequently used to create monolayers that redefine the surface chemistry in just a single-molecule-thick layer. Indeed, STM images reveal rich information about the structure of self-assembled monolayers since they convey chemical and physical properties of the stud…
High accuracy coastline/shoreline extraction from SAR imagery is a crucial step in a number of maritime and coastal monitoring applications. We present a method based on image segmentation using the Generalised Gamma Mixture Model superpixel algorithm (MISP-GGD). MISP-GGD produces superpixels adhering with great accuracy to object edges in the image, such as the coastline. Unsupervised clustering of the generated superpixels according to textural and radiometric features allows for generation of a land/water mask from which a highly accurate coastline can be extracted. We present results of ou…
Reducing Annotation Burden for Femoral Cartilage Segmentation in Knee MRI via Cross-Sequence Transfer Learning
School of Electronic Engineering, Dublin City University（都柏林城市大学电子工程学院）；Bioengineering and Computing Laboratory, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所生物工程与计算实验室）；Dipartimento di Scienze Mediche e Chirurgiche (DIMEC), Alma Mater Studiorum – Università di Bologna（博洛尼亚大学医学与外科科学系（DIMEC））；Diagnostic and Interventional Radiology, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所诊断与介入放射学）；Department of Biomedical and Neuromotor Sciences, University of Bologna（博洛尼亚大学生物医学与神经运动科学系）；nd Orthopedics and Trauma Unit, IRCCS Istituto Ortopedico Rizzoli（里扎利骨科研究所第二骨科与创伤单元）；Independent Researcher（独立研究员）
The need for fast acquisition and automatic analysis of MRI data is growing in the age of big data. Although compressed sensing magnetic resonance imaging (CS-MRI) has been studied to accelerate MRI by reducing k-space measurements, in current CS-MRI techniques MRI applications such as segmentation are overlooked when doing image reconstruction. In this paper, we test the utility of CS-MRI methods in automatic segmentation models and propose a unified deep neural network architecture called SegNetMRI which we apply to the combined CS-MRI reconstruction and segmentation problem. SegNetMRI is bu…
It is a common practice in multimodal medical imaging to undersample the anatomically-derived segmentation images to measure the mean activity of a co-acquired functional image. This practice avoids the resampling-related Gibbs effect that would occur in oversampling the functional image. As sides effect, waste of time and efforts are produced since the anatomical segmentation at full resolution is performed in many hours of computations or manual work. In this work we explain the commonly-used resampling methods and give errors bound in the cases of continuous and discontinuous signals. Then …
In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network where the encoder and decoder sub-networks are connected through a series of nested, dense skip pathways. The re-designed skip pathways aim at reducing the semantic gap between the feature maps of the encoder and decoder sub-networks. We argue that the optimizer would deal with an easier learning task when the feature maps from the decoder and encoder networks are semantically similar. We have evaluated UNet++ in compari…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Reducing outcome variance is an essential task in deep learning based medical image analysis. Bootstrap aggregating, also known as bagging, is a canonical ensemble algorithm for aggregating weak learners to become a strong learner. Random forest is one of the most powerful machine learning algorithms before deep learning era, whose superior performance is driven by fitting bagged decision trees (weak learners). Inspired by the random forest technique, we propose a simple bagging ensemble deep segmentation (BEDs) method to train multiple U-Nets with partial training data to segment dense nuclei…
University of California, Los Angeles（加州大学洛杉矶分校）；NVIDIA（英伟达）
Fully convolutional neural networks (CNNs) have proven to be effective at representing and classifying textural information, thus transforming image intensity into output class masks that achieve semantic image segmentation. In medical image analysis, however, expert manual segmentation often relies on the boundaries of anatomical structures of interest. We propose boundary aware CNNs for medical image segmentation. Our networks are designed to account for organ boundary information, both by providing a special network edge branch and edge-aware loss terms, and they are trainable end-to-end. W…
Pixel-level annotation demands expensive human efforts and limits the performance of deep networks that usually benefits from more such training data. In this work we aim to achieve high quality instance and semantic segmentation results over a small set of pixel-level mask annotations and a large set of box annotations. The basic idea is exploring detection models to simplify the pixel-level supervised learning task and thus reduce the required amount of mask annotations. Our architecture, named DASNet, consists of three modules: detection, attention, and segmentation. The detection module de…
Uppsala University（乌普萨拉大学）；SciLifeLab（瑞典国家生命科学实验室）
There is a limitation in the size of an image that can be processed using computationally demanding methods such as e.g. Convolutional Neural Networks (CNNs). Some imaging modalities - notably biological and medical - can result in images up to a few gigapixels in size, meaning that they have to be divided into smaller parts, or patches, for processing. However, when performing image segmentation, this may lead to undesirable artefacts, such as edge effects in the final re-combined image. We introduce windowing methods from signal processing to effectively reduce such edge effects. With the as…
The digitization of historical maps enables the study of ancient, fragile, unique, and hardly accessible information sources. Main map features can be retrieved and tracked through the time for subsequent thematic analysis. The goal of this work is the vectorization step, i.e., the extraction of vector shapes of the objects of interest from raster images of maps. We are particularly interested in closed shape detection such as buildings, building blocks, gardens, rivers, etc. in order to monitor their temporal evolution. Historical map images present significant pattern recognition challenges.…
University of Alberta（阿尔伯塔大学）；CHU Hospital（里昂大学医疗中心）
Intravascular Ultrasound (IVUS) is an intra-operative imaging modality that facilitates observing and appraising the vessel wall structure of the human coronary arteries. Segmentation of arterial wall boundaries from the IVUS images is not only crucial for quantitative analysis of the vessel walls and plaque characteristics, but is also necessary for generating 3D reconstructed models of the artery. The aim of this study is twofold. Firstly, we investigate the feasibility of using a recently proposed region detector, namely Extremal Region of Extremum Level (EREL) to delineate the luminal and …
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Northeastern University（东北大学）；Caliber Imaging and Diagnostics（Caliber影像与诊断公司）；SUNY Downstate Medical Center（纽约州立大学下州医学中心）；Skin Medical Research Diagnostics, P.L.L.C.（皮肤医学研究诊断有限责任公司）；University of Modena and Reggio Emilia（摩德纳-雷焦艾米利亚大学）
In-vivo optical microscopy is advancing into routine clinical practice for non-invasively guiding diagnosis and treatment of cancer and other diseases, and thus beginning to reduce the need for traditional biopsy. However, reading and analysis of the optical microscopic images are generally still qualitative, relying mainly on visual examination. Here we present an automated semantic segmentation method called "Multiscale Encoder-Decoder Network (MED-Net)" that provides pixel-wise labeling into classes of patterns in a quantitative manner. The novelty in our approach is the modeling of textura…
Due to the limitation of available labeled data, medical image segmentation is a challenging task for deep learning. Traditional data augmentation techniques have been shown to improve segmentation network performances by optimizing the usage of few training examples. However, current augmentation approaches for segmentation do not tackle the strong texture bias of convolutional neural networks, observed in several studies. This work shows on the MoNuSeg dataset that style augmentation, which is already used in classification tasks, helps reducing texture over-fitting and improves segmentation…
The main focus of this work is a novel framework for the joint reconstruction and segmentation of parallel MRI (PMRI) brain data. We introduce an image domain deep network for calibrationless recovery of undersampled PMRI data. The proposed approach is the deep-learning (DL) based generalization of local low-rank based approaches for uncalibrated PMRI recovery including CLEAR [6]. Since the image domain approach exploits additional annihilation relations compared to k-space based approaches, we expect it to offer improved performance. To minimize segmentation errors resulting from undersamplin…
The Hausdorff Distance (HD) is widely used in evaluating medical image segmentation methods. However, existing segmentation methods do not attempt to reduce HD directly. In this paper, we present novel loss functions for training convolutional neural network (CNN)-based segmentation methods with the goal of reducing HD directly. We propose three methods to estimate HD from the segmentation probability map produced by a CNN. One method makes use of the distance transform of the segmentation boundary. Another method is based on applying morphological erosion on the difference between the true an…
Prostate cancer is one of the most prevalent cancers worldwide. One of the key factors in reducing its mortality is based on early detection. The computer-aided diagnosis systems for this task are based on the glandular structural analysis in histology images. Hence, accurate gland detection and segmentation is crucial for a successful prediction. The methodological basis of this work is a prostate gland segmentation based on U-Net convolutional neural network architectures modified with residual and multi-resolution blocks, trained using data augmentation techniques. The residual configuratio…
Promptable segmentation with region exploration enables minimal-effort expert-level prostate cancer delineation
UCL Hawkes Institute（UCL Hawkes Institute；医学物理与生物医学工程系）；Department of Medical Physics and Biomedical Engineering（大学学院伦敦）；University College London（医学成像中心）；Centre of Medical Imaging（肿瘤学系）；Department of Oncology（阿迦·柯尔大学医院）；Aga Khan University Hospital（生物工程中心；工程与材料科学学院）；Centre for Bioengineering；School of Engineering and Materials Science
Detecting and identifying objects in satellite images is a very challenging task: objects of interest are often very small and features can be difficult to recognize even using very high resolution imagery. For most applications, this translates into a trade-off between recall and precision. We present here a dedicated method to detect and identify aircraft, combining two very different convolutional neural networks (CNNs): a segmentation model, based on a modified U-net architecture, and a detection model, based on the RetinaNet architecture. The results we present show that this combination …
EndoCaver: Handling Fog, Blur and Glare in Endoscopic Images via Joint Deblurring-Segmentation
1 CyPhi ( ) AI Research Lab, School of IT, Monash University, Malaysia Campus；2 Department of Electronic \& Computer Engineering, The Hong Kong University of Science；3 Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Leipzig University（莱比锡大学）；IfADo - Leibniz Research Centre for Working Environment and Human Factors（莱布尼茨工作环境与人因研究中心）；INRIA Paris（法国国家信息与自动化研究所巴黎中心）；Sorbonne Université（索邦大学）
We present a novel approach that combines machine learning based interactive image segmentation using supervoxels with a clustering method for the automated identification of similarly colored images in large data sets which enables a guided reuse of classifiers. Our approach solves the problem of significant color variability prevalent and often unavoidable in biological and medical images which typically leads to deteriorated segmentation and quantification accuracy thereby greatly reducing the necessary training effort. This increase in efficiency facilitates the quantification of much larg…
National Research Council, Institute for the Application of Calculus（国家研究委员会，计算应用研究所）
Skin lesion segmentation is one of the crucial steps for an efficient non-invasive computer-aided early diagnosis of melanoma. This paper investigates how color information, besides saliency, can be used to determine the pigmented lesion region automatically. Unlike most existing segmentation methods using only the saliency in order to discriminate against the skin lesion from the surrounding regions, we propose a novel method employing a binarization process coupled with new perceptual criteria, inspired by the human visual perception, related to the properties of saliency and color of the in…
The interpretation of seismic data is vital for characterizing sediments' shape in areas of geological study. In seismic interpretation, deep learning becomes useful for reducing the dependence on handcrafted facies segmentation geometry and the time required to study geological areas. This work presents a Deep Neural Network for Facies Segmentation (DNFS) to obtain state-of-the-art results for seismic facies segmentation. DNFS is trained using a combination of cross-entropy and Jaccard loss functions. Our results show that DNFS obtains highly detailed predictions for seismic facies segmentati…
In recent years, deep learning techniques (e.g., U-Net, DeepLab) have achieved tremendous success in image segmentation. The performance of these models heavily relies on high-quality ground truth segment labels. Unfortunately, in many real-world problems, ground truth segment labels often have geometric annotation errors due to manual annotation mistakes, GPS errors, or visually interpreting background imagery at a coarse resolution. Such location errors will significantly impact the training performance of existing deep learning algorithms. Existing research on label errors either models gro…
Image Segmentation has been an active field of research as it has a wide range of applications, ranging from automated disease detection to self-driving cars. In recent years, various research papers proposed different loss functions used in case of biased data, sparse segmentation, and unbalanced dataset. In this paper, we introduce SemSegLoss, a python package consisting of some of the well-known loss functions widely used for image segmentation. It is developed with the intent to help researchers in the development of novel loss functions and perform an extensive set of experiments on model…
Trained using only image class label, deep weakly supervised methods allow image classification and ROI segmentation for interpretability. Despite their success on natural images, they face several challenges over histology data where ROI are visually similar to background making models vulnerable to high pixel-wise false positives. These methods lack mechanisms for modeling explicitly non-discriminative regions which raises false-positive rates. We propose novel regularization terms, which enable the model to seek both non-discriminative and discriminative regions, while discouraging unbalanc…
One of the major goals of tomorrow's agriculture is to increase agricultural productivity but above all the quality of production while significantly reducing the use of inputs. Meeting this goal is a real scientific and technological challenge. Smart farming is among the promising approaches that can lead to interesting solutions for vineyard management and reduce the environmental impact. Automatic vine disease detection can increase efficiency and flexibility in managing vineyard crops, while reducing the chemical inputs. This is needed today more than ever, as the use of pesticides is comi…
TOM: An Open-Source Tongue Segmentation Method with Multi-Teacher Distillation and Task-Specific Data Augmentation
University of Missouri（密苏里大学）；Northwestern University（西北大学）；Shanghai University of Traditional Chinese Medicine（上海中医药大学）；Community Health Service Center Shanghai Pudong New Area（上海浦东新区社区卫生服务中心）；University of Electronic Science and Technology of China（电子科技大学）
