In this paper, we propose a new image instance segmentation method that segments individual glands (instances) in colon histology images. This is a task called instance segmentation that has recently become increasingly important. The problem is challenging since not only do the glands need to be segmented from the complex background, they are also required to be individually identified. Here we leverage the idea of image-to-image prediction in recent deep learning by building a framework that automatically exploits and fuses complex multichannel information, regional, location and boundary patterns in gland histology images. Our proposed system, deep multichannel framework, alleviates heavy feature design due to the use of convolutional neural networks and is able to meet multifarious requirement by altering channels. Compared to methods reported in the 2015 MICCAI Gland Segmentation Challenge and other currently prevalent methods of instance segmentation, we observe state-of-the-art results based on a number of evaluation metrics.
In this paper, we propose a new image instance segmentation method that segments individual glands (instances) in colon histology images. This is a task called instance segmentation that has recently become increasingly important. The problem is challenging since not only do the glands need to be segmented from the complex background, they are also required to be individually identified. Here we leverage the idea of image-to-image prediction in recent deep learning by building a framework that automatically exploits and fuses complex multichannel information, regional and boundary patterns, wi…
Objective: A new image instance segmentation method is proposed to segment individual glands (instances) in colon histology images. This process is challenging since the glands not only need to be segmented from a complex background, they must also be individually identified. Methods: We leverage the idea of image-to-image prediction in recent deep learning by designing an algorithm that automatically exploits and fuses complex multichannel information - regional, location, and boundary cues - in gland histology images. Our proposed algorithm, a deep multichannel framework, alleviates heavy fe…
University of Warwick（华威大学）；The Chinese University of Hong Kong（香港中文大学）；University Hospitals Coventry and Warwickshire（考文垂与华威大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
The analysis of glandular morphology within colon histopathology images is an important step in determining the grade of colon cancer. Despite the importance of this task, manual segmentation is laborious, time-consuming and can suffer from subjectivity among pathologists. The rise of computational pathology has led to the development of automated methods for gland segmentation that aim to overcome the challenges of manual segmentation. However, this task is non-trivial due to the large variability in glandular appearance and the difficulty in differentiating between certain glandular and non-…
Multi-Sequence Parotid Gland Lesion Segmentation via Expert Text-Guided Segment Anything Model
Department of Radiology, Sun Yat-sen Memorial Hospital, Sun Yat-sen University（中山大学附属第一医院放射科）；Department of Radiology, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Sun Yat-Sen University Cancer Center（中山大学南方学院肿瘤学国家重点实验室、癌症医学创新中心、中山大学肿瘤中心放射科）；Department of Radiology, Shantou Central Hospital（汕头市中心医院放射科）；Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Medical Research Center, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University（广东省恶性肿瘤表观遗传学与基因调控重点实验室、中山大学附属第一医院医学研究中心、中山大学）；Department of Radiology, Shenshan Medical Center, Sun Yat-sen Memorial Hospital, Sun Yat-sen University（中山大学附属山田医院放射科）
Institute of Biophysics, Center for Physiological Medicine, Medical University of Graz（格拉茨医科大学医学生物物理研究所）；Institute of Neuroinformatics, University of Zurich and ETH Zurich（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）；Institute for Computer Graphics and Vision, Graz University of Technology（格拉茨技术大学计算机图形与视觉研究所）；Ludwig Boltzmann Institute for Clinical Forensic Imaging（路德维希·玻尔兹曼临床法医影像研究所）；BioTechMed-Graz（格拉茨生物技术医学联盟）
Segmentation of histopathology sections is an ubiquitous requirement in digital pathology and due to the large variability of biological tissue, machine learning techniques have shown superior performance over standard image processing methods. As part of the GlaS@MICCAI2015 colon gland segmentation challenge, we present a learning-based algorithm to segment glands in tissue of benign and malignant colorectal cancer. Images are preprocessed according to the Hematoxylin-Eosin staining protocol and two deep convolutional neural networks (CNN) are trained as pixel classifiers. The CNN predictions…
Gland segmentation is a critical step to quantitatively assess the morphology of glands in histopathology image analysis. However, it is challenging to separate densely clustered glands accurately. Existing deep learning-based approaches attempted to use contour-based techniques to alleviate this issue but only achieved limited success. To address this challenge, we propose a novel topology-aware network (TA-Net) to accurately separate densely clustered and severely deformed glands. The proposed TA-Net has a multitask learning architecture and enhances the generalization of gland segmentation …
The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen Research Institute（香港科技大学深圳研究院）
Developing an AI-assisted gland segmentation method from histology images is critical for automatic cancer diagnosis and prognosis; however, the high cost of pixel-level annotations hinders its applications to broader diseases. Existing weakly-supervised semantic segmentation methods in computer vision achieve degenerative results for gland segmentation, since the characteristics and problems of glandular datasets are different from general object datasets. We observe that, unlike natural images, the key problem with histology images is the confusion of classes owning to morphological homogene…
Automated medical image segmentation plays a key role in quantitative research and diagnostics. Convolutional neural networks based on the U-Net architecture are the state-of-the-art. A key disadvantage is the hard-coding of the receptive field size, which requires architecture optimization for each segmentation task. Furthermore, increasing the receptive field results in an increasing number of weights. Recently, Neural Ordinary Differential Equations (NODE) have been proposed, a new type of continuous depth deep neural network. This framework allows for a dynamic receptive field at a fixed m…
National University of Sciences and Technology（国家科技大学）
Instance segmentation of nuclei and glands in the histology images is an important step in computational pathology workflow for cancer diagnosis, treatment planning and survival analysis. With the advent of modern hardware, the recent availability of large-scale quality public datasets and the community organized grand challenges have seen a surge in automated methods focusing on domain specific challenges, which is pivotal for technology advancements and clinical translation. In this survey, 126 papers illustrating the AI based methods for nuclei and glands instance segmentation published in …
College of Information Engineering, Shenzhen University（深圳大学信息工程学院）；Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University（深圳大学广东省智能信息处理重点实验室）；Histo Pathology Diagnostic Center, Shanghai（上海组织病理学诊断中心）；School of Computer Science, University of Nottingham（诺丁汉大学计算机学院）；School of Medicine, University of Nottingham（诺丁汉大学医学院）
Supervised semantic segmentation normally assumes the test data being in a similar data domain as the training data. However, in practice, the domain mismatch between the training and unseen data could lead to a significant performance drop. Obtaining accurate pixel-wise label for images in different domains is tedious and labor intensive, especially for histopathology images. In this paper, we propose a dual adaptive pyramid network (DAPNet) for histopathological gland segmentation adapting from one stain domain to another. We tackle the domain adaptation problem on two levels: 1) the image-l…
Instance segmentation has attracted recent attention in computer vision and existing methods in this domain mostly have an object detection stage. In this paper, we study the intrinsic challenge of the instance segmentation problem, the presence of a quotient space (swapping the labels of different instances leads to the same result), and propose new methods that are object proposal- and object detection- free. We propose three alternative methods, namely pixel-based affinity mapping, superpixel-based affinity learning, and boundary-based component segmentation, all focusing on performing labe…
Variational multichannel multiclass segmentation using unsupervised lifting with CNNs
University of Innsbruck（因斯布鲁克大学）；VASCage GmbH（VASCage有限责任公司）；MRC-Laboratory of Molecular Biology（英国医学研究委员会分子生物学实验室）；Medical University of Innsbruck（因斯布鲁克医科大学）
Grading of cancer is important to know the extent of its spread. Prior to grading, segmentation of glandular structures is important. Manual segmentation is a time consuming process and is subject to observer bias. Hence, an automated process is required to segment the gland structures. These glands show a large variation in shape size and texture. This makes the task challenging as the glands cannot be segmented using mere morphological operations and conventional segmentation mechanisms. In this project we propose a method which detects the boundary epithelial cells of glands and then a nove…
Mucous glands lesions analysis and assessing of malignant potential of colon polyps are very important tasks of surgical pathology. However, differential diagnosis of colon polyps often seems impossible by classical methods and it is necessary to involve computer methods capable of assessing minimal differences to extend the capabilities of the classical pathology examination. Accurate segmentation of mucous glands from histology images is a crucial step to obtain reliable morphometric criteria for quantitative diagnostic methods. We review major trends in histological images segmentation and …
The morphology of glands has been used routinely by pathologists to assess the malignancy degree of adenocarcinomas. Accurate segmentation of glands from histology images is a crucial step to obtain reliable morphological statistics for quantitative diagnosis. In this paper, we proposed an efficient deep contour-aware network (DCAN) to solve this challenging problem under a unified multi-task learning framework. In the proposed network, multi-level contextual features from the hierarchical architecture are explored with auxiliary supervision for accurate gland segmentation. When incorporated w…
FlowSDF: Flow Matching for Medical Image Segmentation Using Distance Transforms
Institute of Computer Graphics and Vision（计算机图形与视觉研究所）；Graz University of Technology（格拉茨工业大学）；Photogrammetry and Remote Sensing（摄影测量与遥感研究所）；ETH Zurich（苏黎世联邦理工学院）
University of Notre Dame（圣母大学）；The Hong Kong Polytechnic University（香港理工大学）；The Chinese University of Hong Kong（香港中文大学）
In recent years, deep learning (DL) methods have become powerful tools for biomedical image segmentation. However, high annotation efforts and costs are commonly needed to acquire sufficient biomedical training data for DL models. To alleviate the burden of manual annotation, in this paper, we propose a new weakly supervised DL approach for biomedical image segmentation using boxes only annotation. First, we develop a method to combine graph search (GS) and DL to generate fine object masks from box annotation, in which DL uses box annotation to compute a rough segmentation for GS and then GS i…
University Hospital of Marburg (UKGM)（马尔堡大学医院 (UKGM)）
Prostate cancer is the most abundant cancer in men, with over 200,000 expected new cases and around 28,000 deaths in 2012 in the US alone. In this study, the segmentation results for the prostate central gland (PCG) in MR scans are presented. The aim of this research study is to apply a graph-based algorithm to automated segmentation (i.e. delineation) of organ limits for the prostate central gland. The ultimate goal is to apply automated segmentation approach to facilitate efficient MR-guided biopsy and radiation treatment planning. The automated segmentation algorithm used is graph-driven ba…
Deep neural networks have been a prevailing technique in the field of medical image processing. However, the most popular convolutional neural networks (CNNs) based methods for medical image segmentation are imperfect because they model long-range dependencies by stacking layers or enlarging filters. Transformers and the self-attention mechanism are recently proposed to effectively learn long-range dependencies by modeling all pairs of word-to-word attention regardless of their positions. The idea has also been extended to the computer vision field by creating and treating image patches as emb…
Massachusetts Institute of Technology（麻省理工学院）；Skoltech（斯科尔科沃科学技术研究院）
In design of instance segmentation networks that reconstruct masks, segmentation is often taken as its literal definition -- assigning each pixel a label. This has led to thinking the problem as a template matching one with the goal of minimizing the loss between the reconstructed and the ground truth pixels. Rethinking reconstruction networks as a generator, we define the problem of predicting masks as a GANs game framework: A segmentation network generates the masks, and a discriminator network decides on the quality of the masks. To demonstrate this game, we show effective modifications on …
University of Warwick（华威大学）；Histofy Ltd（Histofy有限公司）；University Hospitals Coventry and Warwickshire NHS Trust（考文垂与沃里克郡大学医院国民保健服务信托）
The recent surge in performance for image analysis of digitised pathology slides can largely be attributed to the advances in deep learning. Deep models can be used to initially localise various structures in the tissue and hence facilitate the extraction of interpretable features for biomarker discovery. However, these models are typically trained for a single task and therefore scale poorly as we wish to adapt the model for an increasing number of different tasks. Also, supervised deep learning models are very data hungry and therefore rely on large amounts of training data to perform well. …
University of Notre Dame（圣母大学）；Harper Cancer Research Institute（哈珀癌症研究所）
Image segmentation is a fundamental problem in biomedical image analysis. Recent advances in deep learning have achieved promising results on many biomedical image segmentation benchmarks. However, due to large variations in biomedical images (different modalities, image settings, objects, noise, etc), to utilize deep learning on a new application, it usually needs a new set of training data. This can incur a great deal of annotation effort and cost, because only biomedical experts can annotate effectively, and often there are too many instances in images (e.g., cells) to annotate. In this pap…
We present Spectral GUI, a multiplatform breast MR image analysis tool designed to facilitate the segmentation of fibro glandular tissues and lesions in T1 weighted breast MR images via a graphical user interface (GUI). Spectral GUIR uses spectrum loft method [1] for breast MR image segmentation. Not only is it interactive, but robust and expeditious at the same time. Being devoid of any machine learning algorithm, it shows exceptionally high execution speed with minimal overheads. The accuracy of the results has been simultaneously measured using performance metrics and expert entailment. The…
University of Warwick（华威大学）；Eindhoven University of Technology（埃因霍温理工大学）；The Chinese University of Hong Kong（香港中文大学）；University of Central Lancashire（中央兰开夏大学）；ExB Research and Development（ExB研发中心）；University of Freiburg（弗莱堡大学）；BIOSS Centre for Biological Signalling Studies, University of Freiburg（弗莱堡大学生物信号研究中心）；Google-DeepMind（谷歌DeepMind）；Sorbonne Universités, UPMC Univ Paris 06, CNRS, INSERM, Biomedical Imaging Laboratory (LIB)（索邦大学，巴黎第六大学，国家科学研究中心，国家健康与医学研究院，生物医学成像实验室）
Colorectal adenocarcinoma originating in intestinal glandular structures is the most common form of colon cancer. In clinical practice, the morphology of intestinal glands, including architectural appearance and glandular formation, is used by pathologists to inform prognosis and plan the treatment of individual patients. However, achieving good inter-observer as well as intra-observer reproducibility of cancer grading is still a major challenge in modern pathology. An automated approach which quantifies the morphology of glands is a solution to the problem. This paper provides an overview to …
University of Notre Dame（圣母大学）；Allen Institute for Cell Science（艾伦细胞科学研究所）
Automatic histopathology image segmentation is crucial to disease analysis. Limited available labeled data hinders the generalizability of trained models under the fully supervised setting. Semi-supervised learning (SSL) based on generative methods has been proven to be effective in utilizing diverse image characteristics. However, it has not been well explored what kinds of generated images would be more useful for model training and how to use such images. In this paper, we propose a new data guided generative method for histopathology image segmentation by leveraging the unlabeled data dist…
Huazhong University of Science and Technology（华中科技大学）
With pervasive applications of medical imaging in health-care, biomedical image segmentation plays a central role in quantitative analysis, clinical diagno- sis, and medical intervention. Since manual anno- tation su ers limited reproducibility, arduous e orts, and excessive time, automatic segmentation is desired to process increasingly larger scale histopathological data. Recently, deep neural networks (DNNs), par- ticularly fully convolutional networks (FCNs), have been widely applied to biomedical image segmenta- tion, attaining much improved performance. At the same time, quantization of …
Prostate cancer (PCa) is the most common cancer in men in the United States. Multiparametic magnetic resonance imaging (mp-MRI) has been explored by many researchers to targeted prostate biopsies and radiation therapy. However, assessment on mp-MRI can be subjective, development of computer-aided diagnosis systems to automatically delineate the prostate gland and the intraprostratic lesions (ILs) becomes important to facilitate with radiologists in clinical practice. In this paper, we first study the implementation of the Mask-RCNN model to segment the prostate and ILs. We trained and evaluate…
We propose LU-Net -- for LiDAR U-Net, a new method for the semantic segmentation of a 3D LiDAR point cloud. Instead of applying some global 3D segmentation method such as PointNet, we propose an end-to-end architecture for LiDAR point cloud semantic segmentation that efficiently solves the problem as an image processing problem. We first extract high-level 3D features for each point given its 3D neighbors. Then, these features are projected into a 2D multichannel range-image by considering the topology of the sensor. Thanks to these learned features and this projection, we can finally perform …
Histology method is vital in the diagnosis and prognosis of cancers and many other diseases. For the analysis of histopathological images, we need to detect and segment all gland structures. These images are very challenging, and the task of segmentation is even challenging for specialists. Segmentation of glands determines the grade of cancer such as colon, breast, and prostate. Given that deep neural networks have achieved high performance in medical images, we propose a method based on the LinkNet network for gland segmentation. We found the effects of using different loss functions. By usi…
Image segmentation is still an open problem especially when intensities of the interested objects are overlapped due to the presence of intensity inhomogeneity (also known as bias field). To segment images with intensity inhomogeneities, a bias correction embedded level set model is proposed where Inhomogeneities are Estimated by Orthogonal Primary Functions (IEOPF). In the proposed model, the smoothly varying bias is estimated by a linear combination of a given set of orthogonal primary functions. An inhomogeneous intensity clustering energy is then defined and membership functions of the clu…
Prostate cancer is one of the most prevalent cancers worldwide. One of the key factors in reducing its mortality is based on early detection. The computer-aided diagnosis systems for this task are based on the glandular structural analysis in histology images. Hence, accurate gland detection and segmentation is crucial for a successful prediction. The methodological basis of this work is a prostate gland segmentation based on U-Net convolutional neural network architectures modified with residual and multi-resolution blocks, trained using data augmentation techniques. The residual configuratio…
National University of Sciences and Technology (NUST)（国立科技大学（NUST））
Medical image segmentation assists in computer-aided diagnosis, surgeries, and treatment. Digitize tissue slide images are used to analyze and segment glands, nuclei, and other biomarkers which are further used in computer-aided medical applications. To this end, many researchers developed different neural networks to perform segmentation on histological images, mostly these networks are based on encoder-decoder architecture and also utilize complex attention modules or transformers. However, these networks are less accurate to capture relevant local and global features with accurate boundary …
We hypothesize that probabilistic voxel-level classification of anatomy and malignancy in prostate MRI, although typically posed as near-identical segmentation tasks via U-Nets, require different loss functions for optimal performance due to inherent differences in their clinical objectives. We investigate distribution, region and boundary-based loss functions for both tasks across 200 patient exams from the publicly-available ProstateX dataset. For evaluation, we conduct a thorough comparative analysis of model predictions and calibration, measured with respect to multi-class volume segmentat…
Parotid gland tumor is a common type of head and neck tumor. Segmentation of the parotid glands and tumors by MR images is important for the treatment of parotid gland tumors. However, segmentation of the parotid glands is particularly challenging due to their variable shape and low contrast with surrounding structures. Recently deep learning has developed rapidly, which can handle complex problems. However, most of the current deep learning methods for processing medical images are still based on supervised learning. Compared with natural images, medical images are difficult to acquire and co…
Gland Segmentation Via Dual Encoders and Boundary-Enhanced Attention
Guangxi Key Laboratory of Image and Graphic Intelligent Processing（广西图像图形智能处理重点实验室）；School of Computer Science and Information Security, Guilin University of Electronic Technology（桂林电子科技大学计算机与信息安全学院）；Department of Pathology, Ganzhou Municipal Hospital（赣州市立医院病理科）
针对病理图像中腺体形态多变、恶性变形及粘连重叠导致的分割难题，提出双分支 DEA 模型，结合主干编解码、边界增强注意力和 DeepLabv3+ 局部语义编码器，在 GlaS 与 CRAG 上取得优于现有方法的性能。
Isfahan University of Technology（伊斯法罕工业大学）；University of Michigan（密歇根大学）；University of Queensland（昆士兰大学）
Histopathology images contain essential information for medical diagnosis and prognosis of cancerous disease. Segmentation of glands in histopathology images is a primary step for analysis and diagnosis of an unhealthy patient. Due to the widespread application and the great success of deep neural networks in intelligent medical diagnosis and histopathology, we propose a modified version of LinkNet for gland segmentation and recognition of malignant cases. We show that using specific handcrafted features such as invariant local binary pattern drastically improves the system performance. The ex…
Background and objective: Parotid gland tumors account for approximately 2% to 10% of head and neck tumors. Preoperative tumor localization, differential diagnosis, and subsequent selection of appropriate treatment for parotid gland tumors are critical. However, the relative rarity of these tumors and the highly dispersed tissue types have left an unmet need for a subtle differential diagnosis of such neoplastic lesions based on preoperative radiomics. Recently, deep learning methods have developed rapidly, especially Transformer beats the traditional convolutional neural network in computer v…
The University of Hong Kong（香港大学）；Imperial College London（伦敦帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）
Myocardial Velocity Mapping Cardiac MR (MVM-CMR) can be used to measure global and regional myocardial velocities with proved reproducibility. Accurate left ventricle delineation is a prerequisite for robust and reproducible myocardial velocity estimation. Conventional manual segmentation on this dataset can be time-consuming and subjective, and an effective fully automated delineation method is highly in demand. By leveraging recently proposed deep learning-based semantic segmentation approaches, in this study, we propose a novel fully automated framework incorporating a 3D-UNet backbone arch…
South Australian Health and Medical Research Institute（南澳大利亚健康与医学研究所）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology（空天地海一体化大数据应用技术国家工程实验室）；Northwestern Polytechnical University（西北工业大学）；The University of Adelaide（阿德莱德大学）
Accurate and automated gland segmentation on histology tissue images is an essential but challenging task in the computer-aided diagnosis of adenocarcinoma. Despite their prevalence, deep learning models always require a myriad number of densely annotated training images, which are difficult to obtain due to extensive labor and associated expert costs related to histology image annotations. In this paper, we propose the pairwise relation-based semi-supervised (PRS^2) model for gland segmentation on histology images. This model consists of a segmentation network (S-Net) and a pairwise relation …
State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat-Sen University（中山大学中山眼科中心眼科国家重点实验室）
Meibography is a non-contact imaging technique used by ophthalmologists to assist in the evaluation and diagnosis of meibomian gland dysfunction (MGD). While artificial qualitative analysis of meibography images could lead to low repeatability and efficiency and multi-parametric analysis is demanding to offer more comprehensive information in discovering subtle changes of meibomian glands during MGD progression, we developed an automated and multi-parametric algorithm for objective and quantitative analysis of meibography images. The full architecture of the algorithm can be divided into three…
Most contemporary robots have depth sensors, and research on semantic segmentation with RGBD images has shown that depth images boost the accuracy of segmentation. Since it is time-consuming to annotate images with semantic labels per pixel, it would be ideal if we could avoid this laborious work by utilizing an existing dataset or a synthetic dataset which we can generate on our own. Robot motions are often tested in a synthetic environment, where multichannel (eg, RGB + depth + instance boundary) images plus their pixel-level semantic labels are available. However, models trained simply on s…
University of Toronto（多伦多大学）；Lunenfeld-Tanenbaum Research Institute, Sinai Health System（Sinai Health System 伦内菲尔德-塔嫩鲍姆研究所）；Brigham and Women’s Hospital（布莱根妇女医院）；The Hospital for Sick Children（SickKids 医院）
The segmentation of prostate whole gland and transition zone in Diffusion Weighted MRI (DWI) are the first step in designing computer-aided detection algorithms for prostate cancer. However, variations in MRI acquisition parameters and scanner manufacturing result in different appearances of prostate tissue in the images. Convolutional neural networks (CNNs) which have shown to be successful in various medical image analysis tasks including segmentation are typically sensitive to the variations in imaging parameters. This sensitivity leads to poor segmentation performance of CNNs trained on a …
Whole gland (WG) segmentation of the prostate plays a crucial role in detection, staging and treatment planning of prostate cancer (PCa). Despite promise shown by deep learning (DL) methods, they rely on the availability of a considerable amount of annotated data. Augmentation techniques such as translation and rotation of images present an alternative to increase data availability. Nevertheless, the amount of information provided by the transformed data is limited due to the correlation between the generated data and the original. Based on the recent success of generative adversarial networks…
To pretrain or not to pretrain? A case study of domain-specific pretraining for semantic segmentation in histopathology
University Of Utah（犹他大学）；Scientific Computing and Imaging Institute（科学计算与成像研究所）；Department of Pathology, University of Utah（犹他大学病理学系）；Kahlert School of Computing, University Of Utah（犹他大学卡尔勒特计算学院）
