每个医学分割任务都应根据其场景考虑使用特定的 AI 算法，以便获得最准确的预测模型。医学分割中最流行的算法 3D U-Net 及其变体可以直接实现肺气管分割任务，但其未能考虑气管特殊的树状结构，表明其分割精度还有很大的提升空间。因此，存在一个研究空白，因为大量最先进的 DL 算法是基础 3D U-Net 结构，在肺气道分割中没有引入特殊自然图像模态所附带的各种性能增强模块。在本文中，我们提出了两种基于 U-Net 结构的不同网络结构 Branch-Level U-Net (B-UNet) 和 Branch-Level CE-UNet (B-CE-UNet)，并在相同数据集上比较了预测结果。特别地，这两个网络都添加了分支损失和中心线损失，以学习气道细枝末端特征。还包含了不确定性估计算法以获得有信心的预测，从而增加我们整个模型整体的可信度。此外，在后处理过程中计算并提取了基于最大连通率的肺气管预测，用于分割细化和修剪。
英文摘要：
Each medical segmentation task should be considered with a specific AI algorithm based on its scenario so that the most accurate prediction model can be obtained. The most popular algorithms in medical segmentation, 3D U-Net and its variants, can directly implement the task of lung trachea segmentation, but its failure to consider the special tree-like structure of the trachea suggests that there is much room for improvement in its segmentation accuracy. Therefore, a research gap exists because a great amount of state-of-the-art DL algorithms are vanilla 3D U-Net structures, which do not introduce the various performance-enhancing modules that come with special natural image modality in lung airway segmentation. In this paper, we proposed two different network structures Branch-Level U-Net (B-UNet) and Branch-Level CE-UNet (B-CE-UNet) which are based on U-Net structure and compared the prediction results with the same dataset. Specially, both of the two networks add branch loss and central line loss to learn the feature of fine branch endings of the airways. Uncertainty estimation algorithms are also included to attain confident predictions and thereby, increase the overall trustworthiness of our whole model. In addition, predictions of the lung trachea based on the maximum connectivity rate were calculated and extracted during post-processing for segmentation refinement and pruning.
Shanghai Jiao Tong University（上海交通大学）；Imperial College London（帝国理工学院）
Airway segmentation on CT scans is critical for pulmonary disease diagnosis and endobronchial navigation. Manual extraction of airway requires strenuous efforts due to the complicated structure and various appearance of airway. For automatic airway extraction, convolutional neural networks (CNNs) based methods have recently become the state-of-the-art approach. However, there still remains a challenge for CNNs to perceive the tree-like pattern and comprehend the connectivity of airway. To address this, we propose a voxel-connectivity aware approach named AirwayNet for accurate airway segmentat…
Sichuan University - Pittsburgh Institute（四川大学-匹兹堡学院）
Airway segmentation from chest computed tomography scans has played an essential role in the pulmonary disease diagnosis. The computer-assisted airway segmentation based on the U-net architecture is more efficient and accurate compared to the manual segmentation. In this paper we employ the U$^2$-net trained by the Dice loss function to model the airway tree from the multi-site CT scans based on 299 training CT scans provided by the ATM'22. The derived saliency probability map from the training is applied to the validation data to extract the corresponding airway trees. The observation shows t…
Airway Segmentation Network for Enhanced Tubular Feature Extraction
School of Optical-electrical and Computer Engineering（光学电子与计算机工程学院）；University of Shanghai for Science and Technology（上海理工大学）；School of Information Technology（信息技术学院）；Jiangsu Open University（江苏开放大学）
Erasmus MC（伊拉斯姆斯医学中心）；Erasmus MC-Sophia Children Hospital（伊拉斯姆斯医学中心索菲亚儿童医院）；University of Copenhagen（哥本哈根大学）；Copenhagen University Hospital（哥本哈根大学医院）；Herlev-Gentofte Hospital（海莱乌-根措夫特医院）
This paper presents a fully automatic and end-to-end optimised airway segmentation method for thoracic computed tomography, based on the U-Net architecture. We use a simple and low-memory 3D U-Net as backbone, which allows the method to process large 3D image patches, often comprising full lungs, in a single pass through the network. This makes the method simple, robust and efficient. We validated the proposed method on three datasets with very different characteristics and various airway abnormalities: i) a dataset of pediatric patients including subjects with cystic fibrosis, ii) a subset of…
Northeastern University（东北大学）；Shenyang University of Technology（沈阳工业大学）；The Chinese University of Hong Kong（香港中文大学）；Second Affiliated Hospital of Dalian Medical University（大连医科大学附属第二医院）；Central Hospital Affiliated to Shenyang Medical College（沈阳医学院附属中心医院）
Accurate airway extraction from computed tomography (CT) images is a critical step for planning navigation bronchoscopy and quantitative assessment of airway-related chronic obstructive pulmonary disease (COPD). The existing methods are challenging to sufficiently segment the airway, especially the high-generation airway, with the constraint of the limited label and cannot meet the clinical use in COPD. We propose a novel two-stage 3D contextual transformer-based U-Net for airway segmentation using CT images. The method consists of two stages, performing initial and refined airway segmentation…
School of Mechanical engineering and Automation, Fuzhou University（福州大学机械工程及自动化学院）；School of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）；Thoracic Department, Fujian Medical University Union Hospital（福建医科大学附属协和医院胸外科）
Automatic airway segmentation from chest computed tomography (CT) scans plays an important role in pulmonary disease diagnosis and computer-assisted therapy. However, low contrast at peripheral branches and complex tree-like structures remain as two mainly challenges for airway segmentation. Recent research has illustrated that deep learning methods perform well in segmentation tasks. Motivated by these works, a coarse-to-fine segmentation framework is proposed to obtain a complete airway tree. Our framework segments the overall airway and small branches via the multi-information fusion convol…
We present a novel graph-based approach for labeling the anatomical branches of a given airway tree segmentation. The proposed method formulates airway labeling as a branch classification problem in the airway tree graph, where branch features are extracted using convolutional neural networks (CNN) and enriched using graph neural networks. Our graph neural network is structure-aware by having each node aggregate information from its local neighbors and position-aware by encoding node positions in the graph. We evaluated the proposed method on 220 airway trees from subjects with various severit…
Progressive Curriculum Learning with Scale-Enhanced U-Net for Continuous Airway Segmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院）；Centre of AI and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（伦敦国王学院生物医学工程与影像科学学院）
Erasmus MC（伊拉斯谟大学医学中心）；Erasmus MC-Sophia Children Hospital（伊拉斯谟大学医学中心-索菲亚儿童医院）；University of Copenhagen（哥本哈根大学）
Segmentation of the airway tree from chest computed tomography (CT) images is critical for quantitative assessment of airway diseases including bronchiectasis and chronic obstructive pulmonary disease (COPD). However, obtaining an accurate segmentation of airways from CT scans is difficult due to the high complexity of airway structures. Recently, deep convolutional neural networks (CNNs) have become the state-of-the-art for many segmentation tasks, and in particular the so-called Unet architecture for biomedical images. However, its application to the segmentation of airways still remains a c…
University College London（伦敦大学学院）；KU Leuven（鲁汶大学）；University Hospitals Leuven（鲁汶大学医院）；University of Antwerp（安特卫普大学）
Several chronic lung diseases, like idiopathic pulmonary fibrosis (IPF) are characterised by abnormal dilatation of the airways. Quantification of airway features on computed tomography (CT) can help characterise disease progression. Physics based airway measurement algorithms have been developed, but have met with limited success in part due to the sheer diversity of airway morphology seen in clinical practice. Supervised learning methods are also not feasible due to the high cost of obtaining precise airway annotations. We propose synthesising airways by style transfer using perceptual losse…
In this research project, we put forward an advanced method for airway segmentation based on the existent convolutional neural network (CNN) and graph neural network (GNN). The method is originated from the vessel segmentation, but we ameliorate it and enable the novel model to perform better for datasets from computed tomography (CT) scans. Current methods for airway segmentation are considering the regular grid only. No matter what the detailed model is, including the 3-dimensional CNN or 2-dimensional CNN in three directions, the overall graph structures are not taken into consideration. In…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；University of the Basque Country UPV/EHU（巴斯克大学（UPV/EHU））；TECNALIA, Basque Research and Technology Alliance (BRTA)（巴斯克研究与技术联盟（BRTA）泰克纳利亚研究中心）；Technical University of Munich（慕尼黑工业大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；University of Alberta（阿尔伯塔大学）；The Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）；Royal Brompton Hospital（皇家布朗普顿医院）
Airway segmentation is crucial for the examination, diagnosis, and prognosis of lung diseases, while its manual delineation is unduly burdensome. To alleviate this time-consuming and potentially subjective manual procedure, researchers have proposed methods to automatically segment airways from computerized tomography (CT) images. However, some small-sized airway branches (e.g., bronchus and terminal bronchioles) significantly aggravate the difficulty of automatic segmentation by machine learning models. In particular, the variance of voxel values and the severe data imbalance in airway branch…
Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医疗机器人研究院）；Medical Image Centre, The First Affiliated Hospital of Jinan University（暨南大学附属第一医院医学影像中心）；Department of Computer Science and Software Engineering, Swinburne University of Technology（斯威本科技大学计算机科学与软件工程系）；Shanghai Center for Brain Science and Brain-Inspired Technology（上海脑科学与类脑研究中心）
3D Convolutional Neural Networks (CNNs) have been widely adopted for airway segmentation. The performance of 3D CNNs is greatly influenced by the dataset while the public airway datasets are mainly clean CT scans with coarse annotation, thus difficult to be generalized to noisy CT scans (e.g. COVID-19 CT scans). In this work, we proposed a new dual-stream network to address the variability between the clean domain and noisy domain, which utilizes the clean CT scans and a small amount of labeled noisy CT scans for airway segmentation. We designed two different encoders to extract the transferab…
RepAir: A Framework for Airway Segmentation and Discontinuity Correction in CT
University of Michigan（密歇根大学）；D Medical, Inc.（4D医疗公司）；University of Massachusetts（马萨诸塞大学）；Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）
Erasmus MC（伊拉斯姆斯医学中心）；Erasmus MC-Sophia Children Hospital（伊拉斯姆斯医学中心索菲亚儿童医院）；University of Copenhagen（哥本哈根大学）；Herlev-Gentofte Hospital, Copenhagen University Hospital（哥本哈根大学医院海尔鲁夫-根措夫特医院）
We present an end-to-end deep learning segmentation method by combining a 3D UNet architecture with a graph neural network (GNN) model. In this approach, the convolutional layers at the deepest level of the UNet are replaced by a GNN-based module with a series of graph convolutions. The dense feature maps at this level are transformed into a graph input to the GNN module. The incorporation of graph convolutions in the UNet provides nodes in the graph with information that is based on node connectivity, in addition to the local features learnt through the downsampled paths. This information can…
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Imperial College London（帝国理工学院）；Royal Brompton Hospital（皇家布朗普顿医院）
Discontinuity in the delineation of peripheral bronchioles hinders the potential clinical application of automated airway segmentation models. Moreover, the deployment of such models is limited by the data heterogeneity across different centres, and pathological abnormalities also make achieving accurate robust segmentation in distal small airways difficult. Meanwhile, the diagnosis and prognosis of lung diseases often rely on evaluating structural changes in those anatomical regions. To address this gap, this paper presents a patch-scale adversarial-based refinement network that takes in prel…
Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医学机器人研究所）；School of Computer Science and Technology, Guizhou University（贵州大学计算机科学与技术学院）；Shanghai Chest Hospital, Shanghai Jiao Tong University（上海交通大学附属胸科医院）；Université de Lyon, INSA Lyon, CREATIS, CNRS, INSERM（里昂大学、里昂国立应用科学学院、CREATIS实验室、法国国家科学研究中心、法国国家健康与医学研究院）
Training convolutional neural networks (CNNs) for segmentation of pulmonary airway, artery, and vein is challenging due to sparse supervisory signals caused by the severe class imbalance between tubular targets and background. We present a CNNs-based method for accurate airway and artery-vein segmentation in non-contrast computed tomography. It enjoys superior sensitivity to tenuous peripheral bronchioles, arterioles, and venules. The method first uses a feature recalibration module to make the best use of features learned from the neural networks. Spatial information of features is properly i…
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Accurately and precisely characterizing the morphology of small pulmonary structures from Computed Tomography (CT) images, such as airways and vessels, is becoming of great importance for diagnosis of pulmonary diseases. The smaller conducting airways are the major site of increased airflow resistance in chronic obstructive pulmonary disease (COPD), while accurately sizing vessels can help identify arterial and venous changes in lung regions that may determine future disorders. However, traditional methods are often limited due to image resolution and artifacts. We propose a Convolutional Neur…
Deep Learning-Based Airway Segmentation in Systemic Lupus Erythematosus Patients with Interstitial Lung Disease (SLE-ILD): A Comparative High-Resolution CT Analysis
Department of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（放射科，北京地坛医院，中国医学科学院与北京联合医学院）；Research and Development Center (RDC), Canon Medical Systems (China)（研发部（RDC），佳能医疗系统（中国））；Canon Medical Systems (China)（佳能医疗系统（中国））
Predicting Patient Survival with Airway Biomarkers using nn-Unet/Radiomics
Normandie Univ., INSA Rouen, LITIS, 76801, Saint Etienne du Rouvray, France（诺曼底大学）；Normandie Univ., UNICAEN, ENSICAEN, CNRS, GREYC, 14000 Caen, France（诺曼底大学）；Nuclear Medicine Department, Henri Becquerel Cancer Center, Rouen, France（贝克勒尔癌症中心核医学部）；Radiotherapy Department, Henri Becquerel Cancer Center, Rouen, France（贝克勒尔癌症中心放射治疗部）；Siemens Healthineers（西门子医疗）
Deep convolutional neural networks for image segmentation do not learn the label structure explicitly and may produce segmentations with an incorrect structure, e.g., with disconnected cylindrical structures in the segmentation of tree-like structures such as airways or blood vessels. In this paper, we propose a novel label refinement method to correct such errors from an initial segmentation, implicitly incorporating information about label structure. This method features two novel parts: 1) a model that generates synthetic structural errors, and 2) a label appearance simulation network that …
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（上海交通大学图像处理与模式识别研究所）；Institute of Medical Robotics, Shanghai Jiao Tong University（上海交通大学医疗机器人研究院）；Department of Respiratory and Critical Care Medicine, Department of Respiratory Endoscopy, Shanghai Chest Hospital, Shanghai Engineering Research Center of Respiratory Endoscopy（上海市胸科医院呼吸与危重症医学科、呼吸内镜中心、上海市呼吸内镜工程技术研究中心）
Airway segmentation is critical for virtual bronchoscopy and computer-aided pulmonary disease analysis. In recent years, convolutional neural networks (CNNs) have been widely used to delineate the bronchial tree. However, the segmentation results of the CNN-based methods usually include many discontinuous branches, which need manual repair in clinical use. A major reason for the breakages is that the appearance of the airway wall can be affected by the lung disease as well as the adjacency of the vessels, while the network tends to overfit to these special patterns in the training set. To lear…
Validated respiratory drug deposition predictions from 2D and 3D medical images with statistical shape models and convolutional neural networks
Heriot-Watt University（赫瑞瓦特大学）；Royal Hospital for Children and Young People, NHS Lothian（NHS洛锡安区儿童与青少年皇家医院）；University Hospital Southampton NHS Foundation Trust（南安普顿大学医院NHS基金会信托）；Brunel University（布鲁内尔大学）；University of Edinburgh（爱丁堡大学）
Technical University of Denmark（丹麦技术大学）；Slagelse Hospital（斯劳厄尔瑟医院）；Region Hovedstaden（首都大区）
Curvilinear structure segmentation is important in medical imaging, quantifying structures such as vessels, airways, neurons, or organ boundaries in 2D slices. Segmentation via pixel-wise classification often fails to capture the small and low-contrast curvilinear structures. Prior topological information is typically used to address this problem, often at an expensive computational cost, and sometimes requiring prior knowledge of the expected topology. We present DTU-Net, a data-driven approach to topology-preserving curvilinear structure segmentation. DTU-Net consists of two sequential, ligh…
Pulmonary lobe segmentation in computed tomography scans is essential for regional assessment of pulmonary diseases. Recent works based on convolution neural networks have achieved good performance for this task. However, they are still limited in capturing structured relationships due to the nature of convolution. The shape of the pulmonary lobes affect each other and their borders relate to the appearance of other structures, such as vessels, airways, and the pleural wall. We argue that such structural relationships play a critical role in the accurate delineation of pulmonary lobes when the…
With the advent of advancements in deep learning approaches, such as deep convolution neural network, residual neural network, adversarial network; U-Net architectures are most widely utilized in biomedical image segmentation to address the automation in identification and detection of the target regions or sub-regions. In recent studies, U-Net based approaches have illustrated state-of-the-art performance in different applications for the development of computer-aided diagnosis systems for early diagnosis and treatment of diseases such as brain tumor, lung cancer, alzheimer, breast cancer, et…
Automatic pathological pulmonary lobe segmentation(PPLS) enables regional analyses of lung disease, a clinically important capability. Due to often incomplete lobe boundaries, PPLS is difficult even for experts, and most prior art requires inference from contextual information. To address this, we propose a novel PPLS method that couples deep learning with the random walker (RW) algorithm. We first employ the recent progressive holistically-nested network (P-HNN) model to identify potential lobar boundaries, then generate final segmentations using a RW that is seeded and weighted by the P-HNN …
School of Computer Sciences, Universiti Sains Malaysia（马来西亚理科大学计算机科学学院）；School of Network Communication, Zhejiang Yuexiu University（浙江越秀外国语学院网络传播学院）
Medical images often exhibit low and blurred contrast between lesions and surrounding tissues, with considerable variation in lesion edges and shapes even within the same disease, leading to significant challenges in segmentation. Therefore, precise segmentation of lesions has become an essential prerequisite for patient condition assessment and formulation of treatment plans. Significant achievements have been made in research related to the U-Net model in recent years. It improves segmentation performance and is extensively applied in the semantic segmentation of medical images to offer tech…
The paper is a short review of medical image segmentation using U-Net and its variants. As we understand going through a medical images is not an easy job for any clinician either radiologist or pathologist. Analysing medical images is the only way to perform non-invasive diagnosis. Segmenting out the regions of interest has significant importance in medical images and is key for diagnosis. This paper also gives a bird eye view of how medical image segmentation has evolved. Also discusses challenge's and success of the deep neural architectures. Following how different hybrid architectures hav…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；VoxelCloud Inc.（体素云公司）
Reliable and automatic segmentation of lung lobes is important for diagnosis, assessment, and quantification of pulmonary diseases. The existing techniques are prohibitively slow, undesirably rely on prior (airway/vessel) segmentation, and/or require user interactions for optimal results. This work presents a reliable, fast, and fully automated lung lobe segmentation based on a progressive dense V-network (PDV-Net). The proposed method can segment lung lobes in one forward pass of the network, with an average runtime of 2 seconds using 1 Nvidia Titan XP GPU, eliminating the need for any prior …
Purdue University Northwest（普渡大学西北校区）；Michigan Technological University（密歇根理工大学）
U-net is an image segmentation technique developed primarily for medical image analysis that can precisely segment images using a scarce amount of training data. These traits provide U-net with a very high utility within the medical imaging community and have resulted in extensive adoption of U-net as the primary tool for segmentation tasks in medical imaging. The success of U-net is evident in its widespread use in all major image modalities from CT scans and MRI to X-rays and microscopy. Furthermore, while U-net is largely a segmentation tool, there have been instances of the use of U-net in…
Pulmonary lobe segmentation is an important preprocessing task for the analysis of lung diseases. Traditional methods relying on fissure detection or other anatomical features, such as the distribution of pulmonary vessels and airways, could provide reasonably accurate lobe segmentations. Deep learning based methods can outperform these traditional approaches, but require large datasets. Deep multi-task learning is expected to utilize labels of multiple different structures. However, commonly such labels are distributed over multiple datasets. In this paper, we proposed a multi-task semi-super…
ETH Zürich（苏黎世联邦理工学院）；Università di Trento（特伦托大学）；Università di Bologna（博洛尼亚大学）
Training quantised neural networks (QNNs) is a non-differentiable optimisation problem since weights and features are output by piecewise constant functions. The standard solution is to apply the straight-through estimator (STE), using different functions during the inference and gradient computation steps. Several STE variants have been proposed in the literature aiming to maximise the task accuracy of the trained network. In this paper, we analyse STE variants and study their impact on QNN training. We first observe that most such variants can be modelled as stochastic regularisations of sta…
Idiap Research Institute（伊迪亚普研究所）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
We present a semi-blind, spatially-variant deconvolution technique aimed at optical microscopy that combines a local estimation step of the point spread function (PSF) and deconvolution using a spatially variant, regularized Richardson-Lucy algorithm. To find the local PSF map in a computationally tractable way, we train a convolutional neural network to perform regression of an optical parametric model on synthetically blurred image patches. We deconvolved both synthetic and experimentally-acquired data, and achieved an improvement of image SNR of 1.00 dB on average, compared to other deconvo…
ASTRA-Net: Anatomy-Specific Transfer and Representation Alignment for Drug-Induced Sleep Endoscopy Segmentation
Department of Otolaryngology, Peking University Third Hospital（北京大学第三医院耳鼻咽喉科）；Institute of Medical Technology, Peking University Health Science Center（北京大学医学部医学技术研究所）；National Biomedical Imaging Center, College of Future Technology, Peking University（北京大学未来技术学院国家生物医学影像中心）
