Modeling 3D context is essential for high-performance 3D medical image analysis. Although 2D networks benefit from large-scale 2D supervised pretraining, it is weak in capturing 3D context. 3D networks are strong in 3D context yet lack supervised pretraining. As an emerging technique, \emph{3D context fusion operator}, which enables conversion from 2D pretrained networks, leverages the advantages of both and has achieved great success. Existing 3D context fusion operators are designed to be spatially symmetric, i.e., performing identical operations on each 2D slice like convolutions. However, these operators are not truly equivariant to translation, especially when only a few 3D slices are used as inputs. In this paper, we propose a novel asymmetric 3D context fusion operator (A3D), which uses different weights to fuse 3D context from different 2D slices. Notably, A3D is NOT translation-equivariant while it significantly outperforms existing symmetric context fusion operators without introducing large computational overhead. We validate the effectiveness of the proposed method by extensive experiments on DeepLesion benchmark, a large-scale public dataset for universal lesion detection from computed tomography (CT). The proposed A3D consistently outperforms symmetric context fusion operators by considerable margins, and establishes a new \emph{state of the art} on DeepLesion. To facilitate open research, our code and model in PyTorch are available at https://github.com/M3DV/AlignShift.
New Jersey Institute of Technology（新泽西理工学院）；Kessler Foundation（凯斯勒基金会）；Emory University（埃默里大学）；Atlanta VA Medical Center（亚特兰大退伍军人事务医疗中心）；Medical College of Wisconsin（威斯康星医学院）；Rutgers University - Newark（罗格斯大学纽瓦克分校）
We propose a fully 3D multi-path convolutional network to predict stroke lesions from 3D brain MRI images. Our multi-path model has independent encoders for different modalities containing residual convolutional blocks, weighted multi-path feature fusion from different modalities, and weighted fusion modules to combine encoder and decoder features. Compared to existing 3D CNNs like DeepMedic, 3D U-Net, and AnatomyNet, our networks achieves the highest statistically significant cross-validation accuracy of 60.5% on the large ATLAS benchmark of 220 patients. We also test our model on multi-modal…
Cross Feature Fusion of Fundus Image and Generated Lesion Map for Referable Diabetic Retinopathy Classification
Sungkyunkwan University（成均馆大学）；College of Computing and Informatics, Sungkyunkwan University（成均馆大学计算与信息学院）；Dept. of Electrical and Computer Engineering, Sungkyunkwan University（成均馆大学电气与计算机工程系）
Small Lesions-aware Bidirectional Multimodal Multiscale Fusion Network for Lung Disease Classification
Xidian University（西安电子科技大学）；Hangzhou Dianzi University（杭州电子科技大学）；Zhejiang College of Security Technology, School of Artificial Intelligence（浙江安全技术学院人工智能学院）；Shenzhen Polytechnic University（深圳职业技术学院）；Shenzhen University（深圳大学）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
In clinical practice, radiologists are reliant on the lesion size when distinguishing metastatic from non-metastatic lesions. A prerequisite for lesion sizing is their detection, as it promotes the downstream assessment of tumor spread. However, lesions vary in their size and appearance in CT scans, and radiologists often miss small lesions during a busy clinical day. To overcome these challenges, we propose the use of state-of-the-art detection neural networks to flag suspicious lesions present in the NIH DeepLesion dataset for sizing. Additionally, we incorporate a bounding box fusion techni…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）；Princeton University（普林斯顿大学）
Automatic lung lesions segmentation of chest CT scans is considered a pivotal stage towards accurate diagnosis and severity measurement of COVID-19. Traditional U-shaped encoder-decoder architecture and its variants suffer from diminutions of contextual information in pooling/upsampling operations with increased semantic gaps among encoded and decoded feature maps as well as instigate vanishing gradient problems for its sequential gradient propagation that result in sub-optimal performance. Moreover, operating with 3D CT-volume poses further limitations due to the exponential increase of compu…
Magnetic resonance imaging (MRI) is highly sensitive for lesion detection in the breasts. Sequences obtained with different settings can capture the specific characteristics of lesions. Such multi-parameter MRI information has been shown to improve radiologist performance in lesion classification, as well as improving the performance of artificial intelligence models in various tasks. However, obtaining multi-parameter MRI makes the examination costly in both financial and time perspectives, and there may be safety concerns for special populations, thus making acquisition of the full spectrum …
IAFI-FCOS: Intra- and across-layer feature interaction FCOS model for lesion detection of CT images
College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；The First Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第一医院）
Deep neural networks have become the gold-standard approach for the automated segmentation of 3D medical images. Their full acceptance by clinicians remains however hampered by the lack of intelligible uncertainty assessment of the provided results. Most approaches to quantify their uncertainty, such as the popular Monte Carlo dropout, restrict to some measure of uncertainty in prediction at the voxel level. In addition not to be clearly related to genuine medical uncertainty, this is not clinically satisfying as most objects of interest (e.g. brain lesions) are made of groups of voxels whose …
Clinical Center（临床中心）；National Center for Biotechnology Information（国家生物技术信息中心）；National Library of Medicine（国家医学图书馆）；National Institutes of Health（美国国立卫生研究院）
When reading medical images such as a computed tomography (CT) scan, radiologists generally search across the image to find lesions, characterize and measure them, and then describe them in the radiological report. To automate this process, we propose a multitask universal lesion analysis network (MULAN) for joint detection, tagging, and segmentation of lesions in a variety of body parts, which greatly extends existing work of single-task lesion analysis on specific body parts. MULAN is based on an improved Mask R-CNN framework with three head branches and a 3D feature fusion strategy. It achi…
The rapid increment of morbidity of brain stroke in the last few years have been a driving force towards fast and accurate segmentation of stroke lesions from brain MRI images. With the recent development of deep-learning, computer-aided and segmentation methods of ischemic stroke lesions have been useful for clinicians in early diagnosis and treatment planning. However, most of these methods suffer from inaccurate and unreliable segmentation results because of their inability to capture sufficient contextual features from the MRI volumes. To meet these requirements, 3D convolutional neural ne…
Machine learning is widely used in developing computer-aided diagnosis (CAD) schemes of medical images. However, CAD usually computes large number of image features from the targeted regions, which creates a challenge of how to identify a small and optimal feature vector to build robust machine learning models. In this study, we investigate feasibility of applying a random projection algorithm to build an optimal feature vector from the initially CAD-generated large feature pool and improve performance of machine learning model. We assemble a retrospective dataset involving 1,487 cases of mamm…
DynSegNet:Dynamic Architecture Adjustment for Adversarial Learning in Segmenting Hemorrhagic Lesions from Fundus Images
School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Information Engineering, Sichuan Agricultural University（四川农业大学信息工程学院）
Existing universal lesion detection (ULD) methods utilize compute-intensive anchor-based architectures which rely on predefined anchor boxes, resulting in unsatisfactory detection performance, especially in small and mid-sized lesions. Further, these default fixed anchor-sizes and ratios do not generalize well to different datasets. Therefore, we propose a robust one-stage anchor-free lesion detection network that can perform well across varying lesions sizes by exploiting the fact that the box predictions can be sorted for relevance based on their center rather than their overlap with the obj…
Convolutional neural networks (CNNs) have achieved the state-of-the-art performance in skin lesion analysis. Compared with single CNN classifier, combining the results of multiple classifiers via fusion approaches shows to be more effective and robust. Since the skin lesion datasets are usually limited and statistically biased, while designing an effective fusion approach, it is important to consider not only the performance of each classifier on the training/validation dataset, but also the relative discriminative power (e.g., confidence) of each classifier regarding an individual sample in t…
PAII Inc.（PAII公司）；Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）；Ping An Insurance (Group) Company of China, Ltd.（中国平安保险（集团）股份有限公司）
Large-scale datasets with high-quality labels are desired for training accurate deep learning models. However, due to the annotation cost, datasets in medical imaging are often either partially-labeled or small. For example, DeepLesion is such a large-scale CT image dataset with lesions of various types, but it also has many unlabeled lesions (missing annotations). When training a lesion detector on a partially-labeled dataset, the missing annotations will generate incorrect negative signals and degrade the performance. Besides DeepLesion, there are several small single-type datasets, such as …
University of Pittsburgh（匹兹堡大学）；Tangdu Hospital（唐都医院）
This study aims to automatically diagnose thoracic diseases depicted on the chest x-ray (CXR) images using deep convolutional neural networks. The existing methods generally used the entire CXR images for training purposes, but this strategy may suffer from two drawbacks. First, potential misalignment or the existence of irrelevant objects in the entire CXR images may cause unnecessary noise and thus limit the network performance. Second, the relatively low image resolution caused by the resizing operation, which is a common preprocessing procedure for training neural networks, may lead to the…
Focal Modulation and Bidirectional Feature Fusion Network for Medical Image Segmentation
Department of Computing, Abasyn University Islamabad Campus (AUIC)（阿巴斯扬大学伊斯兰堡校区计算机系）；School of Computer Science and Informatics, De Montfort University（德蒙福特大学计算机科学与信息学学院）；Naif Arab University for Security Sciences（纳夫阿拉伯安全科学大学）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；Center of Excellence in Precision Medicine and Digital Health, Chulalongkorn University（朱拉隆功大学精准医学与数字健康卓越中心）；School of Biomedical Engineering, UNSW Sydney（新南威尔士大学悉尼分校生物医学工程学院）
The reliable and timely stratification of bone lesion evolution risk in smoldering Multiple Myeloma plays an important role in identifying prime markers of the disease's advance and in improving the patients' outcome. In this work we provide an asymmetric cascade network for the longitudinal prediction of future bone lesions for T1 weighted whole body MR images. The proposed cascaded architecture, consisting of two distinct configured U-Nets, first detects the bone regions and subsequently predicts lesions within bones in a patch based way. The algorithm provides a full volumetric risk score m…
University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Guizhou Provincial People’s Hospital（贵州省人民医院）；Henan Provincial People’s Hospital（河南省人民医院）；Shenzhen University（深圳大学）
Segmenting stroke lesions from T1-weighted MR images is of great value for large-scale stroke rehabilitation neuroimaging analyses. Nevertheless, there are great challenges with this task, such as large range of stroke lesion scales and the tissue intensity similarity. The famous encoder-decoder convolutional neural network, which although has made great achievements in medical image segmentation areas, may fail to address these challenges due to the insufficient uses of multi-scale features and context information. To address these challenges, this paper proposes a Cross-Level fusion and Cont…
Southern University of Science and Technology（南方科技大学）；Research Institute of Trustworthy Autonomous Systems Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Shenzhen People’s Hospital（深圳市人民医院）
Hard exudates (HE) is the most specific biomarker for retina edema. Precise HE segmentation is vital for disease diagnosis and treatment, but automatic segmentation is challenged by its large variation of characteristics including size, shape and position, which makes it difficult to detect tiny lesions and lesion boundaries. Considering the complementary features between segmentation and super-resolution tasks, this paper proposes a novel hard exudates segmentation method named SS-MAF with an auxiliary super-resolution task, which brings in helpful detailed features for tiny lesion and bounda…
Wavelet-based Global-Local Interaction Network with Cross-Attention for Multi-View Diabetic Retinopathy Detection
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；Shenzhen Key Laboratory of Visual Object Detection and Recognition（深圳市视觉目标检测与识别重点实验室）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）
Nanjing University of Science and Technology（南京理工大学）；Jinling Hospital（金陵医院）；Medical School of Nanjing University（南京大学医学院）
This work first attempts to automatically recognize pancreatitis on CT scan images. However, different form the traditional object recognition, such pancreatitis recognition is challenging due to the fine-grained and non-rigid appearance variability of the local diseased regions. To this end, we propose a customized Region-Manipulated Fusion Networks (RMFN) to capture the key characteristics of local lesion for pancreatitis recognition. Specifically, to effectively highlight the imperceptible lesion regions, a novel region-manipulated scheme in RMFN is proposed to force the lesion regions whil…
UNISELF: A Unified Network with Instance Normalization and Self-Ensembled Lesion Fusion for Multiple Sclerosis Lesion Segmentation
Department of Electrical and Computer Engineering, Johns Hopkins University, Baltimore, MD 21218, USA（电气与计算机工程系，约翰霍普金斯大学）；Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN 37215, USA（电气与计算机工程系，范德比尔特大学）；Department of Neurology, Johns Hopkins School of Medicine, Baltimore, MD 21287, USA（神经病学系，约翰霍普金斯医学院）；Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218, USA（计算机科学系，约翰霍普金斯大学）；Department of Radiology, Uniformed Services University of the Health Sciences, Bethesda, MD, 20814, USA（放射学系，统一服务大学健康科学学院）
Uncertainty-Aware Multimodal Fusion for Oral Lesion Classification
1 School of Medical Science \& Technology, IIT Kharagpur, India 2 Department of Electrical Engineering, IIT Kharagpur, India 3 Dr. B.C. Roy Multispeciality Medical Research Centre, IIT Kharagpur, India
Shenzhen University（深圳大学）；University of Sydney（悉尼大学）；Northwestern Polytechnical University（西北工业大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Assessing the location and extent of lesions caused by chronic stroke is critical for medical diagnosis, surgical planning, and prognosis. In recent years, with the rapid development of 2D and 3D convolutional neural networks (CNN), the encoder-decoder structure has shown great potential in the field of medical image segmentation. However, the 2D CNN ignores the 3D information of medical images, while the 3D CNN suffers from high computational resource demands. This paper proposes a new architecture called dimension-fusion-UNet (D-UNet), which combines 2D and 3D convolution innovatively in the…
South Central University for Nationalities（中南民族大学）；Nanjing Eastern Theater General Hospital（南京东部战区总医院）
Wireless Capsule Endoscopy(WCE) has been widely used for the screening of gastrointestinal(GI) diseases, especially the small intestine, due to its advantages of non-invasive and painless imaging of the entire digestive tract.However, the huge amount of image data captured by WCE makes manual reading a process that requires a huge amount of tasks and can easily lead to missed detection and false detection of lesions.Therefore, In this paper, we propose a \textbf{T}wo-stage \textbf{M}ulti-scale \textbf{F}eature-fusion learning network(\textbf{TMFNet}) to automatically detect small intestinal po…
A Wavelet Guided Attention Module for Skin Cancer Classification with Gradient-based Feature Fusion
Jadavpur University（jadavpur大学）；International Institute of Information Technology（国际信息技术学院）；China University of Mining and Technology（中国矿业大学）；Saint Petersburg Electrotechnical University "LETI"（圣彼得堡电工技术大学“LETI”）
Xiamen University（厦门大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究院）；The Hong Kong Polytechnic University（香港理工大学）
Breast lesion detection in ultrasound is critical for breast cancer diagnosis. Existing methods mainly rely on individual 2D ultrasound images or combine unlabeled video and labeled 2D images to train models for breast lesion detection. In this paper, we first collect and annotate an ultrasound video dataset (188 videos) for breast lesion detection. Moreover, we propose a clip-level and video-level feature aggregated network (CVA-Net) for addressing breast lesion detection in ultrasound videos by aggregating video-level lesion classification features and clip-level temporal features. The clip-…
University of Science and Technology of China（中国科学技术大学）
Universal Lesion Detection (ULD) in computed tomography plays an essential role in computer-aided diagnosis. Promising ULD results have been reported by multi-slice-input detection approaches which model 3D context from multiple adjacent CT slices, but such methods still experience difficulty in obtaining a global representation among different slices and within each individual slice since they only use convolution-based fusion operations. In this paper, we propose a novel Slice Attention Transformer (SATr) block which can be easily plugged into convolution-based ULD backbones to form hybrid n…
Automated delineation of COVID-19 lesions from lung CT scans aids the diagnosis and prognosis for patients. The asymmetric shapes and positioning of the infected regions make the task extremely difficult. Capturing information at multiple scales will assist in deciphering features, at global and local levels, to encompass lesions of variable size and texture. We introduce the Full-scale Deeply Supervised Attention Network (FuDSA-Net), for efficient segmentation of corona-infected lung areas in CT images. The model considers activation responses from all levels of the encoding path, encompassin…
VRVis Zentrum für Virtual Reality und Visualisierung Forschungs-GmbH（VRVis虚拟现实与可视化研究中心有限公司）
Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently invest…
Integrating Radiomics with Deep Learning Enhances Multiple Sclerosis Lesion Delineation
Skolkovo Institute of Science and Technology（斯克尔科夫科学与技术研究所）；Biomedically Informed Artificial Intelligence Laboratory（生物医学引导人工智能实验室）；University of Sharjah（沙迦大学）；College of Medicine and Health Sciences（医学与健康科学学院）；United Arab Emirates University（阿拉伯联合酋长国大学）
