2.75D: Boosting learning by representing 3D Medical imaging to 2D features for small data
Netherlands Cancer Institute（荷兰癌症研究所）；Erasmus Medical Center Rotterdam（鹿特丹伊拉斯姆斯医学中心）；Radboud University Medical Center（拉德堡德大学医学中心）；Southern University of Science and Technology（南方科技大学）；Aberystwyth University（阿伯里斯特威斯大学）
Xin Wang, Ruisheng Su, Weiyi Xie, Wenjin Wang, Yi Xu, Ritse Mann, Jungong Han, Tao Tan
In medical-data driven learning, 3D convolutional neural networks (CNNs) have started to show superior performance to 2D CNNs in numerous deep learning tasks, proving the added value of 3D spatial information in feature representation. However, the difficulty in collecting more training samples to converge, more computational resources and longer execution time make this approach less applied. Also, applying transfer learning on 3D CNN is challenging due to a lack of publicly available pre-trained 3D models. To tackle these issues, we proposed a novel 2D strategical representation of volumetric data, namely 2.75D. In this work, the spatial information of 3D images is captured in a single 2D view by a spiral-spinning technique. As a result, 2D CNN networks can also be used to learn volumetric information. Besides, we can fully leverage pre-trained 2D CNNs for downstream vision problems. We also explore a multi-view 2.75D strategy, 2.75D 3 channels (2.75Dx3), to boost the advantage of 2.75D. We evaluated the proposed methods on three public datasets with different modalities or organs (Lung CT, Breast MRI, and Prostate MRI), against their 2D, 2.5D, and 3D counterparts in classification tasks. Results show that the proposed methods significantly outperform other counterparts when all methods were trained from scratch on the lung dataset. Such performance gain is more pronounced with transfer learning or in the case of limited training data. Our methods also achieved comparable performance on other datasets. In addition, our methods achieved a substantial reduction in time consumption of training and inference compared with the 2.5D or 3D method.
National University of Singapore（新加坡国立大学）；National University Hospital（国立大学医院）；Zhejiang University（浙江大学）
In this paper, we propose a novel semi-supervised learning (SSL) framework named BoostMIS that combines adaptive pseudo labeling and informative active annotation to unleash the potential of medical image SSL models: (1) BoostMIS can adaptively leverage the cluster assumption and consistency regularization of the unlabeled data according to the current learning status. This strategy can adaptively generate one-hot "hard" labels converted from task model predictions for better task model training. (2) For the unselected unlabeled images with low confidence, we introduce an Active learning (AL) …
Human visual attention has recently shown its distinct capability in boosting machine learning models. However, studies that aim to facilitate medical tasks with human visual attention are still scarce. To support the use of visual attention, this paper describes a novel deep learning model for visual saliency prediction on chest X-ray (CXR) images. To cope with data deficiency, we exploit the multi-task learning method and tackles disease classification on CXR simultaneously. For a more robust training process, we propose a further optimized multi-task learning scheme to better handle model o…
Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Hupan Lab（虎盘实验室）
Boosting Few-Shot Learning with Disentangled Self-Supervised Learning and Meta-Learning for Medical Image Classification
Institute of Information Science and Technologies, National Research Council（国家研究委员会信息科学与技术研究所）；Department of Information Engineering, University of Pisa（比萨大学信息工程系）；School of Engineering, University of Edinburgh（爱丁堡大学工程学院）
Boosting Memory Efficiency in Transfer Learning for High-Resolution Medical Image Classification
Department of Electronic and Electrical Engineering, Southern University of Science and Technology（电子与电气工程系，南方科技大学）；School of Biomedical Engineering, The University of British Columbia（生物医学工程学院，不列颠哥伦比亚大学）；Department of Electrical and Electronic Engineering, The University of Hong Kong（电气与电子工程系，香港大学）；Jiaxing Research Institute, Southern University of Science and Technology（嘉兴研究院，南方科技大学）
Boosting Medical Image-based Cancer Detection via Text-guided Supervision from Reports
DAMO Academy, Alibaba Group（阿里巴巴集团达摩学院）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；Hupan Lab（湖畔实验室）；Sun Yat-sen University Cancer Center（中山大学肿瘤防治中心）；South China Hospital, Medical School, Shenzhen University（深圳大学医学院华南医院）
Hybrid Convolution Neural Network Integrated with Pseudo-Newton Boosting for Lumbar Spine Degeneration Detection
School of Computer Science and Engineering, Vellore Institute of Technology（韦洛尔理工大学计算机科学与工程学院）；Department of Information Science and Technology, College of Engineering Guindy, Anna University（安娜大学金迪工程学院信息科学与技术系）；Integrated Management Coastal Zones Research Institute, Universitat Politecnica de Valencia（瓦伦西亚理工大学海岸带综合管理研究所）
LMU München（慕尼黑大学）；Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art performance for image segmentation in medical imaging. Recently, squeeze and excitation (SE) modules and variations thereof have been introduced to recalibrate feature maps channel- and spatial-wise, which can boost performance while only minimally increasing model complexity. So far, the development of SE has focused on 2D images. In this paper, we propose `Project & Excite' (PE) modules that base upon the ideas of SE and extend them to operating on 3D volumetric images. `Project & Excite' does not perform global average po…
South China University of Technology（华南理工大学）；University of Utah（犹他大学）
We introduce the idea of inter-slice image augmentation whereby the numbers of the medical images and the corresponding segmentation labels are increased between two consecutive images in order to boost medical image segmentation accuracy. Unlike conventional data augmentation methods in medical imaging, which only increase the number of training samples directly by adding new virtual samples using simple parameterized transformations such as rotation, flipping, scaling, etc., we aim to augment data based on the relationship between two consecutive images, which increases not only the number b…
Yale Center for Medical Informatics（耶鲁医学信息学中心）；VA CT Healthcare System（美国退伍军人事务部康涅狄格医疗系统）
Medical image datasets can have large number of images representing patients with different health conditions and various disease severity. When dealing with raw unlabeled image datasets, the large number of samples often makes it hard for experts and non-experts to understand the variety of images present in a dataset. Supervised learning methods rely on labeled images which requires a considerable effort by medical experts to first understand the communities of images present in the data and then labeling the images. Here, we propose an algorithm to facilitate the automatic identification of…
Airdoc（鹰瞳科技）；Harbin Engineering University（哈尔滨工程大学）；IIAI（国际人工智能研究院）；Monash University（莫纳什大学）；The University of Sydney（悉尼大学）
Deep neural networks are known to be data-driven and label noise can have a marked impact on model performance. Recent studies have shown great robustness to classic image recognition even under a high noisy rate. In medical applications, learning from datasets with label noise is more challenging since medical imaging datasets tend to have asymmetric (class-dependent) noise and suffer from high observer variability. In this paper, we systematically discuss and define the two common types of label noise in medical images - disagreement label noise from inconsistency expert opinions and single-…
Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision
Arizona State University（亚利桑那州立大学）；Mayo Clinic（梅奥诊所）
Brain tumors are masses or abnormal growths of cells within the brain or the central spinal canal with symptoms such as headaches, seizures, weakness or numbness in the arms or legs, changes in personality or behaviour, nausea, vomiting, vision or hearing problems and dizziness. Conventional diagnosis of brain tumour involves some tests and procedure which may include the consideration of medical history, physical examination, imaging tests (such as CT or MRI scans), and biopsy (removal and examination of a small piece of the tumor tissue). These procedures, while effective, are mentally stren…
Shape priors have long been known to be effective when reconstructing 3D shapes from noisy or incomplete data. When using a deep-learning based shape representation, this often involves learning a latent representation, which can be either in the form of a single global vector or of multiple local ones. The latter allows more flexibility but is prone to overfitting. In this paper, we advocate a hybrid approach representing shapes in terms of 3D meshes with a separate latent vector at each vertex. During training the latent vectors are constrained to have the same value, which avoids overfittin…
Boosting Medical Image Segmentation Performance with Adaptive Convolution Layer
LIPN-UMR-CNRS 7030（法国国家科学研究中心UMR 7030 LIPN实验室）；Sorbonne University Paris Nord（巴黎第十一大学（索邦大学巴黎北校区））；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Université Paris-Est Créteil（巴黎东部克雷泰伊大学）
University of Waterloo（滑铁卢大学）；Vector Institute（向量研究所）
This paper introduces the `Projectron' as a new neural network architecture that uses Radon projections to both classify and represent medical images. The motivation is to build shallow networks which are more interpretable in the medical imaging domain. Radon transform is an established technique that can reconstruct images from parallel projections. The Projectron first applies global Radon transform to each image using equidistant angles and then feeds these transformations for encoding to a single layer of neurons followed by a layer of suitable kernels to facilitate a linear separation of…
Decision support tools that rely on supervised learning require large amounts of expert annotations. Using past radiological reports obtained from hospital archiving systems has many advantages as training data above manual single-class labels: they are expert annotations available in large quantities, covering a population-representative variety of pathologies, and they provide additional context to pathology diagnoses, such as anatomical location and severity. Learning to auto-generate such reports from images present many challenges such as the difficulty in representing and generating long…
VoxelCloud, Inc.（体素云公司）；Second Military Medical University（第二军医大学）；Changzheng Hospital（长征医院）；Shanghai Jiaotong University School of Medicine（上海交通大学医学院）；Ruijin Hospital（瑞金医院）
We investigate the effectiveness of a simple solution to the common problem of deep learning in medical image analysis with limited quantities of labeled training data. The underlying idea is to assign artificial labels to abundantly available unlabeled medical images and, through a process known as surrogate supervision, pre-train a deep neural network model for the target medical image analysis task lacking sufficient labeled training data. In particular, we employ 3 surrogate supervision schemes, namely rotation, reconstruction, and colorization, in 4 different medical imaging applications …
The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）
Computational cytology is a critical, rapid-developing, yet challenging topic in the field of medical image computing which analyzes the digitized cytology image by computer-aided technologies for cancer screening. Recently, an increasing number of deep learning (DL) algorithms have made significant progress in medical image analysis, leading to the boosting publications of cytological studies. To investigate the advanced methods and comprehensive applications, we survey more than 120 publications of DL-based cytology image analysis in this article. We first introduce various deep learning met…
MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Heidelberg University（海德堡大学）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；University Medical Center Heidelberg（海德堡大学医院）；Helmholtz Imaging（亥姆霍兹成像机构）
German Cancer Research Center（德国癌症研究中心）；Heidelberg University（海德堡大学）
Image registration is the basis for many applications in the fields of medical image computing and computer assisted interventions. One example is the registration of 2D X-ray images with preoperative three-dimensional computed tomography (CT) images in intraoperative surgical guidance systems. Due to the high safety requirements in medical applications, estimating registration uncertainty is of a crucial importance in such a scenario. However, previously proposed methods, including classical iterative registration methods and deep learning-based methods have one characteristic in common: They…
A Novel Momentum-Based Deep Learning Techniques for Medical Image Classification and Segmentation
Northwestern University（西北大学）；IIIT Delhi（德里印度信息技术学院）；NSIT Delhi（德里国家信息技术学院）；Space Research Institute (IWF) of Austrian Academy of Sciences（奥地利科学院空间研究所）
AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Models
Carnegie Mellon University（卡内基梅隆大学）；Brown University（布朗大学）；National Institute of Technology Karnataka（印度卡纳塔克国家理工学院）；University of Pittsburgh（匹兹堡大学）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Boosting Convolution with Efficient MLP-Permutation for Volumetric Medical Image Segmentation
Department of Computer Science and Engineering（计算机科学与工程系）；The Hong Kong University of Science and Technology（香港科学与技术大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（HKUST深圳-香港协同创新研究院）
Semi-supervised learning methods have been explored in medical image segmentation tasks due to the scarcity of pixel-level annotation in the real scenario. Proto-type alignment based consistency constraint is an intuitional and plausible solu-tion to explore the useful information in the unlabeled data. In this paper, we propose a mutual- and self- prototype alignment (MSPA) framework to better utilize the unlabeled data. In specific, mutual-prototype alignment enhances the information interaction between labeled and unlabeled data. The mutual-prototype alignment imposes two consistency constr…
UniSegDiff: Boosting Unified Lesion Segmentation via a Staged Diffusion Model
Dalian University of Technology, Dalian, China（大连理工大学）；Department of Urology, Affiliated Zhongshan Hospital of Dalian University, Dalian, China（大连大学附属中山医院泌尿外科部）
University of California, Los Angeles（加州大学洛杉矶分校）；NVIDIA（英伟达）
Fully convolutional neural networks (CNNs) have proven to be effective at representing and classifying textural information, thus transforming image intensity into output class masks that achieve semantic image segmentation. In medical image analysis, however, expert manual segmentation often relies on the boundaries of anatomical structures of interest. We propose boundary aware CNNs for medical image segmentation. Our networks are designed to account for organ boundary information, both by providing a special network edge branch and edge-aware loss terms, and they are trainable end-to-end. W…
Huazhong University of Science and Technology（华中科技大学）；College of Life Science and Technology, Huazhong University of Science and Technology（华中科技大学生命科学与技术学院）
Deformable medical image registration plays an important role in clinical diagnosis and treatment. Recently, the deep learning (DL) based image registration methods have been widely investigated and showed excellent performance in computational speed. However, these methods cannot provide enough registration accuracy because of insufficient ability in representing both the global and local features of the moving and fixed images. To address this issue, this paper has proposed the transformer based image registration method. This method uses the distinctive transformer to extract the global and…
Graph Neural Networks (GNNs) have been shown to be a powerful tool for generating predictions from biological data. Their application to neuroimaging data such as functional magnetic resonance imaging (fMRI) scans has been limited. However, applying GNNs to fMRI scans may substantially improve predictive accuracy and could be used to inform clinical diagnosis in the future. In this paper, we present a novel approach to representing resting-state fMRI data as a graph containing nodes and edges without omitting any of the voxels and thus reducing information loss. We compare multiple GNN archite…
Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art performance for segmentation tasks in computer vision and medical imaging. Recently, computational blocks termed squeeze and excitation (SE) have been introduced to recalibrate F-CNN feature maps both channel- and spatial-wise, boosting segmentation performance while only minimally increasing the model complexity. So far, the development of SE blocks has focused on 2D architectures. For volumetric medical images, however, 3D F-CNNs are a natural choice. In this article, we extend existing 2D recalibration methods to 3D and p…
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Convolutional Neural Networks (CNNs) have shown to be powerful medical image segmentation models. In this study, we address some of the main unresolved issues regarding these models. Specifically, training of these models on small medical image datasets is still challenging, with many studies promoting techniques such as transfer learning. Moreover, these models are infamous for producing over-confident predictions and for failing silently when presented with out-of-distribution (OOD) data at test time. In this paper, we advocate for multi-task learning, i.e., training a single model on severa…
