BSDA: Bayesian Random Semantic Data Augmentation for Medical Image Classification
Chengdu Institute of Computer Application, Chinese Academy of Sciences（中国科学院成都计算机应用研究所）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）
Data augmentation is a crucial regularization technique for deep neural networks, particularly in medical image classification. Mainstream data augmentation (DA) methods are usually applied at the image level. Due to the specificity and diversity of medical imaging, expertise is often required to design effective DA strategies, and improper augmentation operations can degrade model performance. Although automatic augmentation methods exist, they are computationally intensive. Semantic data augmentation can implemented by translating features in feature space. However, over-translation may violate the image label. To address these issues, we propose \emph{Bayesian Random Semantic Data Augmentation} (BSDA), a computationally efficient and handcraft-free feature-level DA method. BSDA uses variational Bayesian to estimate the distribution of the augmentable magnitudes, and then a sample from this distribution is added to the original features to perform semantic data augmentation. We performed experiments on nine 2D and five 3D medical image datasets. Experimental results show that BSDA outperforms current DA methods. Additionally, BSDA can be easily assembled into CNNs or Transformers as a plug-and-play module, improving the network's performance. The code is available online at \url{https://github.com/YaoyaoZhu19/BSDA}.
Automated data augmentation, which aims at engineering augmentation policy automatically, recently draw a growing research interest. Many previous auto-augmentation methods utilized a Density Matching strategy by evaluating policies in terms of the test-time augmentation performance. In this paper, we theoretically and empirically demonstrated the inconsistency between the train and validation set of small-scale medical image datasets, referred to as in-domain sampling bias. Next, we demonstrated that the in-domain sampling bias might cause the inefficiency of Density Matching. To address the …
Image segmentation is an important task in many medical applications. Methods based on convolutional neural networks attain state-of-the-art accuracy; however, they typically rely on supervised training with large labeled datasets. Labeling medical images requires significant expertise and time, and typical hand-tuned approaches for data augmentation fail to capture the complex variations in such images. We present an automated data augmentation method for synthesizing labeled medical images. We demonstrate our method on the task of segmenting magnetic resonance imaging (MRI) brain scans. Our …
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）；University of Electronic Science and Technology of China（电子科技大学）；University Hospitals Leuven（鲁汶大学医院）；Institute for Women’s Health（妇女健康研究所）
Despite the state-of-the-art performance for medical image segmentation, deep convolutional neural networks (CNNs) have rarely provided uncertainty estimations regarding their segmentation outputs, e.g., model (epistemic) and image-based (aleatoric) uncertainties. In this work, we analyze these different types of uncertainties for CNN-based 2D and 3D medical image segmentation tasks. We additionally propose a test-time augmentation-based aleatoric uncertainty to analyze the effect of different transformations of the input image on the segmentation output. Test-time augmentation has been previo…
University of Central Florida（中佛罗里达大学）；Northwestern University（西北大学）；Mayo Clinic（梅奥诊所）；Sheikh Shakhbout Medical City（谢赫·沙赫布特医疗城）
Current data augmentation techniques and transformations are well suited for improving the size and quality of natural image datasets but are not yet optimized for medical imaging. We hypothesize that sub-optimal data augmentations can easily distort or occlude medical images, leading to false positives or negatives during patient diagnosis, prediction, or therapy/surgery evaluation. In our experimental results, we found that utilizing commonly used intensity-based data augmentation distorts the MRI scans and leads to texture information loss, thus negatively affecting the overall performance …
While deep learning methods have shown great success in medical image analysis, they require a number of medical images to train. Due to data privacy concerns and unavailability of medical annotators, it is oftentimes very difficult to obtain a lot of labeled medical images for model training. In this paper, we study cross-modality data augmentation to mitigate the data deficiency issue in the medical imaging domain. We propose a discriminative unpaired image-to-image translation model which translates images in source modality into images in target modality where the translation task is condu…
Current medical image synthetic augmentation techniques rely on intensive use of generative adversarial networks (GANs). However, the nature of GAN architecture leads to heavy computational resources to produce synthetic images and the augmentation process requires multiple stages to complete. To address these challenges, we introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. The teacher learns to generate curriculum to feed into the student model for data augmentation and guides the stude…
Hebei University of Technology（河北工业大学）；University of Macau（澳门大学）；University of Oxford（牛津大学）；Harbin Institute of Technology（哈尔滨工业大学）
Recently deep learning methods, in particular, convolutional neural networks (CNNs), have led to a massive breakthrough in the range of computer vision. Also, the large-scale annotated dataset is the essential key to a successful training procedure. However, it is a huge challenge to get such datasets in the medical domain. Towards this, we present a data augmentation method for generating synthetic medical images using cycle-consistency Generative Adversarial Networks (GANs). We add semi-supervised attention modules to generate images with convincing details. We treat tumor images and normal …
Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学技术研究生院）；Research Center for Medical Bigdata, National Institute of Informatics（国立信息学研究所医疗大数据研究中心）
Convolutional Neural Network (CNN)-based accurate prediction typically requires large-scale annotated training data. In Medical Imaging, however, both obtaining medical data and annotating them by expert physicians are challenging; to overcome this lack of data, Data Augmentation (DA) using Generative Adversarial Networks (GANs) is essential, since they can synthesize additional annotated training data to handle small and fragmented medical images from various scanners--those generated images, realistic but completely novel, can further fill the real image distribution uncovered by the origina…
Tel Aviv University（特拉维夫大学）；The Chaim Sheba Medical Center（夏姆谢巴医疗中心）；Bar-Ilan University（巴伊兰大学）
Deep learning methods, and in particular convolutional neural networks (CNNs), have led to an enormous breakthrough in a wide range of computer vision tasks, primarily by using large-scale annotated datasets. However, obtaining such datasets in the medical domain remains a challenge. In this paper, we present methods for generating synthetic medical images using recently presented deep learning Generative Adversarial Networks (GANs). Furthermore, we show that generated medical images can be used for synthetic data augmentation, and improve the performance of CNN for medical image classificatio…
Deep learning models with large learning capacities often overfit to medical imaging datasets. This is because training sets are often relatively small due to the significant time and financial costs incurred in medical data acquisition and labelling. Data augmentation is therefore often used to expand the availability of training data and to increase generalization. However, augmentation strategies are often chosen on an ad-hoc basis without justification. In this paper, we present an augmentation policy search method with the goal of improving model classification performance. We include in …
University of Electronic Science and Technology of China（电子科技大学）；Shanghai AI Lab（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；ShanghaiTech University（上海科技大学）；Sensetime Research（商汤科技研究院）；Sichuan University（四川大学）；Sichuan Cancer Hospital and Institute（四川省肿瘤医院（四川省肿瘤研究所））
Generalization to previously unseen images with potential domain shifts and different styles is essential for clinically applicable medical image segmentation, and the ability to disentangle domain-specific and domain-invariant features is key for achieving Domain Generalization (DG). However, existing DG methods can hardly achieve effective disentanglement to get high generalizability. To deal with this problem, we propose an efficient Contrastive Domain Disentanglement and Style Augmentation (CDDSA) framework for generalizable medical image segmentation. First, a disentangle network is propo…
MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis
Department of Mechanical Engineering, National University of Singapore（新加坡国立大学机械工程系）；School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）；Department of Electronic Engineering, Jinan University（济南大学电子工程系）
GenMix: Combining Generative and Mixture Data Augmentation for Medical Image Classification
School of Electrical Engineering（电气工程学院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Department of Software Convergence（软件融合系）；Seoul Women’s University（首尔女子大学）
Self-supervised Visualisation of Medical Image Datasets
Hertie Institute for AI in Brain Health（赫蒂脑健康人工智能研究院）；University of Tübingen（蒂宾根大学）；Tübingen AI Center（蒂宾根人工智能中心）；Heidelberg University（海德堡大学）；IWR, Heidelberg University（海德堡大学IWR研究所）
Rutgers University（罗格斯大学）；SenseBrain；Shanghai AI Laboratory（上海人工智能实验室）；Centre for Perceptual and Interactive Intelligence（感知与交互智能中心）
Data augmentation has proved extremely useful by increasing training data variance to alleviate overfitting and improve deep neural networks' generalization performance. In medical image analysis, a well-designed augmentation policy usually requires much expert knowledge and is difficult to generalize to multiple tasks due to the vast discrepancies among pixel intensities, image appearances, and object shapes in different medical tasks. To automate medical data augmentation, we propose a regularized adversarial training framework via two min-max objectives and three differentiable augmentation…
Xiamen University（厦门大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Deepwise AI Laboratory（深睿医疗人工智能实验室）
Medical imaging datasets usually exhibit domain shift due to the variations of scanner vendors, imaging protocols, etc. This raises the concern about the generalization capacity of machine learning models. Domain generalization (DG), which aims to learn a model from multiple source domains such that it can be directly generalized to unseen test domains, seems particularly promising to medical imaging community. To address DG, recent model-agnostic meta-learning (MAML) has been introduced, which transfers the knowledge from previous training tasks to facilitate the learning of novel testing tas…
pyMEAL: A Multi-Encoder Augmentation-Aware-Learning Toolbox for Robust Medical Image Translation
Hong Kong Centre for Cerebro-Cardiovascular Health Engineering (COCHE)（香港脑心血管健康工程中心）；Department of Informatics, Sokoine University of Agriculture（农业科学学院信息系）；Department of Diagnostic Radiology, The University of Hong Kong（香港大学诊断放射科）；State Key Laboratory of Brain and Cognitive Sciences, The University of Hong Kong（香港大学脑科学与认知科学国家重点实验室）；Alzheimer’s Disease Research Network, The University of Hong Kong（香港大学阿尔茨海默病研究网络）；Department of Biomedical Engineering, City University of Hong Kong（城市大学生物医学工程系）；Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院放射学与放射科学系）；City University of Hong Kong Shenzhen Research Institute（城市大学深圳研究 institute）
University of New South Wales（新南威尔士大学）；Stanford University（斯坦福大学）；Lehigh University（理海大学）
The shortage of annotated medical images is one of the biggest challenges in the field of medical image computing. Without a sufficient number of training samples, deep learning based models are very likely to suffer from over-fitting problem. The common solution is image manipulation such as image rotation, cropping, or resizing. Those methods can help relieve the over-fitting problem as more training samples are introduced. However, they do not really introduce new images with additional information and may lead to data leakage as the test set may contain similar samples which appear in the …
The Pennsylvania State University（宾夕法尼亚州立大学）；National Institutes of Health（美国国立卫生研究院）
Supervised training of an automated medical image analysis system often requires a large amount of expert annotations that are hard to collect. Moreover, the proportions of data available across different classes may be highly imbalanced for rare diseases. To mitigate these issues, we investigate a novel data augmentation pipeline that selectively adds new synthetic images generated by conditional Adversarial Networks (cGANs), rather than extending directly the training set with synthetic images. The selection mechanisms that we introduce to the synthetic augmentation pipeline are motivated by…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Lenovo Corporate Research & Development, Lenovo Ltd.（联想有限公司联想企业研究院）
Deep learning models are notoriously data-hungry. Thus, there is an urging need for data-efficient techniques in medical image analysis, where well-annotated data are costly and time consuming to collect. Motivated by the recently revived "Copy-Paste" augmentation, we propose TumorCP, a simple but effective object-level data augmentation method tailored for tumor segmentation. TumorCP is online and stochastic, providing unlimited augmentation possibilities for tumors' subjects, locations, appearances, as well as morphologies. Experiments on kidney tumor segmentation task demonstrate that Tumor…
In the realm of medical imaging, the training of machine learning models necessitates a large and varied training dataset to ensure robustness and interoperability. However, acquiring such diverse and heterogeneous data can be difficult due to the need for expert labeling of each image and privacy concerns associated with medical data. To circumvent these challenges, data augmentation has emerged as a promising and cost-effective technique for increasing the size and diversity of the training dataset. In this study, we provide a comprehensive review of the specific data augmentation techniques…
An important component of human analysis of medical images and their context is the ability to relate newly seen things to related instances in our memory. In this paper we mimic this ability by using multi-modal retrieval augmentation and apply it to several tasks in chest X-ray analysis. By retrieving similar images and/or radiology reports we expand and regularize the case at hand with additional knowledge, while maintaining factual knowledge consistency. The method consists of two components. First, vision and language modalities are aligned using a pre-trained CLIP model. To enforce that …
Southeast University（东南大学）；Centre de Recherche en Information Biomédicale Sino-Français (CRIBs)（中法生物医学信息研究中心）；Univ Rennes（雷恩大学）；Inserm（法国国家健康与医学研究院）；LTSI - UMR1099（信号与图像处理实验室）
Deep learning-based medical image registration and segmentation joint models utilize the complementarity (augmentation data or weakly supervised data from registration, region constraints from segmentation) to bring mutual improvement in complex scene and few-shot situation. However, further adoption of the joint models are hindered: 1) the diversity of augmentation data is reduced limiting the further enhancement of segmentation, 2) misaligned regions in weakly supervised data disturb the training process, 3) lack of label-based region constraints in few-shot situation limits the registration…
PairAug: What Can Augmented Image-Text Pairs Do for Radiology?
Australian Institute for Machine Learning (AIML)（澳大利亚机器学习研究所）；The University of Adelaide（阿德莱德大学）；South Australia Medical Imaging（南澳大利亚医学影像中心）；School of Computer Science and Engineering, Northwestern Polytechnical University（西北工业大学计算机科学与技术学院）；Ningbo Institute of Northwestern Polytechnical University（西北工业大学宁波研究院）
Dublin City University（都柏林城市大学）；University College Dublin（都柏林大学学院）
Over the years, the paradigm of medical image analysis has shifted from manual expertise to automated systems, often using deep learning (DL) systems. The performance of deep learning algorithms is highly dependent on data quality. Particularly for the medical domain, it is an important aspect as medical data is very sensitive to quality and poor quality can lead to misdiagnosis. To improve the diagnostic performance, research has been done both in complex DL architectures and in improving data quality using dataset dependent static hyperparameters. However, the performance is still constraine…
University College London（伦敦大学学院）；King’s College London（伦敦国王学院）
Processing of medical images such as MRI or CT presents unique challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment of volumes. We present TorchIO, an open-source Python library to enable effi…
Flow Matching with Optimized Subclass Priors for Medical Image Augmentation
Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, 91052 Erlangen, Germany（生物医学工程人工智能系，弗里德里希-亚历山大-厄林根-纽伦堡大学）；Department of Computing, Imperial College London, London SW7 2AZ, UK（计算系，帝国理工学院伦敦分校）
DIKU University of Copenhagen（哥本哈根大学DIKU）；Biomediq A/S（Biomediq公司）；Cerebriu A/S（Cerebriu公司）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
For proper generalization performance of convolutional neural networks (CNNs) in medical image segmentation, the learnt features should be invariant under particular non-linear shape variations of the input. To induce invariance in CNNs to such transformations, we propose Probabilistic Augmentation of Data using Diffeomorphic Image Transformation (PADDIT) -- a systematic framework for generating realistic transformations that can be used to augment data for training CNNs. We show that CNNs trained with PADDIT outperforms CNNs trained without augmentation and with generic augmentation in segmen…
Interpretability of deep learning (DL) systems is gaining attention in medical imaging to increase experts' trust in the obtained predictions and facilitate their integration in clinical settings. We propose a deep visualization method to generate interpretability of DL classification tasks in medical imaging by means of visual evidence augmentation. The proposed method iteratively unveils abnormalities based on the prediction of a classifier trained only with image-level labels. For each image, initial visual evidence of the prediction is extracted with a given visual attribution technique. T…
Medical image classification is one of the most critical problems in the image recognition area. One of the major challenges in this field is the scarcity of labelled training data. Additionally, there is often class imbalance in datasets as some cases are very rare to happen. As a result, accuracy in classification task is normally low. Deep Learning models, in particular, show promising results on image segmentation and classification problems, but they require very large datasets for training. Therefore, there is a need to generate more of synthetic samples from the same distribution. Previ…
Imperial College London（帝国理工学院）；Technical University of Munich（慕尼黑工业大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）；University of Edinburgh（爱丁堡大学）
Deep learning-based segmentation methods are vulnerable to unforeseen data distribution shifts during deployment, e.g. change of image appearances or contrasts caused by different scanners, unexpected imaging artifacts etc. In this paper, we present a cooperative framework for training image segmentation models and a latent space augmentation method for generating hard examples. Both contributions improve model generalization and robustness with limited data. The cooperative training framework consists of a fast-thinking network (FTN) and a slow-thinking network (STN). The FTN learns decoupled…
