Transfer learning has become an essential part of medical imaging classification algorithms, often leveraging ImageNet weights. The domain shift from natural to medical images has prompted alternatives such as RadImageNet, often showing comparable classification performance. However, it remains unclear whether the performance gains from transfer learning stem from improved generalization or shortcut learning. To address this, we conceptualize confounders by introducing the Medical Imaging Contextualized Confounder Taxonomy (MICCAT) and investigate a range of confounders across it -- whether synthetic or sampled from the data -- using two public chest X-ray and CT datasets. We show that ImageNet and RadImageNet achieve comparable classification performance, yet ImageNet is much more prone to overfitting to confounders. We recommend that researchers using ImageNet-pretrained models reexamine their model robustness by conducting similar experiments. Our code and experiments are available at https://github.com/DovileDo/source-matters.
This compendium gathers all the accepted extended abstracts from the Third International Conference on Medical Imaging with Deep Learning (MIDL 2020), held in Montreal, Canada, 6-9 July 2020. Note that only accepted extended abstracts are listed here, the Proceedings of the MIDL 2020 Full Paper Track are published in the Proceedings of Machine Learning Research (PMLR).
Incorporating boundaries of the imaging object as a priori information to imaging algorithms can significantly improve the performance of electromagnetic medical imaging systems. To avoid overly complicating the system by using different sensors and the adverse effect of the subject's movement, a learning-based method is proposed to estimate the boundary (external contour) of the imaged object using the same electromagnetic imaging data. While imaging techniques may discard the reflection coefficients for being dominant and uninformative for imaging, these parameters are made use of for bounda…
Queensland University of Technology（昆士兰科技大学）；University of Jaen（哈恩大学）；The University of Queensland（昆士兰大学）；University of Sumer（苏美尔大学）；University of Baghdad（巴格达大学）；The University of Information Technology & Communications（信息与通信技术大学）；University of Missouri（密苏里大学）
Deep Learning (DL) requires a large amount of training data to provide quality outcomes. However, the field of medical imaging suffers from the lack of sufficient data for properly training DL models because medical images require manual labelling carried out by clinical experts thus the process is time-consuming, expensive, and error-prone. Recently, transfer learning (TL) was introduced to reduce the need for the annotation procedure by means of transferring the knowledge performed by a previous task and then fine-tuning the result using a relatively small dataset. Nowadays, multiple classif…
We propose performing imbalanced classification by regrouping majority classes into small classes so that we turn the problem into balanced multiclass classification. This new idea is dramatically different from popular loss reweighting and class resampling methods. Our preliminary result on imbalanced medical image classification shows that this natural idea can substantially boost the classification performance as measured by average precision (approximately area-under-the-precision-recall-curve, or AUPRC), which is more appropriate for evaluating imbalanced classification than other metrics…
Benchmarking Dependence Measures to Prevent Shortcut Learning in Medical Imaging
Hertie Institute for AI in Brain Health, University of Tübingen（蒂宾根大学赫蒂脑健康人工智能研究所）；University Hospital of Tübingen（蒂宾根大学医院）；University of Stuttgart（斯图加特大学）；University of Bern（伯尔尼大学）；Inselspital, Bern University Hospital（伯尔尼大学医院 Inselspital）；Stanford University（斯坦福大学）
Jordan University of Science and Technology（约旦科技大学）
The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the fiel…
Beyond Knowledge Silos: Task Fingerprinting for Democratization of Medical Imaging AI
National Center for Tumor Diseases (NCT)（肿瘤疾病国家中心）；German Cancer Research Center (DKFZ) Heidelberg（德国癌症研究中心（Heidelberg））；Faculty of Mathematics and Computer Science（数学和计算机科学学院）；Heidelberg University（海德堡大学）；HIDSS4Health - Helmholtz Information and Data Science School for Health（HIDSS4Health - 健康信息与数据科学学校）；Medical Faculty（医学学院）；Pattern Analysis and Learning Group（模式分析与学习小组）
While deep learning methods have shown great success in medical image analysis, they require a number of medical images to train. Due to data privacy concerns and unavailability of medical annotators, it is oftentimes very difficult to obtain a lot of labeled medical images for model training. In this paper, we study cross-modality data augmentation to mitigate the data deficiency issue in the medical imaging domain. We propose a discriminative unpaired image-to-image translation model which translates images in source modality into images in target modality where the translation task is condu…
Medical imaging AI systems such as disease classification and segmentation are increasingly inspired and transformed from computer vision based AI systems. Although an array of adversarial training and/or loss function based defense techniques have been developed and proved to be effective in computer vision, defending against adversarial attacks on medical images remains largely an uncharted territory due to the following unique challenges: 1) label scarcity in medical images significantly limits adversarial generalizability of the AI system; 2) vastly similar and dominant fore- and backgroun…
Transfer learning from natural image datasets, particularly ImageNet, using standard large models and corresponding pretrained weights has become a de-facto method for deep learning applications to medical imaging. However, there are fundamental differences in data sizes, features and task specifications between natural image classification and the target medical tasks, and there is little understanding of the effects of transfer. In this paper, we explore properties of transfer learning for medical imaging. A performance evaluation on two large scale medical imaging tasks shows that surprisin…
Recent work has investigated the distributions of learned convolution filters through a large-scale study containing hundreds of heterogeneous image models. Surprisingly, on average, the distributions only show minor drifts in comparisons of various studied dimensions including the learned task, image domain, or dataset. However, among the studied image domains, medical imaging models appeared to show significant outliers through "spikey" distributions, and, therefore, learn clusters of highly specific filters different from other domains. Following this observation, we study the collected med…
This compendium gathers all the accepted extended abstracts from the Second International Conference on Medical Imaging with Deep Learning (MIDL 2019), held in London, UK, 8-10 July 2019. Note that only accepted extended abstracts are listed here, the Proceedings of the MIDL 2019 Full Paper Track are published as Volume 102 of the Proceedings of Machine Learning Research (PMLR) http://proceedings.mlr.press/v102/.
Microsoft Health and Life Sciences（微软健康与生命科学部）；Stanford Center for Artificial Intelligence in Medicine and Imaging（斯坦福医学与成像人工智能中心）
Clinical Artificial lntelligence (AI) applications are rapidly expanding worldwide, and have the potential to impact to all areas of medical practice. Medical imaging applications constitute a vast majority of approved clinical AI applications. Though healthcare systems are eager to adopt AI solutions a fundamental question remains: \textit{what happens after the AI model goes into production?} We use the CheXpert and PadChest public datasets to build and test a medical imaging AI drift monitoring workflow to track data and model drift without contemporaneous ground truth. We simulate drift in…
Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Karolinska Institute（卡罗林斯卡学院）；Karolinska University Hospital（卡罗林斯卡大学医院）
Recently, we have witnessed great progress in the field of medical imaging classification by adopting deep neural networks. However, the recent advanced models still require accessing sufficiently large and representative datasets for training, which is often unfeasible in clinically realistic environments. When trained on limited datasets, the deep neural network is lack of generalization capability, as the trained deep neural network on data within a certain distribution (e.g. the data captured by a certain device vendor or patient population) may not be able to generalize to the data with a…
California Institute of Technology（加州理工学院）；Stanford University（斯坦福大学）
Magnetic Resonance Imaging (MRI) suffers from several artifacts, the most common of which are motion artifacts. These artifacts often yield images that are of non-diagnostic quality. To detect such artifacts, images are prospectively evaluated by experts for their diagnostic quality, which necessitates patient-revisits and rescans whenever non-diagnostic quality scans are encountered. This motivates the need to develop an automated framework capable of accessing medical image quality and detecting diagnostic and non-diagnostic images. In this paper, we explore several convolutional neural netw…
In this work we examine the performance enhancement in classification of medical imaging data when image features are combined with associated non-image data. We compare the performance of eight state-of-the-art deep neural networks in classification tasks when using only image features, compared to when these are combined with patient metadata. We utilise transfer learning with networks pretrained on ImageNet used directly as feature extractors and fine tuned on the target domain. Our experiments show that performance can be significantly enhanced with the inclusion of metadata and use interp…
Application of deep neural networks to medical imaging tasks has in some sense become commonplace. Still, a "thorn in the side" of the deep learning movement is the argument that deep networks are prone to overfitting and are thus unable to generalize well when datasets are small (as is common in medical imaging tasks). One way to bolster confidence is to provide mathematical guarantees, or bounds, on network performance after training which explicitly quantify the possibility of overfitting. In this work, we explore recent advances using the PAC-Bayesian framework to provide bounds on general…
Digital medical images are always displayed scaled to fit particular view. Interpolation is responsible for this scaling, and if not done properly, can significantly degrade diagnostic image quality. However, theoretically-optimal interpolation algorithms may also be the most time-consuming and impractical. We propose a new approach, adapted to the needs of digital medical imaging, to combine high interpolation speed and superior L2-optimal image quality.
Google Research（谷歌研究院）；Georgia Institute of Technology（佐治亚理工学院）；Northwestern University（西北大学）
Recent progress in Medical Artificial Intelligence (AI) has delivered systems that can reach clinical expert level performance. However, such systems tend to demonstrate sub-optimal "out-of-distribution" performance when evaluated in clinical settings different from the training environment. A common mitigation strategy is to develop separate systems for each clinical setting using site-specific data [1]. However, this quickly becomes impractical as medical data is time-consuming to acquire and expensive to annotate [2]. Thus, the problem of "data-efficient generalization" presents an ongoing …
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Arkansas（阿肯色大学）；University of Houston（休斯顿大学）；INRIA, Sophia Antipolis-Mediterranean Centre（法国国家信息与自动化研究所索菲亚·安蒂波利斯地中海中心）
Deep reinforcement learning (DRL) augments the reinforcement learning framework, which learns a sequence of actions that maximizes the expected reward, with the representative power of deep neural networks. Recent works have demonstrated the great potential of DRL in medicine and healthcare. This paper presents a literature review of DRL in medical imaging. We start with a comprehensive tutorial of DRL, including the latest model-free and model-based algorithms. We then cover existing DRL applications for medical imaging, which are roughly divided into three main categories: (I) parametric med…
Transformer, a model comprising attention-based encoder-decoder architecture, have gained prevalence in the field of natural language processing (NLP) and recently influenced the computer vision (CV) space. The similarities between computer vision and medical imaging, reviewed the question among researchers if the impact of transformers on computer vision be translated to medical imaging? In this paper, we attempt to provide a comprehensive and recent review on the application of transformers in medical imaging by; describing the transformer model comparing it with a diversity of convolutional…
Transfer learning is a standard technique to improve performance on tasks with limited data. However, for medical imaging, the value of transfer learning is less clear. This is likely due to the large domain mismatch between the usual natural-image pre-training (e.g. ImageNet) and medical images. However, recent advances in transfer learning have shown substantial improvements from scale. We investigate whether modern methods can change the fortune of transfer learning for medical imaging. For this, we study the class of large-scale pre-trained networks presented by Kolesnikov et al. on three …
In this paper we broadly consider techniques which utilize projections on rays for data collection, with particular emphasis on optical techniques. We formulate a variety of imaging techniques as either special cases or extensions of tomographic reconstruction. We then consider how the techniques must be extended to describe objects containing occlusion, as with a self-occluding opaque object. We formulate the reconstruction problem as a regularized nonlinear optimization problem to simultaneously solve for object brightness and attenuation, where the attenuation can become infinite. We demons…
Sun Yat-sen University（中山大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Xi’an Jiaotong University（西安交通大学）；Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院北京协和医院）；University of Portsmouth（朴茨茅斯大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；Qingdao Academy of Intelligent Industries（青岛智能产业技术研究院）
There has been much progress in data-driven artificial intelligence technology for medical image analysis in the last decades. However, it still remains challenging due to its distinctive complexity of acquiring and annotating image data, extracting medical domain knowledge, and explaining the diagnostic decision for medical image analysis. In this paper, we propose a data-knowledge-driven framework termed as Parallel Medical Imaging (PMI) for intelligent medical image analysis based on the methodology of interactive ACP-based parallel intelligence. In the PMI framework, computational experime…
Gordon Center for Medical Imaging（戈登医学影像中心）；Massachusetts General Hospital（麻省总医院）；Martinos Center for Biomedical Imaging（马蒂诺斯生物医学影像中心）；University of California, Davis（加州大学戴维斯分校）
Recently deep neural networks have been widely and successfully applied in computer vision tasks and attracted growing interests in medical imaging. One barrier for the application of deep neural networks to medical imaging is the need of large amounts of prior training pairs, which is not always feasible in clinical practice. In this work we propose a personalized representation learning framework where no prior training pairs are needed, but only the patient's own prior images. The representation is expressed using a deep neural network with the patient's prior images as network input. We th…
Multi-environment lifelong deep reinforcement learning for medical imaging
Rice University（莱斯大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）；McGovern Medical School, UTHealth Houston（休斯顿UTHealth麦戈文医学院）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；University of Maryland Medical Intelligent Imaging (UM2ii)（马里兰大学医学智能成像中心）；University of Maryland School of Medicine（马里兰大学医学院）
Haukeland University Hospital（豪克兰大学医院）；Western Norway University of Applied Sciences（西挪威应用科学大学）；University of Bergen（卑尔根大学）
What has happened in machine learning lately, and what does it mean for the future of medical image analysis? Machine learning has witnessed a tremendous amount of attention over the last few years. The current boom started around 2009 when so-called deep artificial neural networks began outperforming other established models on a number of important benchmarks. Deep neural networks are now the state-of-the-art machine learning models across a variety of areas, from image analysis to natural language processing, and widely deployed in academia and industry. These developments have a huge poten…
Image-based characterization and disease understanding involve integrative analysis of morphological, spatial, and topological information across biological scales. The development of graph convolutional networks (GCNs) has created the opportunity to address this information complexity via graph-driven architectures, since GCNs can perform feature aggregation, interaction, and reasoning with remarkable flexibility and efficiency. These GCNs capabilities have spawned a new wave of research in medical imaging analysis with the overarching goal of improving quantitative disease understanding, mon…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Generative adversarial networks (GANs) have been widely investigated for many potential applications in medical imaging. DatasetGAN is a recently proposed framework based on modern GANs that can synthesize high-quality segmented images while requiring only a small set of annotated training images. The synthesized annotated images could be potentially employed for many medical imaging applications, where images with segmentation information are required. However, to the best of our knowledge, there are no published studies focusing on its applications to medical imaging. In this work, prelimina…
This work provides a strong baseline for the problem of multi-source multi-target domain adaptation and generalization in medical imaging. Using a diverse collection of ten chest X-ray datasets, we empirically demonstrate the benefits of training medical imaging deep learning models on varied patient populations for generalization to out-of-sample domains.
KU Leuven（鲁汶大学）；Vector Institute（向量研究所）；NVIDIA（英伟达）；University of Toronto（多伦多大学）；University of Leeds（利兹大学）
Labelling data is expensive and time consuming especially for domains such as medical imaging that contain volumetric imaging data and require expert knowledge. Exploiting a larger pool of labeled data available across multiple centers, such as in federated learning, has also seen limited success since current deep learning approaches do not generalize well to images acquired with scanners from different manufacturers. We aim to address these problems in a common, learning-based image simulation framework which we refer to as Federated Simulation. We introduce a physics-driven generative appro…
Stanford University（斯坦福大学）；Columbia University（哥伦比亚大学）；Stanford Artificial Intelligence in Medicine Center（斯坦福医学人工智能中心）
Advances in computing power, deep learning architectures, and expert labelled datasets have spurred the development of medical imaging artificial intelligence systems that rival clinical experts in a variety of scenarios. The National Institutes of Health in 2018 identified key focus areas for the future of artificial intelligence in medical imaging, creating a foundational roadmap for research in image acquisition, algorithms, data standardization, and translatable clinical decision support systems. Among the key issues raised in the report: data availability, need for novel computing archite…
Towards Data-Efficient Medical Imaging: A Generative and Semi-Supervised Framework
Dept. of Electrical and Electronic Engineering Imperial College London（电气与电子工程系帝国理工学院伦敦）；Centre for Vision, Speech and Signal Processing University of Surrey（视觉、语音和信号处理中心萨里大学）
Chinese Academy of Sciences（中国科学院）；Johns Hopkins Medical Institutes（约翰斯·霍普金斯医学院）；Vanderbilt University（范德堡大学）；University of Science and Technology of China（中国科学技术大学）
Transformer, the latest technological advance of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that cha…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；Siemens Healthineers（西门子医疗）
Although deep learning prediction models have been successful in the discrimination of different classes, they can often suffer from poor calibration across challenging domains including healthcare. Moreover, the long-tail distribution poses great challenges in deep learning classification problems including clinical disease prediction. There are approaches proposed recently to calibrate deep prediction in computer vision, but there are no studies found to demonstrate how the representative models work in different challenging contexts. In this paper, we bridge the confidence calibration from …
Towards Universal Unsupervised Anomaly Detection in Medical Imaging
Technical University of Munich（慕尼黑工业大学）；Helmholtz AI and Helmholtz Center Munich（亥姆霍兹人工智能与亥姆霍兹慕尼黑中心）；Klinikum Rechts der Isar（伊萨尔河右岸医院）；Imperial College London（伦敦帝国理工学院）
Deep neural networks (DNN) have achieved unprecedented performance in computer-vision tasks almost ubiquitously in business, technology, and science. While substantial efforts are made to engineer highly accurate architectures and provide usable model explanations, most state-of-the-art approaches are first designed for natural vision and then translated to the medical domain. This dissertation seeks to address this gap by proposing novel architectures that integrate the domain-specific constraints of medical imaging into the DNN model and explanation design.
Polytechnique Montreal（蒙特利尔综合理工大学）；Pontifícia Universidade Católica do Rio Grande do Sul（南里奥格兰德天主教大学）；Universite de Montreal（蒙特利尔大学）
Recent advances in deep learning methods have come to define the state-of-the-art for many medical imaging applications, surpassing even human judgment in several tasks. Those models, however, when trained to reduce the empirical risk on a single domain, fail to generalize when applied to other domains, a very common scenario in medical imaging due to the variability of images and anatomical structures, even across the same imaging modality. In this work, we extend the method of unsupervised domain adaptation using self-ensembling for the semantic segmentation task and explore multiple facets …
Recent Advances in Medical Imaging Segmentation: A Survey
Junia, UMR 8520, CNRS, Centrale Lille, Univerity of Polytechnique Hauts-de-France（Junia研究院、法国国家科学研究中心、Centrale Lille、法国高等技术大学 Hauts-de-France）；Sorbonne Center for Artificial Intelligence (SCAI), Abu Dhabi, UAE（索邦人工智能中心（SCAI））
A Systematic Review of Low-Rank and Local Low-Rank Matrix Approximation in Big Data Medical Imaging
Rhodes University（罗德斯大学）；Osnabrück University（奥斯纳布吕克大学）；International Centre for Radio Astronomy Research (ICRAR)（国际射电天文研究中心）；Curtin University（科廷大学）；ARC Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)（ARC全天天体物理学卓越中心（ASTRO 3D））；Curtin Institute of Radio Astronomy（科廷射电天文研究所）
The optical resolution of a digital camera is one of its most crucial parameters with broad relevance for consumer electronics, surveillance systems, remote sensing, or medical imaging. However, resolution is physically limited by the optics and sensor characteristics. In addition, practical and economic reasons often stipulate the use of out-dated or low-cost hardware. Super-resolution is a class of retrospective techniques that aims at high-resolution imagery by means of software. Multi-frame algorithms approach this task by fusing multiple low-resolution frames to reconstruct high-resolutio…
Limited availability of annotated medical imaging data poses a challenge for deep learning algorithms. Although transfer learning minimizes this hurdle in general, knowledge transfer across disparate domains is shown to be less effective. On the other hand, smaller architectures were found to be more compelling in learning better features. Consequently, we propose a lightweight architecture that uses mixed asymmetric kernels (MAKNet) to reduce the number of parameters significantly. Additionally, we train the proposed architecture using semi-supervised learning to provide pseudo-labels for a l…
