Current deep-learning based methods do not easily integrate to clinical protocols, neither take full advantage of medical knowledge. In this work, we propose and compare several strategies relying on curriculum learning, to support the classification of proximal femur fracture from X-ray images, a challenging problem as reflected by existing intra- and inter-expert disagreement. Our strategies are derived from knowledge such as medical decision trees and inconsistencies in the annotations of multiple experts, which allows us to assign a degree of difficulty to each training sample. We demonstrate that if we start learning "easy" examples and move towards "hard", the model can reach a better performance, even with fewer data. The evaluation is performed on the classification of a clinical dataset of about 1000 X-ray images. Our results show that, compared to class-uniform and random strategies, the proposed medical knowledge-based curriculum, performs up to 15% better in terms of accuracy, achieving the performance of experienced trauma surgeons.
Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高级研究学院）；Ecole Centrale de Nantes（南特中央理工学院）；Technische Universität München（慕尼黑工业大学）；LMU München（慕尼黑大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
An adequate classification of proximal femur fractures from X-ray images is crucial for the treatment choice and the patients' clinical outcome. We rely on the commonly used AO system, which describes a hierarchical knowledge tree classifying the images into types and subtypes according to the fracture's location and complexity. In this paper, we propose a method for the automatic classification of proximal femur fractures into 3 and 7 AO classes based on a Convolutional Neural Network (CNN). As it is known, CNNs need large and representative datasets with reliable labels, which are hard to co…
University of Pittsburgh（匹兹堡大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；Children’s Hospital of Pittsburgh（匹兹堡儿童医院）
Elbow fracture diagnosis often requires patients to take both frontal and lateral views of elbow X-ray radiographs. In this paper, we propose a multiview deep learning method for an elbow fracture subtype classification task. Our strategy leverages transfer learning by first training two single-view models, one for frontal view and the other for lateral view, and then transferring the weights to the corresponding layers in the proposed multiview network architecture. Meanwhile, quantitative medical knowledge was integrated into the training process through a curriculum learning framework, whic…
We explore different curriculum learning methods for training convolutional neural networks on the task of deformable pairwise 3D medical image registration. To the best of our knowledge, we are the first to attempt to improve performance by training medical image registration models using curriculum learning, starting from an easy training setup in the first training stages, and gradually increasing the complexity of the setup. On the one hand, we consider two existing curriculum learning approaches, namely curriculum dropout and curriculum by smoothing. On the other hand, we propose a novel …
Elbow fractures are one of the most common fracture types. Diagnoses on elbow fractures often need the help of radiographic imaging to be read and analyzed by a specialized radiologist with years of training. Thanks to the recent advances of deep learning, a model that can classify and detect different types of bone fractures needs only hours of training and has shown promising results. However, most existing deep learning models are purely data-driven, lacking incorporation of known domain knowledge from human experts. In this work, we propose a novel deep learning method to diagnose elbow fr…
PAII Inc.（PAII公司）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Exploiting available medical records to train high performance computer-aided diagnosis (CAD) models via the semi-supervised learning (SSL) setting is emerging to tackle the prohibitively high labor costs involved in large-scale medical image annotations. Despite the extensive attentions received on SSL, previous methods failed to 1) account for the low disease prevalence in medical records and 2) utilize the image-level diagnosis indicated from the medical records. Both issues are unique to SSL for CAD models. In this work, we propose a new knowledge distillation method that effectively explo…
University of Pittsburgh（匹兹堡大学）；Loma Linda University（罗马琳达大学）
Curriculum learning is a learning method that trains models in a meaningful order from easier to harder samples. A key here is to devise automatic and objective difficulty measures of samples. In the medical domain, previous work applied domain knowledge from human experts to qualitatively assess classification difficulty of medical images to guide curriculum learning, which requires extra annotation efforts, relies on subjective human experience, and may introduce bias. In this work, we propose a new automated curriculum learning technique using the variance of gradients (VoG) to compute an o…
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Universität Ulm（乌尔姆大学）
Effective semi-supervised learning (SSL) in medical image analysis (MIA) must address two challenges: 1) work effectively on both multi-class (e.g., lesion classification) and multi-label (e.g., multiple-disease diagnosis) problems, and 2) handle imbalanced learning (because of the high variance in disease prevalence). One strategy to explore in SSL MIA is based on the pseudo labelling strategy, but it has a few shortcomings. Pseudo-labelling has in general lower accuracy than consistency learning, it is not specifically designed for both multi-class and multi-label problems, and it can be cha…
Robust Source-Free Domain Adaptation for Medical Image Segmentation based on Curriculum Learning
Shanghai Digital Medicine Innovation Center, Ruijin Hospital（上海数字医学创新中心，瑞金医院）；Tsinghua University（清华大学）；Medical AI ReSearch (MARS) Group, Guangxi Key Laboratory for Genomic and Personalized Medicine, Guangxi Medical University（医学AI研究组（MARS）, 广西基因组与个性化医学重点实验室，广西医科大学）；Jarvis Research Center, Tencent Youtu Lab（Jarvis研究中心，腾讯优图实验室）；University of California, Berkeley（加州大学伯克利分校）；Westlake University（西湖大学）
Current medical image synthetic augmentation techniques rely on intensive use of generative adversarial networks (GANs). However, the nature of GAN architecture leads to heavy computational resources to produce synthetic images and the augmentation process requires multiple stages to complete. To address these challenges, we introduce a novel generative meta curriculum learning method that trains the task-specific model (student) end-to-end with only one additional teacher model. The teacher learns to generate curriculum to feed into the student model for data augmentation and guides the stude…
Hiroshima University（广岛大学）；Nagoya Institute of Technology（名古屋工业大学）
In this paper we propose a novel method which leverages the uncertainty measures provided by Bayesian deep networks through curriculum learning so that the uncertainty estimates are fed back to the system to resample the training data more densely in areas where uncertainty is high. We show in the concrete setting of a semantic segmentation task (iPS cell colony segmentation) that the proposed system is able to increase significantly the reliability of the model.
MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
A.I.MATICS Inc（A.I.MATICS公司）；Boston University（波士顿大学）；University of Southern California（南加州大学）；Heuron（Heuron公司）；MODULABS, Open Neural Networks Research Lab（MODULABS，开放神经网络研究实验室）
Shenzhen University（深圳大学）；Universitat de Barcelona（巴塞罗那大学）；University of Leeds（利兹大学）；KU Leuven（荷语鲁汶大学）
The performance of deep segmentation models often degrades due to distribution shifts in image intensities between the training and test data sets. This is particularly pronounced in multi-centre studies involving data acquired using multi-vendor scanners, with variations in acquisition protocols. It is challenging to address this degradation because the shift is often not known \textit{a priori} and hence difficult to model. We propose a novel framework to ensure robust segmentation in the presence of such distribution shifts. Our contribution is three-fold. First, inspired by the spirit of c…
Imperial College London（帝国理工学院）；National University of Singapore（新加坡国立大学）；National Institute of Technology Tiruchirappalli（蒂鲁吉拉伯利国家理工学院）；The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
Purpose: In curriculum learning, the idea is to train on easier samples first and gradually increase the difficulty, while in self-paced learning, a pacing function defines the speed to adapt the training progress. While both methods heavily rely on the ability to score the difficulty of data samples, an optimal scoring function is still under exploration. Methodology: Distillation is a knowledge transfer approach where a teacher network guides a student network by feeding a sequence of random samples. We argue that guiding student networks with an efficient curriculum strategy can improve mod…
SecondOpinion: Anatomy-Aware Gated Reasoning for Efficient Medical Image Analysis
Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）；Department of Computer Science and Engineering, Independent University, Bangladesh（孟加拉国独立大学计算机科学与工程系）；Graduate School of Engineering, University of Hyogo（兵库大学工程研究生院）
Curriculum-Based Augmented Fourier Domain Adaptation for Robust Medical Image Segmentation
The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）；National University of Singapore（新加坡国立大学）；NUSRI Suzhou（新加坡国立大学苏州研究院）；University College London（伦敦大学学院）；Wellcome/EPSRC Centre for Interventional and Surgical Sciences（维康/英国工程与自然科学研究委员会介入与外科科学中心）
University of Zurich（苏黎世大学）；Zurich Institute of Forensic Medicine（苏黎世法医研究所）；University of St. Thomas（圣托马斯大学）
Imaging techniques is widely used for medical diagnostics. This leads in some cases to a real bottleneck when there is a lack of medical practitioners and the images have to be manually processed. In such a situation there is a need to reduce the amount of manual work by automating part of the analysis. In this article, we investigate the potential of a machine learning algorithm for medical image processing by computing a topological invariant classifier. First, we select retrospectively from our database of postmortem computed tomography images of rib fractures. The images are prepared by ap…
The University of Tokyo（东京大学）；The University of Tokyo Hospital（东京大学医院）
A computer-aided detection (CAD) system based on machine learning is expected to assist radiologists in making a diagnosis. It is desirable to build CAD systems for the various types of diseases accumulating daily in a hospital. An obstacle in developing a CAD system for a disease is that the number of medical images is typically too small to improve the performance of the machine learning model. In this paper, we aim to explore ways to address this problem through a sim2real transfer approach in medical image fields. To build a platform to evaluate the performance of sim2real transfer methods…
MedKCO: Medical Vision-Language Pretraining via Knowledge-Driven Cognitive Orchestration
School of Computer Science and Engineering, Southeast University, China（东南大学计算机科学与工程学院）；Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Ministry of Education, China（教育部新一代人工智能技术及其交叉应用重点实验室）；School of Computer Science and Engineering, Nanjing University of Science and Technology, China（南京理工大学计算机科学与工程学院）
Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医疗中心）
Applying curriculum learning requires both a range of difficulty in data and a method for determining the difficulty of examples. In many tasks, however, satisfying these requirements can be a formidable challenge. In this paper, we contend that histopathology image classification is a compelling use case for curriculum learning. Based on the nature of histopathology images, a range of difficulty inherently exists among examples, and, since medical datasets are often labeled by multiple annotators, annotator agreement can be used as a natural proxy for the difficulty of a given example. Hence,…
PAII Inc.（PAII 公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Rochester（罗切斯特大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Visual cues of enforcing bilaterally symmetric anatomies as normal findings are widely used in clinical practice to disambiguate subtle abnormalities from medical images. So far, inadequate research attention has been received on effectively emulating this practice in CAD methods. In this work, we exploit semantic anatomical symmetry or asymmetry analysis in a complex CAD scenario, i.e., anterior pelvic fracture detection in trauma PXRs, where semantically pathological (refer to as fracture) and non-pathological (e.g., pose) asymmetries both occur. Visually subtle yet pathologically critical f…
Lung nodules suffer large variation in size and appearance in CT images. Nodules less than 10mm can easily lose information after down-sampling in convolutional neural networks, which results in low sensitivity. In this paper, a combination of 3D image and feature pyramid is exploited to integrate lower-level texture features with high-level semantic features, thus leading to a higher recall. However, 3D operations are time and memory consuming, which aggravates the situation with the explosive growth of medical images. To tackle this problem, we propose a general curriculum training strategy …
University of São Paulo（圣保罗大学）；University of Rondônia（朗多尼亚大学）
Segmentation of medical images is a critical issue: several process of analysis and classification rely on this segmentation. With the growing number of people presenting back pain and problems related to it, the automatic or semi-automatic segmentation of fractured vertebral bodies became a challenging task. In general, those fractures present several regions with non-homogeneous intensities and the dark regions are quite similar to the structures nearby. Aimed at overriding this challenge, in this paper we present a semi-automatic segmentation method, called Balanced Growth (BGrowth). The ex…
OrthoInsight: Rib Fracture Diagnosis and Report Generation Based on Multi-Modal Large Models
Organizational Management Department, School of Management, Xi’an Jiaotong University（管理学院组织管理部，西安交通大学）；West China Longquan Hospital, Sichuan University（四川大学西部临床医学院）；School of Electronic Science and Engineering, Xi’an Jiaotong University（西安交通大学电子科学与工程学院）；Systems Engineering Institute, Xi’an Jiaotong University（西安交通大学系统工程研究院）；Institute of Medical Artificial Intelligence, the Second Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第二附属医院医学人工智能研究所）；School of Human Settlements and Civil Engineering, Xi’an Jiaotong University（西安交通大学人居环境与土木工程学院）；School of Life Science and Technology, Xi’an Jiaotong University（西安交通大学生命科学与技术学院）
Object detection methods are widely adopted for computer-aided diagnosis using medical images. Anomalous findings are usually treated as objects that are described by bounding boxes. Yet, many pathological findings, e.g., bone fractures, cannot be clearly defined by bounding boxes, owing to considerable instance, shape and boundary ambiguities. This makes bounding box annotations, and their associated losses, highly ill-suited. In this work, we propose a new bone fracture detection method for X-ray images, based on a labor effective and flexible annotation scheme suitable for abnormal findings…
King's College London（伦敦国王学院）；Guy's and St Thomas' Hospital NHS Foundation Trust（盖伊和圣托马斯国民保健服务基金会信托医院）；Imperial College London（帝国理工学院）
Good quality of medical images is a prerequisite for the success of subsequent image analysis pipelines. Quality assessment of medical images is therefore an essential activity and for large population studies such as the UK Biobank (UKBB), manual identification of artefacts such as those caused by unanticipated motion is tedious and time-consuming. Therefore, there is an urgent need for automatic image quality assessment techniques. In this paper, we propose a method to automatically detect the presence of motion-related artefacts in cardiac magnetic resonance (CMR) cine images. We compare tw…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Hospital（约翰斯·霍普金斯医院）；Technical University of Munich（慕尼黑工业大学）
We present a novel methodology to detect imperfect bilateral symmetry in CT of human anatomy. In this paper, the structurally symmetric nature of the pelvic bone is explored and is used to provide interventional image augmentation for treatment of unilateral fractures in patients with traumatic injuries. The mathematical basis of our solution is on the incorporation of attributes and characteristics that satisfy the properties of intrinsic and extrinsic symmetry and are robust to outliers. In the first step, feature points that satisfy intrinsic symmetry are automatically detected in the Möbiu…
University of Pittsburgh（匹兹堡大学）；Magee-Womens Hospital, University of Pittsburgh Medical Center（匹兹堡大学医学中心玛吉妇女医院）
Mammography is used as a standard screening procedure for the potential patients of breast cancer. Over the past decade, it has been shown that deep learning techniques have succeeded in reaching near-human performance in a number of tasks, and its application in mammography is one of the topics that medical researchers most concentrate on. In this work, we propose an end-to-end Curriculum Learning (CL) strategy in task space for classifying the three categories of Full-Field Digital Mammography (FFDM), namely Malignant, Negative, and False recall. Specifically, our method treats this three-cl…
Gazi University（加齐大学）；Kafkas University（卡夫卡斯大学）；Huawei Turkey R&D Center（华为土耳其研发中心）
Hospitals, especially their emergency services, receive a high number of wrist fracture cases. For correct diagnosis and proper treatment of these, images obtained from various medical equipment must be viewed by physicians, along with the patients medical records and physical examination. The aim of this study is to perform fracture detection by use of deep learning on wrist Xray images to support physicians in the diagnosis of these fractures, particularly in the emergency services. Using SABL, RegNet, RetinaNet, PAA, Libra R_CNN, FSAF, Faster R_CNN, Dynamic R_CNN and DCN deep learning based…
The University of Adelaide（阿德莱德大学）；Queensland University of Technology（昆士兰科技大学）
We developed an automated deep learning system to detect hip fractures from frontal pelvic x-rays, an important and common radiological task. Our system was trained on a decade of clinical x-rays (~53,000 studies) and can be applied to clinical data, automatically excluding inappropriate and technically unsatisfactory studies. We demonstrate diagnostic performance equivalent to a human radiologist and an area under the ROC curve of 0.994. Translated to clinical practice, such a system has the potential to increase the efficiency of diagnosis, reduce the need for expensive additional testing, e…
Australian Institute for Machine Learning（澳大利亚机器学习研究所）；University of Adelaide（阿德莱德大学）；Peter Faber Business School, Australian Catholic University（澳大利亚天主教大学彼得·法伯商学院）；Australian Catholic University（澳大利亚天主教大学）；Amsterdam UMC（阿姆斯特丹大学医学中心）；Flinders Medical Centre（弗林德斯医学中心）；Flinders University（弗林德斯大学）；Groningen University Medical Centre（格罗宁根大学医学中心）
Convolutional neural networks (CNNs) have gained significant popularity in orthopedic imaging in recent years due to their ability to solve fracture classification problems. A common criticism of CNNs is their opaque learning and reasoning process, making it difficult to trust machine diagnosis and the subsequent adoption of such algorithms in clinical setting. This is especially true when the CNN is trained with limited amount of medical data, which is a common issue as curating sufficiently large amount of annotated medical imaging data is a long and costly process. While interest has been d…
Progressive Growing of Patch Size: Curriculum Learning for Accelerated and Improved Medical Image Segmentation
School of Computation, Information and Technology, Technical University Munich（技术大学慕尼黑计算、信息与技术学院）；Department of Radiation Oncology, TUM School of Medicine, TUM University Hospital rechts der Isar, Technical University of Munich（肿瘤放射治疗科，TUM医学院，TUM大学医院rechts der Isar，技术大学慕尼黑）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（生物医学影像中的机器学习研究所，海德堡慕尼黑）；Institute of Radiation Medicine, Helmholtz Center Munich（放射医学研究所，海德堡慕尼黑）；Munich Center of Machine Learning (MCML)（慕尼黑机器学习中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（生物医学工程与成像科学学院，伦敦国王学院）；Departament de Matemàtiques i Informàtica, Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM), Universitat de Barcelona（数学与计算机科学系，巴塞罗那人工智能在医学实验室（BCN-AIM），巴塞罗那大学）
Exploring the Efficacy of Partial Denoising Using Bit Plane Slicing for Enhanced Fracture Identification: A Comparative Study of Deep Learning-Based Approaches and Handcrafted Feature Extraction Techniques
Harvard Medical School（哈佛医学院）；Harvard University（哈佛大学）；The University of Adelaide（阿德莱德大学）
Medical students and radiology trainees typically view thousands of images in order to "train their eye" to detect the subtle visual patterns necessary for diagnosis. Nevertheless, infrastructural and legal constraints often make it difficult to access and quickly query an abundance of images with a user-specified feature set. In this paper, we use a conditional generative adversarial network (GAN) to synthesize $1024\times1024$ pixel pelvic radiographs that can be queried with conditioning on fracture status. We demonstrate that the conditional GAN learns features that distinguish fractures f…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）；Kharkevich Institute for Information Transmission Problems（哈尔克维奇信息传输问题研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Research and Practical Clinical Center of Diagnostics and Telemedicine Technologies（诊断与远程医疗技术研究与实用临床中心）
Vertebral body compression fractures are reliable early signs of osteoporosis. Though these fractures are visible on Computed Tomography (CT) images, they are frequently missed by radiologists in clinical settings. Prior research on automatic methods of vertebral fracture classification proves its reliable quality; however, existing methods provide hard-to-interpret outputs and sometimes fail to process cases with severe abnormalities such as highly pathological vertebrae or scoliosis. We propose a new two-step algorithm to localize the vertebral column in 3D CT images and then to simultaneous…
We introduce CASED, a novel curriculum sampling algorithm that facilitates the optimization of deep learning segmentation or detection models on data sets with extreme class imbalance. We evaluate the CASED learning framework on the task of lung nodule detection in chest CT. In contrast to two-stage solutions, wherein nodule candidates are first proposed by a segmentation model and refined by a second detection stage, CASED improves the training of deep nodule segmentation models (e.g. UNet) to the point where state of the art results are achieved using only a trivial detection stage. CASED im…
St. Michael’s Hospital（圣迈克尔医院）；Unity Health Toronto（多伦多统一健康中心）；University of Toronto（多伦多大学）；Stanford University（斯坦福大学）
Fractures of the cervical spine are a medical emergency and may lead to permanent paralysis and even death. Accurate diagnosis in patients with suspected fractures by computed tomography (CT) is critical to patient management. In this paper, we propose a deep convolutional neural network (DCNN) with a bidirectional long-short term memory (BLSTM) layer for the automated detection of cervical spine fractures in CT axial images. We used an annotated dataset of 3,666 CT scans (729 positive and 2,937 negative cases) to train and validate the model. The validation results show a classification accur…
Pediatric Wrist Fracture Detection Using Feature Context Excitation Modules in X-ray Images
Graduate Institute of Networking and Multimedia（网络与多媒体研究生院）；National Taiwan University（台湾大学）；Department of Electrical and Computer Engineering（电子工程系）；Tamkang University（Tamkang大学）；College of Energy and Mechanical Engineering（能源与机械工程学院）；Shanghai University of Electric Power（上海电力大学）
Airdoc（鹰瞳科技）；Harbin Engineering University（哈尔滨工程大学）；IIAI（国际人工智能研究院）；Monash University（莫纳什大学）；The University of Sydney（悉尼大学）
Deep neural networks are known to be data-driven and label noise can have a marked impact on model performance. Recent studies have shown great robustness to classic image recognition even under a high noisy rate. In medical applications, learning from datasets with label noise is more challenging since medical imaging datasets tend to have asymmetric (class-dependent) noise and suffer from high observer variability. In this paper, we systematically discuss and define the two common types of label noise in medical images - disagreement label noise from inconsistency expert opinions and single-…
Deep Omni-supervised Learning for Rib Fracture Detection from Chest Radiology Images
Imsight AI Research Lab（Imsight AI 研究实验室）；The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳-香港协同创新研究院）；The Chinese University of Hong Kong（香港中文大学）
IBM Research – Almaden Research Center（IBM研究院阿尔马登研究中心）
Deep learning has shown promising results in medical image analysis, however, the lack of very large annotated datasets confines its full potential. Although transfer learning with ImageNet pre-trained classification models can alleviate the problem, constrained image sizes and model complexities can lead to unnecessary increase in computational cost and decrease in performance. As many common morphological features are usually shared by different classification tasks of an organ, it is greatly beneficial if we can extract such features to improve classification with limited samples. Therefore…
