Deep neural network-based medical image classifications often use "hard" labels for training, where the probability of the correct category is 1 and those of others are 0. However, these hard targets can drive the networks over-confident about their predictions and prone to overfit the training data, affecting model generalization and adaption. Studies have shown that label smoothing and softening can improve classification performance. Nevertheless, existing approaches are either non-data-driven or limited in applicability. In this paper, we present the Class-Correlation Learning Network (CCL-Net) to learn interclass visual correlations from given training data, and produce soft labels to help with classification tasks. Instead of letting the network directly learn the desired correlations, we propose to learn them implicitly via distance metric learning of class-specific embeddings with a lightweight plugin CCL block. An intuitive loss based on a geometrical explanation of correlation is designed for bolstering learning of the interclass correlations. We further present end-to-end training of the proposed CCL block as a plugin head together with the classification backbone while generating soft labels on the fly. Our experimental results on the International Skin Imaging Collaboration 2018 dataset demonstrate effective learning of the interclass correlations from training data, as well as consistent improvements in performance upon several widely used modern network structures with the CCL block.
In standard classification, we typically treat class categories as independent of one-another. In many problems, however, we would be neglecting the natural relations that exist between categories, which are often dictated by an underlying biological or physical process. In this work, we propose novel formulations of the classification problem, based on a realization that the assumption of class-independence is a limiting factor that leads to the requirement of more training data. First, we propose manual ways to reduce our data needs by reintroducing knowledge about problem-specific interclas…
Macquarie University（麦考瑞大学）；University of Technology Sydney（悉尼科技大学）；The University of Sydney（悉尼大学）
Automatic tumor segmentation is a crucial step in medical image analysis for computer-aided diagnosis. Although the existing methods based on convolutional neural networks (CNNs) have achieved the state-of-the-art performance, many challenges still remain in medical tumor segmentation. This is because regular CNNs can only exploit translation invariance, ignoring further inherent symmetries existing in medical images such as rotations and reflections. To mitigate this shortcoming, we propose a novel group equivariant segmentation framework by encoding those inherent symmetries for learning mor…
The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a …
Exploiting Scale-Variant Attention for Segmenting Small Medical Objects
Centre for Robotics and Automation, City University of Hong Kong（香港城市大学机器人与自动化中心）；Department of Electronic Engineering, The Chinese University of Hong Kong（香港中文大学电子工程系）
The University of Texas Arlington（德克萨斯大学阿灵顿分校）；RIKEN（理化学研究所）；University of Tokyo（东京大学）；National Cancer Center Research Institute（国立癌症中心研究所）；National Library of Medicine - National Institutes of Health（美国国立卫生研究院国家医学图书馆）；Temple University（天普大学）
Medical visual question answering (VQA) aims to answer clinically relevant questions regarding input medical images. This technique has the potential to improve the efficiency of medical professionals while relieving the burden on the public health system, particularly in resource-poor countries. Existing medical VQA methods tend to encode medical images and learn the correspondence between visual features and questions without exploiting the spatial, semantic, or medical knowledge behind them. This is partially because of the small size of the current medical VQA dataset, which often includes…
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Beihang University（北京航空航天大学）；Microsoft Research（微软研究院）
3D medical image registration is of great clinical importance. However, supervised learning methods require a large amount of accurately annotated corresponding control points (or morphing), which are very difficult to obtain. Unsupervised learning methods ease the burden of manual annotation by exploiting unlabeled data without supervision. In this paper, we propose a new unsupervised learning method using convolutional neural networks under an end-to-end framework, Volume Tweening Network (VTN), for 3D medical image registration. We propose three innovative technical components: (1) An end-t…
IRIT UMR CNRS 5505（法国国家科学研究中心IRIT联合研究单元（UMR 5505））；University of Toulouse（图卢兹大学）
The interest of compressive sampling in ultrasound imaging has been recently extensively evaluated by several research teams. Following the different application setups, it has been shown that the RF data may be reconstructed from a small number of measurements and/or using a reduced number of ultrasound pulse emissions. Nevertheless, RF image spatial resolution, contrast and signal to noise ratio are affected by the limited bandwidth of the imaging transducer and the physical phenomenon related to US wave propagation. To overcome these limitations, several deconvolution-based image processing…
Tencent AI Lab（腾讯人工智能实验室）；Peking Union Medical College Hospital（北京协和医院）；Peking Union Medical College（北京协和医学院）；Chinese Academy of Medical Sciences（中国医学科学院）
Doctors often make diagonostic decisions based on patient's image scans, such as magnetic resonance imaging (MRI), and patient's electronic health records (EHR) such as age, gender, blood pressure and so on. Despite a lot of automatic methods have been proposed for either image or text analysis in computer vision or natural language research areas, much fewer studies have been developed for the fusion of medical image and EHR data for medical problems. Among existing early or intermediate fusion methods, concatenation of features from both modalities is still a mainstream. For a better exploit…
Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning
Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Southern University of Science and Technology（南方科技大学广东省类脑智能计算重点实验室）；Singapore Eye Research Institute（新加坡眼科研究所）
Self-Supervised Learning for Medical Image Data with Anatomy-Oriented Imaging Planes
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Jarvis Research Center, Tencent YouTu Lab（腾讯优图实验室贾维斯研究中心）
University of Rome Tor Vergata（罗马第二大学）；University of Cambridge（剑桥大学）；Athinoula A. Martinos Center for Biomedical Imaging（阿西诺拉·A·马蒂诺斯生物医学成像中心）
The lack of large labeled medical imaging datasets, along with significant inter-individual variability compared to clinically established disease classes, poses significant challenges in exploiting medical imaging information in a precision medicine paradigm, where in principle dense patient-specific data can be employed to formulate individual predictions and/or stratify patients into finer-grained groups which may follow more homogeneous trajectories and therefore empower clinical trials. In order to efficiently explore the effective degrees of freedom underlying variability in medical imag…
Less Could Be Better: Parameter-efficient Fine-tuning Advances Medical Vision Foundation Models
School of Informatics, Xiamen University（厦门大学信息学院）；Department of Biomedical Informatics, Harvard University（哈佛大学生物医学信息学系）；Department of Computer Science, The University of Hong Kong（香港大学计算机科学系）
本文实证评估 LoRA 对胸部 X 线摄影自监督基础模型的微调效果，证明其以不足 1% 参数在多数任务上超过全参数微调，并刷新数据高效学习基准。
Indian Institute of Technology Madras（印度理工学院马德拉斯分校）；Healthcare Technology Innovation Centre（医疗技术创新中心）；Indian Institute of Information Technology Design & Manufacturing Kancheepuram（金奈设计与制造印度理工学院）；Zasti（Zasti公司）
Recently, state-of-the-art results have been achieved in semantic segmentation using fully convolutional networks (FCNs). Most of these networks employ encoder-decoder style architecture similar to U-Net and are trained with images and the corresponding segmentation maps as a pixel-wise classification task. Such frameworks only exploit class information by using the ground truth segmentation maps. In this paper, we propose a multi-task learning framework with the main aim of exploiting structural and spatial information along with the class information. We modify the decoder part of the FCN to…
Arizona State University（亚利桑那州立大学）；Mayo Clinic（梅奥诊所）
This paper introduces a new concept called "transferable visual words" (TransVW), aiming to achieve annotation efficiency for deep learning in medical image analysis. Medical imaging--focusing on particular parts of the body for defined clinical purposes--generates images of great similarity in anatomy across patients and yields sophisticated anatomical patterns across images, which are associated with rich semantics about human anatomy and which are natural visual words. We show that these visual words can be automatically harvested according to anatomical consistency via self-discovery, and …
PAII Inc.（PAII 公司）；Johns Hopkins University（约翰斯·霍普金斯大学）；University of Rochester（罗切斯特大学）；Chang Gung Memorial Hospital（长庚纪念医院）；Ping An Technology（平安科技）
Visual cues of enforcing bilaterally symmetric anatomies as normal findings are widely used in clinical practice to disambiguate subtle abnormalities from medical images. So far, inadequate research attention has been received on effectively emulating this practice in CAD methods. In this work, we exploit semantic anatomical symmetry or asymmetry analysis in a complex CAD scenario, i.e., anterior pelvic fracture detection in trauma PXRs, where semantically pathological (refer to as fracture) and non-pathological (e.g., pose) asymmetries both occur. Visually subtle yet pathologically critical f…
WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration
School of Computer Science, University of Birmingham（伯明翰大学计算机学院）；Department of Computing and Mathematics, Manchester Metropolitan University（曼彻斯特城市大学计算与数学系）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学大数据系统计算技术国家工程实验室）
Witnessed the development of deep learning, increasing number of studies try to build computer aided diagnosis systems for 3D volumetric medical data. However, as the annotations of 3D medical data are difficult to acquire, the number of annotated 3D medical images is often not enough to well train the deep learning networks. The self-supervised learning deeply exploiting the information of raw data is one of the potential solutions to loose the requirement of training data. In this paper, we propose a self-supervised learning framework for the volumetric medical images. A novel proxy task, i.…
KU Leuven（鲁汶大学）；Vector Institute（向量研究所）；NVIDIA（英伟达）；University of Toronto（多伦多大学）；University of Leeds（利兹大学）
Labelling data is expensive and time consuming especially for domains such as medical imaging that contain volumetric imaging data and require expert knowledge. Exploiting a larger pool of labeled data available across multiple centers, such as in federated learning, has also seen limited success since current deep learning approaches do not generalize well to images acquired with scanners from different manufacturers. We aim to address these problems in a common, learning-based image simulation framework which we refer to as Federated Simulation. We introduce a physics-driven generative appro…
FUJIFILM Corporation（富士胶片公司）；AWS AI Labs（AWS人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Stanford University（斯坦福大学）
Learning visual representations of medical images (e.g., X-rays) is core to medical image understanding but its progress has been held back by the scarcity of human annotations. Existing work commonly relies on fine-tuning weights transferred from ImageNet pretraining, which is suboptimal due to drastically different image characteristics, or rule-based label extraction from the textual report data paired with medical images, which is inaccurate and hard to generalize. Meanwhile, several recent studies show exciting results from unsupervised contrastive learning from natural images, but we fin…
Wasserstein-Aligned Localisation for VLM-Based Distributional OOD Detection in Medical Imaging
Department of Computing, Imperial College London, UK（伦敦帝国理工学院计算机系）；Technical University Munich, DE（慕尼黑技术大学）；Munich Center for Machine Learning (MCML), DE（慕尼黑机器学习中心（MCML））
Although self-supervised learning enables us to bootstrap the training by exploiting unlabeled data, the generic self-supervised methods for natural images do not sufficiently incorporate the context. For medical images, a desirable method should be sensitive enough to detect deviation from normal-appearing tissue of each anatomical region; here, anatomy is the context. We introduce a novel approach with two levels of self-supervised representation learning objectives: one on the regional anatomical level and another on the patient-level. We use graph neural networks to incorporate the relatio…
The University of Hong Kong（香港大学）；ShanghaiTech University（上海科技大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；South China University of Technology（华南理工大学）
Well-annotated medical images are costly and sometimes even impossible to acquire, hindering landmark detection accuracy to some extent. Semi-supervised learning alleviates the reliance on large-scale annotated data by exploiting the unlabeled data to understand the population structure of anatomical landmarks. The global shape constraint is the inherent property of anatomical landmarks that provides valuable guidance for more consistent pseudo labelling of the unlabeled data, which is ignored in the previously semi-supervised methods. In this paper, we propose a model-agnostic shape-regulated…
Exploiting Test-Time Augmentation in Federated Learning for Brain Tumor MRI Classification
Institute of Exact and Technological Sciences, Federal University of Viçosa - UFV, Rio Paranaíba-MG, Brazil（精确与技术科学研究所，弗拉维亚联邦大学-UFV，里奥帕拉纳伊巴-MG，巴西）；Department of Computing, Federal University of São Carlos, São Carlos-SP, Brazil（计算系，萨o卡洛斯联邦大学，萨o卡洛斯-SP，巴西）
Performance Benchmarking and Optimisation of Clustering Algorithms for Local and Non-Local Similarity Measure in Medical Image Analysis
Department of Mathematics, Rhodes University, PO Box 94, Makhanda, 6140, South Africa（数学系，罗德斯大学）；Department of Statistics, Rhodes University, PO Box 94, Makhanda, 6140, South Africa（统计系，罗德斯大学）；National Institute for Theoretical and Computational Sciences (NITheCS), Stellenbosch 7600, South Africa（理论与计算科学国家研究所）
Chest X-Ray (CXR) images are commonly used for clinical screening and diagnosis. Automatically writing reports for these images can considerably lighten the workload of radiologists for summarizing descriptive findings and conclusive impressions. The complex structures between and within sections of the reports pose a great challenge to the automatic report generation. Specifically, the section Impression is a diagnostic summarization over the section Findings; and the appearance of normality dominates each section over that of abnormality. Existing studies rarely explore and consider this fun…
National University of Singapore（新加坡国立大学）；National University Hospital（国立大学医院）；Zhejiang University（浙江大学）
In this paper, we propose a novel semi-supervised learning (SSL) framework named BoostMIS that combines adaptive pseudo labeling and informative active annotation to unleash the potential of medical image SSL models: (1) BoostMIS can adaptively leverage the cluster assumption and consistency regularization of the unlabeled data according to the current learning status. This strategy can adaptively generate one-hot "hard" labels converted from task model predictions for better task model training. (2) For the unselected unlabeled images with low confidence, we introduce an Active learning (AL) …
Supervised learning method requires a large volume of annotated datasets. Collecting such datasets is time-consuming and expensive. Until now, very few annotated COVID-19 imaging datasets are available. Although self-supervised learning enables us to bootstrap the training by exploiting unlabeled data, the generic self-supervised methods for natural images do not sufficiently incorporate the context. For medical images, a desirable method should be sensitive enough to detect deviation from normal-appearing tissue of each anatomical region; here, anatomy is the context. We introduce a novel app…
Graz University of Technology（格拉茨技术大学）；University of Graz（格拉茨大学）
Diverse inverse problems in imaging can be cast as variational problems composed of a task-specific data fidelity term and a regularization term. In this paper, we propose a novel learnable general-purpose regularizer exploiting recent architectural design patterns from deep learning. We cast the learning problem as a discrete sampled optimal control problem, for which we derive the adjoint state equations and an optimality condition. By exploiting the variational structure of our approach, we perform a sensitivity analysis with respect to the learned parameters obtained from different trainin…
Hasso-Plattner Institute, Potsdam University（波茨坦大学哈索·普拉特纳研究所）；SAP AI Research（SAP人工智能研究院）
Self-supervised learning approaches leverage unlabeled samples to acquire generic knowledge about different concepts, hence allowing for annotation-efficient downstream task learning. In this paper, we propose a novel self-supervised method that leverages multiple imaging modalities. We introduce the multimodal puzzle task, which facilitates rich representation learning from multiple image modalities. The learned representations allow for subsequent fine-tuning on different downstream tasks. To achieve that, we learn a modality-agnostic feature embedding by confusing image modalities at the da…
Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）
Multimodal medical image fusion helps in combining contrasting features from two or more input imaging modalities to represent fused information in a single image. One of the pivotal clinical applications of medical image fusion is the merging of anatomical and functional modalities for fast diagnosis of malignant tissues. In this paper, we present a novel end-to-end unsupervised learning-based Convolutional Neural Network (CNN) for fusing the high and low frequency components of MRI-PET grayscale image pairs, publicly available at ADNI, by exploiting Structural Similarity Index (SSIM) as the …
Technical University of Munich（慕尼黑工业大学）；University of Zurich（苏黎世大学）；Southern University of Science and Technology（南方科技大学）；ETH Zurich（苏黎世联邦理工学院）；Macau University of Science and Technology（澳门科技大学）；Klinikum rechts der Isar（伊萨尔河右岸医院）
Radiomic representations can quantify properties of regions of interest in medical image data. Classically, they account for pre-defined statistics of shape, texture, and other low-level image features. Alternatively, deep learning-based representations are derived from supervised learning but require expensive annotations from experts and often suffer from overfitting and data imbalance issues. In this work, we address the challenge of learning representations of 3D medical images for an effective quantification under data imbalance. We propose a \emph{self-supervised} representation learning…
The Devil is in the Statistics: Mitigating and Exploiting Statistics Difference for Generalizable Semi-supervised Medical Image Segmentation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National Institute of Healthcare Data Science, Nanjing University（南京大学健康医疗大数据国家研究院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Data and Computer Science, Shandong Women’s University（山东女子学院数据与计算机科学学院）
SYNPRED: A Synergistic Approach to Multimodal Learning for Clinical Prediction
Medical University of Vienna（维也纳医科大学）；Comprehensive Center for Artificial Intelligence in Medicine, Medical University of Vienna（维也纳医科大学医学人工智能综合中心）；Christian Doppler Laboratory for Machine Learning Driven Precision Imaging（克里斯蒂安·多普勒机器学习驱动精准成像实验室）
The Chinese University of Hong Kong（香港中文大学）；Beihang University（北京航空航天大学）；The University of British Columbia（不列颠哥伦比亚大学）
Despite recent progress on semi-supervised federated learning (FL) for medical image diagnosis, the problem of imbalanced class distributions among unlabeled clients is still unsolved for real-world use. In this paper, we study a practical yet challenging problem of class imbalanced semi-supervised FL (imFed-Semi), which allows all clients to have only unlabeled data while the server just has a small amount of labeled data. This imFed-Semi problem is addressed by a novel dynamic bank learning scheme, which improves client training by exploiting class proportion information. This scheme consist…
The Chinese University of Hong Kong（香港中文大学）；Sangfor Technologies Inc.（深信服科技股份有限公司）
Deep learning has become the de-facto computational paradigm for various kinds of perception problems, including many privacy-sensitive applications such as online medical image analysis. No doubt to say, the data privacy of these deep learning systems is a serious concern. Different from previous research focusing on exploiting privacy leakage from deep learning models, in this paper, we present the first attack on the implementation of deep learning models. To be specific, we perform the attack on an FPGA-based convolutional neural network accelerator and we manage to recover the input image…
Medical University Vienna（维也纳医科大学）；University of Vienna（维也纳大学）；NVIDIA Corporation（英伟达公司）
Diagnosis and treatment guidance are aided by detecting relevant biomarkers in medical images. Although supervised deep learning can perform accurate segmentation of pathological areas, it is limited by requiring a-priori definitions of these regions, large-scale annotations, and a representative patient cohort in the training set. In contrast, anomaly detection is not limited to specific definitions of pathologies and allows for training on healthy samples without annotation. Anomalous regions can then serve as candidates for biomarker discovery. Knowledge about normal anatomical structure br…
