Exploring Instance-Level Uncertainty for Medical Detection
University of California, Los Angeles（加州大学洛杉矶分校）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Jiawei Yang, Yuan Liang, Yao Zhang, Weinan Song, Kun Wang, Lei He
The ability of deep learning to predict with uncertainty is recognized as key for its adoption in clinical routines. Moreover, performance gain has been enabled by modelling uncertainty according to empirical evidence. While previous work has widely discussed the uncertainty estimation in segmentation and classification tasks, its application on bounding-box-based detection has been limited, mainly due to the challenge of bounding box aligning. In this work, we explore to augment a 2.5D detection CNN with two different bounding-box-level (or instance-level) uncertainty estimates, i.e., predictive variance and Monte Carlo (MC) sample variance. Experiments are conducted for lung nodule detection on LUNA16 dataset, a task where significant semantic ambiguities can exist between nodules and non-nodules. Results show that our method improves the evaluating score from 84.57% to 88.86% by utilizing a combination of both types of variances. Moreover, we show the generated uncertainty enables superior operating points compared to using the probability threshold only, and can further boost the performance to 89.52%. Example nodule detections are visualized to further illustrate the advantages of our method.
Eindhoven University of Technology（埃因霍温理工大学）；Erasmus Medical Center（伊拉斯谟医学中心）；University Medical Center Utrecht（乌得勒支大学医学中心）
Supervised learning is ubiquitous in medical image analysis. In this paper we consider the problem of meta-learning -- predicting which methods will perform well in an unseen classification problem, given previous experience with other classification problems. We investigate the first step of such an approach: how to quantify the similarity of different classification problems. We characterize datasets sampled from six classification problems by performance ranks of simple classifiers, and define the similarity by the inverse of Euclidean distance in this meta-feature space. We visualize the s…
Eindhoven University of Technology（埃因霍温理工大学）；Erasmus Medical Center（伊拉斯谟医学中心）；University of Copenhagen（哥本哈根大学）；University Medical Center Utrecht（乌得勒支大学医学中心）
Machine learning (ML) algorithms have made a tremendous impact in the field of medical imaging. While medical imaging datasets have been growing in size, a challenge for supervised ML algorithms that is frequently mentioned is the lack of annotated data. As a result, various methods which can learn with less/other types of supervision, have been proposed. We review semi-supervised, multiple instance, and transfer learning in medical imaging, both in diagnosis/detection or segmentation tasks. We also discuss connections between these learning scenarios, and opportunities for future research.
Institute of Computer Engineering, Technische Universität Wien (TU Wien)（维也纳工业大学（TU Wien）计算机工程研究所）；Division of Engineering, New York University Abu Dhabi（纽约大学阿布扎比分校工程系）
Reliable classification and detection of certain medical conditions, in images, with state-of-the-art semantic segmentation networks, require vast amounts of pixel-wise annotation. However, the public availability of such datasets is minimal. Therefore, semantic segmentation with image-level labels presents a promising alternative to this problem. Nevertheless, very few works have focused on evaluating this technique and its applicability to the medical sector. Due to their complexity and the small number of training examples in medical datasets, classifier-based weakly supervised networks lik…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shandong University of Science and Technology（山东科技大学）
Efficient medical image segmentation aims to provide accurate pixel-wise predictions for medical images with a lightweight implementation framework. However, lightweight frameworks generally fail to achieve superior performance and suffer from poor generalizable ability on cross-domain tasks. In this paper, we explore the generalizable knowledge distillation for the efficient segmentation of cross-domain medical images. Considering the domain gaps between different medical datasets, we propose the Model-Specific Alignment Networks (MSAN) to obtain the domain-invariant representations. Meanwhil…
Institute of High Performance Computing, A*STAR（A*STAR高性能计算研究所）；Artificial Intelligence Initiative, A*STAR（A*STAR人工智能倡议组织）；Singapore Eye Research Institute（新加坡眼科研究所）
Deep learning for medical image classification faces three major challenges: 1) the number of annotated medical images for training are usually small; 2) regions of interest (ROIs) are relatively small with unclear boundaries in the whole medical images, and may appear in arbitrary positions across the x,y (and also z in 3D images) dimensions. However often only labels of the whole images are annotated, and localized ROIs are unavailable; and 3) ROIs in medical images often appear in varying sizes (scales). We approach these three challenges with a Multi-Instance Multi-Scale (MIMS) CNN: 1) We …
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）
Despite the great progress achieved in unsupervised feature embedding, existing contrastive learning methods typically pursue view-invariant representations through attracting positive sample pairs and repelling negative sample pairs in the embedding space, while neglecting to systematically explore instance relations. In this paper, we explore instance relations including intra-instance multi-view relation and inter-instance interpolation relation for unsupervised feature embedding. Specifically, we embed intra-instance multi-view relation by aligning the distribution of the distance between …
Probabilistic smooth attention for deep multiple instance learning in medical imaging
Department of Computer Science and AI, University of Granada（计算机科学与人工智能系，格拉纳达大学）；Department of Statistics and Operations Research, University of Granada（统计与运筹学系，格拉纳达大学）；Research Centre for Information and Communication Technologies (CITIC), University of Granada（信息与通信技术研究中心（CITIC），格拉纳达大学）；Department of Electrical and Computer Engineering, Northwestern University（电气与计算机工程系，西北大学）；Center for Computational Imaging and Signal Analytics in Medicine, Northwestern University（医学计算成像与信号分析中心，西北大学）
Circle Representation for Medical Instance Object Segmentation
Department of Electrical Engineering and Computer Science, Vanderbilt University（电气工程与计算机科学系，范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；Department of Pathology, Vanderbilt University Medical Center（病理学系，范德堡大学医学中心）；Department of Pediatrics, Vanderbilt University Medical Center（儿科学系，范德堡大学医学中心）
Beyond Instance-Level Self-Supervision in 3D Multi-Modal Medical Imaging
Fudan University, China（复旦大学）；University of Queensland, Australia（昆士兰大学）；Shanghai Academy of AI for Science, China（上海人工智能科学研究院）；Huashan Hospital, National Center for Neurological Disorders, Fudan University, China（华山医院，国家神经系统疾病中心，复旦大学）；Bioinformatics Institute (BII), Agency for Science, Technology and Research (A*STAR), Singapore（生物信息研究所（BII），科技研究局（A*STAR），新加坡）
Deep learning is the current bet for image classification. Its greed for huge amounts of annotated data limits its usage in medical imaging context. In this scenario transfer learning appears as a prominent solution. In this report we aim to clarify how transfer learning schemes may influence classification results. We are particularly focused in the automated melanoma screening problem, a case of medical imaging in which transfer learning is still not widely used. We explored transfer with and without fine-tuning, sequential transfers and usage of pre-trained models in general and specific da…
Key Patches Are All You Need: A Multiple Instance Learning Framework For Robust Medical Diagnosis
Institute for Systems and Robotics（系统与机器人研究所）；Instituto Superior Técnico（里斯本高等理工学院）；Institute of Computing（计算研究所）；University of Campinas（坎皮纳斯大学）；Lisbon ELLIS Unit（里斯本ELLIS单元）
DExTeR: Weakly Semi-Supervised Object Detection with Class and Instance Experts for Medical Imaging
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France（斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、法国）；University Hospital of Strasbourg, France（斯特拉斯堡大学医院、法国）
Why does my medical AI look at pictures of birds? Exploring the efficacy of transfer learning across domain boundaries
University Duisburg-Essen（杜伊斯堡-埃森大学）；University Medicine Essen (AöR)（埃森大学医学中心（公法机构））；Ruhr University Bochum（波鸿鲁尔大学）；Monash University（莫纳什大学）；Cancer Research Center Cologne Essen (CCCE)（科隆埃森癌症研究中心）；TU Dortmund University（多特蒙德工业大学）
Semi-supervised segmentation remains challenging in medical imaging since the amount of annotated medical data is often scarce and there are many blurred pixels near the adhesive edges or in the low-contrast regions. To address the issues, we advocate to firstly constrain the consistency of pixels with and without strong perturbations to apply a sufficient smoothness constraint and further encourage the class-level separation to exploit the low-entropy regularization for the model training. Particularly, in this paper, we propose the SS-Net for semi-supervised medical image segmentation tasks,…
Technische Universität Darmstadt（达姆施塔特工业大学）；Fraunhofer IGD（弗劳恩霍夫计算机图形学研究所）
Failure cases of black-box deep learning, e.g. adversarial examples, might have severe consequences in healthcare. Yet such failures are mostly studied in the context of real-world images with calibrated attacks. To demystify the adversarial examples, rigorous studies need to be designed. Unfortunately, complexity of the medical images hinders such study design directly from the medical images. We hypothesize that adversarial examples might result from the incorrect mapping of image space to the low dimensional generation manifold by deep networks. To test the hypothesis, we simplify a complex…
Arizona State University（亚利桑那州立大学）；Mayo Clinic（梅奥诊所）
Recently, self-supervised instance discrimination methods have achieved significant success in learning visual representations from unlabeled photographic images. However, given the marked differences between photographic and medical images, the efficacy of instance-based objectives, focusing on learning the most discriminative global features in the image (i.e., wheels in bicycle), remains unknown in medical imaging. Our preliminary analysis showed that high global similarity of medical images in terms of anatomy hampers instance discrimination methods for capturing a set of distinct features…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins Hospital（约翰斯·霍普金斯医院）；Technical University of Munich（慕尼黑工业大学）
We present a novel methodology to detect imperfect bilateral symmetry in CT of human anatomy. In this paper, the structurally symmetric nature of the pelvic bone is explored and is used to provide interventional image augmentation for treatment of unilateral fractures in patients with traumatic injuries. The mathematical basis of our solution is on the incorporation of attributes and characteristics that satisfy the properties of intrinsic and extrinsic symmetry and are robust to outliers. In the first step, feature points that satisfy intrinsic symmetry are automatically detected in the Möbiu…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Circle representation has recently been introduced as a medical imaging optimized representation for more effective instance object detection on ball-shaped medical objects. With its superior performance on instance detection, it is appealing to extend the circle representation to instance medical object segmentation. In this work, we propose CircleSnake, a simple end-to-end circle contour deformation-based segmentation method for ball-shaped medical objects. Compared to the prevalent DeepSnake method, our contribution is three-fold: (1) We replace the complicated bounding box to octagon conto…
The Hong Kong University of Science and Technology（香港科技大学）；Shanghai Jiao Tong University（上海交通大学）
Multiple instance learning (MIL) has emerged as a popular method for classifying histopathology whole slide images (WSIs). However, existing approaches typically rely on pre-trained models from large natural image datasets, such as ImageNet, to generate instance features, which can be sub-optimal due to the significant differences between natural images and histopathology images that lead to a domain shift. In this paper, we present a novel, simple yet effective method for learning domain-specific knowledge transformation from pre-trained models to histopathology images. Our approach entails u…
Artificial Intelligence Research Institute (AIRI)（人工智能研究院（AIRI））；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
When applying a Deep Learning model to medical images, it is crucial to estimate the model uncertainty. Voxel-wise uncertainty is a useful visual marker for human experts and could be used to improve the model's voxel-wise output, such as segmentation. Moreover, uncertainty provides a solid foundation for out-of-distribution (OOD) detection, improving the model performance on the image-wise level. However, one of the frequent tasks in medical imaging is the segmentation of distinct, local structures such as tumors or lesions. Here, the structure-wise uncertainty allows more precise operations …
Co-Seg++: Mutual Prompt-Guided Collaborative Learning for Versatile Medical Segmentation
School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波校区计算机科学学院）；University of Nottingham, UK（诺丁汉大学（英国））；Graduate School of Information, Production and Systems, Waseda University, Japan（早稻田大学信息、生产与系统研究生院）；Yale University（耶鲁大学）
Multi-Scale Global-Instance Prompt Tuning for Continual Test-time Adaptation in Medical Image Segmentation
School of Computer Science, University of Nottingham, UK（诺丁汉大学计算机科学学院）；School of Artificial Intelligence, Shenzhen University, China（深圳大学人工智能学院）；Department of Information Engineering and Computer Science, University of Trento, Italy（特伦托大学信息工程与计算机科学系）；School of Computer Science and Information Engineering, Hefei University of Technology, China（合肥工业大学计算机科学与信息工程学院）
Center for Translational Research in Neuroimaging and Data Science (TReNDS)（神经影像与数据科学转化研究中心（TReNDS））
Deep learning has become a prominent computational modeling tool in the areas of computer vision and image processing in recent years. This research comprehensively analyzes the different deep-learning methods used for image-to-image translation and reconstruction in the natural and medical imaging domains. We examine the famous deep learning frameworks, such as convolutional neural networks and generative adversarial networks, and their variants, delving into the fundamental principles and difficulties of each. In the field of natural computer vision, we investigate the development and extens…
Exploring scalable medical image encoders beyond text supervision
Health Futures, Microsoft Research（微软研究院健康未来部门）；Microsoft Health and Life Sciences（微软健康与生命科学部门）；Department of Radiology, University of Cambridge（剑桥大学放射学系）；Cambridge University Hospitals NHS Foundation Trust（剑桥大学医院NHS基金会信托）；Microsoft Azure AI（微软Azure人工智能部门）
University of Science and Technology of China（中国科学技术大学）
Multi-Instance Learning(MIL) aims to learn the mapping between a bag of instances and the bag-level label. Therefore, the relationships among instances are very important for learning the mapping. In this paper, we propose an MIL algorithm based on a graph built by structural relationship among instances within a bag. Then, Graph Convolutional Network(GCN) and the graph-attention mechanism are used to learn bag-embedding. In the task of medical image classification, our GCN-based MIL algorithm makes full use of the structural relationships among patches(instances) in an original image space do…
Horizon Robotics（地平线机器人）；Huazhong University of Science & Technology（华中科技大学）
Labeling objects with pixel-wise segmentation requires a huge amount of human labor compared to bounding boxes. Most existing methods for weakly supervised instance segmentation focus on designing heuristic losses with priors from bounding boxes. While, we find that box-supervised methods can produce some fine segmentation masks and we wonder whether the detectors could learn from these fine masks while ignoring low-quality masks. To answer this question, we present BoxTeacher, an efficient and end-to-end training framework for high-performance weakly supervised instance segmentation, which le…
Due to the lack of expertise for medical image annotation, the investigation of label-efficient methodology for medical image segmentation becomes a heated topic. Recent progresses focus on the efficient utilization of weak annotations together with few strongly-annotated labels so as to achieve comparable segmentation performance in many unprofessional scenarios. However, these approaches only concentrate on the supervision inconsistency between strongly- and weakly-annotated instances but ignore the instance inconsistency inside the weakly-annotated instances, which inevitably leads to perfo…
Tecnológico de Monterrey（蒙特雷科技大学）；University of Oxford（牛津大学）
Image-based tracking of laparoscopic instruments plays a fundamental role in computer and robotic-assisted surgeries by aiding surgeons and increasing patient safety. Computer vision contests, such as the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge, seek to encourage the development of robust models for such purposes, providing large, diverse, and annotated datasets. To date, most of the existing models for instance segmentation of medical instruments were based on two-stage detectors, which provide robust results but are nowhere near to the real-time (5 frames-per-second (fp…
MFM-DA: Instance-Aware Adaptor and Hierarchical Alignment for Efficient Domain Adaptation in Medical Foundation Models
Lanzhou University（兰州大学）；Shanghai Jiaotong University（上海交通大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Supervised training of deep learning models requires large labeled datasets. There is a growing interest in obtaining such datasets for medical image analysis applications. However, the impact of label noise has not received sufficient attention. Recent studies have shown that label noise can significantly impact the performance of deep learning models in many machine learning and computer vision applications. This is especially concerning for medical applications, where datasets are typically small, labeling requires domain expertise and suffers from high inter- and intra-observer variability…
In this paper, we tackle the domain adaptive object detection problem, where the main challenge lies in significant domain gaps between source and target domains. Previous work seeks to plainly align image-level and instance-level shifts to eventually minimize the domain discrepancy. However, they still overlook to match crucial image regions and important instances across domains, which will strongly affect domain shift mitigation. In this work, we propose a simple but effective categorical regularization framework for alleviating this issue. It can be applied as a plug-and-play component on …
ProMIL: Probabilistic Multiple Instance Learning for Medical Imaging
Faculty of Mathematics and Computer Science, Jagiellonian University（雅盖隆大学数学与计算机科学学院）；Faculty of Applied Computer Science, University of Information Technology and Management in Rzeszow（热舒夫信息技术与管理大学应用计算机科学学院）；Medical University of Gdansk（格但斯克医科大学）；UES Ltd.（UES有限公司）
