Recent trends in semi-supervised learning have significantly boosted the performance of 3D semi-supervised medical image segmentation. Compared with 2D images, 3D medical volumes involve information from different directions, e.g., transverse, sagittal, and coronal planes, so as to naturally provide complementary views. These complementary views and the intrinsic similarity among adjacent 3D slices inspire us to develop a novel annotation way and its corresponding semi-supervised model for effective segmentation. Specifically, we firstly propose the orthogonal annotation by only labeling two orthogonal slices in a labeled volume, which significantly relieves the burden of annotation. Then, we perform registration to obtain the initial pseudo labels for sparsely labeled volumes. Subsequently, by introducing unlabeled volumes, we propose a dual-network paradigm named Dense-Sparse Co-training (DeSCO) that exploits dense pseudo labels in early stage and sparse labels in later stage and meanwhile forces consistent output of two networks. Experimental results on three benchmark datasets validated our effectiveness in performance and efficiency in annotation. For example, with only 10 annotated slices, our method reaches a Dice up to 86.93% on KiTS19 dataset.
Imperial College London（帝国理工学院）；BioMedIA Group, Department of Computing（计算系BioMedIA小组）
Few-shot semantic segmentation (FSS) has great potential for medical imaging applications. Most of the existing FSS techniques require abundant annotated semantic classes for training. However, these methods may not be applicable for medical images due to the lack of annotations. To address this problem we make several contributions: (1) A novel self-supervised FSS framework for medical images in order to eliminate the requirement for annotations during training. Additionally, superpixel-based pseudo-labels are generated to provide supervision; (2) An adaptive local prototype pooling module pl…
Northwestern Polytechnical University（西北工业大学）；The University of Adelaide（阿德莱德大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）
Manual annotation of medical images is highly subjective, leading to inevitable and huge annotation biases. Deep learning models may surpass human performance on a variety of tasks, but they may also mimic or amplify these biases. Although we can have multiple annotators and fuse their annotations to reduce stochastic errors, we cannot use this strategy to handle the bias caused by annotators' preferences. In this paper, we highlight the issue of annotator-related biases on medical image segmentation tasks, and propose a Preference-involved Annotation Distribution Learning (PADL) framework to …
University of Michigan（密歇根大学）；Indian Institute of Technology - Bombay（印度理工学院孟买分校）
For medical image segmentation, most fully convolutional networks (FCNs) need strong supervision through a large sample of high-quality dense segmentations, which is taxing in terms of costs, time and logistics involved. This burden of annotation can be alleviated by exploiting weak inexpensive annotations such as bounding boxes and anatomical landmarks. However, it is very difficult to \textit{a priori} estimate the optimal balance between the number of annotations needed for each supervision type that leads to maximum performance with the least annotation cost. To optimize this cost-performa…
Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentation
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；School of Data Science, Fudan University（复旦大学大数据学院）；School of Engineering Medicine, Beihang University（北京航空航天大学医工交叉创新研究院）；Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与产业研究院）；School of Biological Science and Medical Engineering, Beihang University（北京航空航天大学生物与医学工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Beijing Anding Hospital, Capital Medical University（首都医科大学附属北京安定医院）；Key Laboratory for Biomechanics and Mechanobiology of Ministry of Education, Beihang University（北京航空航天大学生物力学与力学生物学教育部重点实验室）；Hefei Innovation Research Institute, Beihang University（北京航空航天大学合肥创新研究院）
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Carl Zeiss AG（卡尔蔡司公司）；University of Applied Sciences Berlin（柏林应用科学大学）
Pixel-wise segmentation is one of the most data and annotation hungry tasks in our field. Providing representative and accurate annotations is often mission-critical especially for challenging medical applications. In this paper, we propose a semi-weakly supervised segmentation algorithm to overcome this barrier. Our approach is based on a new formulation of deep supervision and student-teacher model and allows for easy integration of different supervision signals. In contrast to previous work, we show that care has to be taken how deep supervision is integrated in lower layers and we present …
Transformer-based Annotation Bias-aware Medical Image Segmentation
Northwestern Polytechnical University（西北工业大学）；The University of Adelaide（阿德莱德大学）；Australian Institute for Machine Learning（澳大利亚机器学习研究所）；Ningbo Institute of Northwestern Polytechnical University（西北工业大学宁波研究院）；Research and Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）
Deep neural networks have achieved satisfactory performance in piles of medical image analysis tasks. However the training of deep neural network requires a large amount of samples with high-quality annotations. In medical image segmentation, it is very laborious and expensive to acquire precise pixel-level annotations. Aiming at training deep segmentation models on datasets with probably corrupted annotations, we propose a novel Meta Corrupted Pixels Mining (MCPM) method based on a simple meta mask network. Our method is targeted at automatically estimate a weighting map to evaluate the impor…
FSDA-DG: Improving Cross-Domain Generalizability of Medical Image Segmentation with Few Source Domain Annotations
School of Biomedical Engineering（生物医学工程学院）；College of Information Science and Technology（信息科学与技术学院）；State Key Laboratory of Industrial Control Technology（工业控制技术国家重点实验室）；School of Information（信息学院）；Yunnan Key Laboratory of Intelligent Systems and Computing（云南智能系统与计算重点实验室）；Guangdong Provincial Key Laboratory of Medical Image Processing（广东省医学图像处理重点实验室）；Guangdong Province Engineering Laboratory for Medical Imaging and Diagnostic Technology（广东省医学影像与诊断技术工程实验室）；Pazhou Lab（琶洲实验室）
Medical image annotation is a major hurdle for developing precise and robust machine learning models. Annotation is expensive, time-consuming, and often requires expert knowledge, particularly in the medical field. Here, we suggest using minimal user interaction in the form of extreme point clicks to train a segmentation model which, in effect, can be used to speed up medical image annotation. An initial segmentation is generated based on the extreme points utilizing the random walker algorithm. This initial segmentation is then used as a noisy supervision signal to train a fully convolutional…
Xidian University（西安电子科技大学）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
3D image segmentation is one of the most important and ubiquitous problems in medical image processing. It provides detailed quantitative analysis for accurate disease diagnosis, abnormal detection, and classification. Currently deep learning algorithms are widely used in medical image segmentation, most algorithms trained models with full annotated datasets. However, obtaining medical image datasets is very difficult and expensive, and full annotation of 3D medical image is a monotonous and time-consuming work. Partially labelling informative slices in 3D images will be a great relief of manu…
National Institutes of Health Clinical Center（美国国立卫生研究院临床中心）；Beijing Institute of Big Data Research（北京大数据研究院）
Image segmentation is a fundamental problem in medical image analysis. In recent years, deep neural networks achieve impressive performances on many medical image segmentation tasks by supervised learning on large manually annotated data. However, expert annotations on big medical datasets are tedious, expensive or sometimes unavailable. Weakly supervised learning could reduce the effort for annotation but still required certain amounts of expertise. Recently, deep learning shows a potential to produce more accurate predictions than the original erroneous labels. Inspired by this, we introduce…
RefineSeg: Dual Coarse-to-Fine Learning for Medical Image Segmentation
School of Engineering, College of Engineering and Physical Sciences（工程学院，工程与物理科学学院）；University of Birmingham（伯明翰大学）；Oxford Center for Clinical Magnetic Resonance Research (OCMR)（牛津临床磁共振研究所以（OCMR））；University of Oxford（牛津大学）；John Radcliffe Hospital（约翰·拉德克利夫医院）；Johns Hopkins University School of Medicine（约翰霍普金斯大学医学院）；Queen Mary University London（伦敦女王学院）；Barts Health NHS Trust（巴特斯医疗信托）
FedIA: Federated Medical Image Segmentation with Heterogeneous Annotation Completeness
School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Engineering, Hong Kong University of Science and Technology（香港科技大学工程学院）
Volumetric Medical Image Segmentation via Scribble Annotations and Shape Priors
Shanghai Jiao Tong University（上海交通大学）；Ruijin Hospital, Shanghai Jiao Tong University of Medicine（上海交通大学医学院附属瑞金医院）；Shanghai Jiao Tong University of Medicine（上海交通大学医学院）
MILA (Quebec Artificial Intelligence Institute)（魁北克人工智能研究所MILA）；McGill University（麦吉尔大学）；NeuroRx Research（NeuroRx研究公司）；University of Edinburgh（爱丁堡大学）；The Alan Turing Institute（艾伦·图灵研究所）
Generalization is an important attribute of machine learning models, particularly for those that are to be deployed in a medical context, where unreliable predictions can have real world consequences. While the failure of models to generalize across datasets is typically attributed to a mismatch in the data distributions, performance gaps are often a consequence of biases in the 'ground-truth' label annotations. This is particularly important in the context of medical image segmentation of pathological structures (e.g. lesions), where the annotation process is much more subjective, and affecte…
Diversified and Personalized Multi-rater Medical Image Segmentation
University of Electronic Science and Technology of China（电子科技大学）；The Chinese University of Hong Kong（香港中文大学）；University of Oxford（牛津大学）；Monash University（莫纳什大学）
Multi-rater Prompting for Ambiguous Medical Image Segmentation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Institute of Wenzhou, Zhejiang University（浙江大学温州研究院）；State Key Laboratory of Transvascular Implantation Devices of The Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管植入装置国家重点实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Notre Dame（圣母大学）；School of Public Health Zhejiang University（浙江大学公共卫生学院）
From Few to More: Scribble-based Medical Image Segmentation via Masked Context Modeling and Continuous Pseudo Labels
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, School of Computer Science and Engineering, Northwestern Polytechnical University（集成空天地海大数据应用技术国家工程实验室，计算机科学与工程学院，西北工业大学）
Interactive 3D Medical Image Segmentation with SAM 2
East China Normal University（华东师范大学）；Tongji University（同济大学）；Shanghai United Imaging Healthcare Advanced Technology Research Institute Co., Ltd.（上海联影医疗科技先进技术研究院有限公司）
本文提出将 3D 医学切片视为视频帧、利用 SAM 2 零样本传播单帧标注的交互式分割流程，实验表明其可在特定场景缩小与监督方法差距并降低标注成本。
Carleton University（卡尔顿大学）；Canada CIFAR AI Chair（加拿大CIFAR人工智能讲席）
Medical image segmentation methods typically rely on numerous dense annotated images for model training, which are notoriously expensive and time-consuming to collect. To alleviate this burden, weakly supervised techniques have been exploited to train segmentation models with less expensive annotations. In this paper, we propose a novel point-supervised contrastive variance method (PSCV) for medical image semantic segmentation, which only requires one pixel-point from each organ category to be annotated. The proposed method trains the base segmentation network by using a novel contrastive vari…
Stitching, Fine-tuning, Re-training: A SAM-enabled Framework for Semi-supervised 3D Medical Image Segmentation
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学现代软件工程国家重点实验室）；National Institute of Healthcare Data Science, Nanjing University（南京大学医疗数据科学国家研究院）
The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitat…
Curating a large scale fully-annotated dataset can be both labour-intensive and expertise-demanding, especially for medical images. To alleviate this problem, we propose to utilize solely scribble annotations for weakly supervised segmentation. Existing solutions mainly leverage selective losses computed solely on annotated areas and generate pseudo gold standard segmentation by propagating labels to adjacent areas. However, these methods could suffer from the inaccurate and sometimes unrealistic pseudo segmentation due to the insufficient supervision and incomplete shape features. Different f…
We propose adversarial constrained-CNN loss, a new paradigm of constrained-CNN loss methods, for weakly supervised medical image segmentation. In the new paradigm, prior knowledge is encoded and depicted by reference masks, and is further employed to impose constraints on segmentation outputs through adversarial learning with reference masks. Unlike pseudo label methods for weakly supervised segmentation, such reference masks are used to train a discriminator rather than a segmentation network, and thus are not required to be paired with specific images. Our new paradigm not only greatly facil…
Coordinated Science Laboratory, UIUC（伊利诺伊大学厄巴纳-香槟分校协同科学实验室）；VinUni-Illinois Smart Health Center & CECS, VinUniversity（VinUniversity温大-伊利诺伊智能健康中心与计算机工程与计算机科学学院）；VinUniversity（温大）；Vingroup Big Data Institute（Vingroup大数据研究所）
The work discusses the use of machine learning algorithms for anomaly detection in medical image analysis and how the performance of these algorithms depends on the number of annotators and the quality of labels. To address the issue of subjectivity in labeling with a single annotator, we introduce a simple and effective approach that aggregates annotations from multiple annotators with varying levels of expertise. We then aim to improve the efficiency of predictive models in abnormal detection tasks by estimating hidden labels from multiple annotations and using a re-weighted loss function to…
Paul C. Lauterbur Research Center for Biomedical Imaging, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院保罗·C·劳特伯生物医学成像研究中心）；Peng Cheng Laboratory（鹏城实验室）；Pazhou Laboratory（琶洲实验室）；Guangdong General Hospital, Guangdong Academy of Medical Sciences（广东省人民医院（广东省医学科学院））；Renmin Hospital of Wuhan University（武汉大学人民医院）；School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University（北京大学深圳研究生院电子与计算机工程学院）；Brain Cognition and Brain Disease Institute, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院脑认知与脑疾病研究所）；ETS Montreal（蒙特利尔高等技术学院）
Automatic medical image segmentation plays a critical role in scientific research and medical care. Existing high-performance deep learning methods typically rely on large training datasets with high-quality manual annotations, which are difficult to obtain in many clinical applications. Here, we introduce Annotation-effIcient Deep lEarning (AIDE), an open-source framework to handle imperfect training datasets. Methodological analyses and empirical evaluations are conducted, and we demonstrate that AIDE surpasses conventional fully-supervised models by presenting better performance on open dat…
College of Computer Science and Technology, Huaqiao University（华侨大学计算机科学与技术学院）；School of Economics and Finance, Huaqiao University（华侨大学经济与金融学院）
Despite the remarkable performance of deep learning methods on various tasks, most cutting-edge models rely heavily on large-scale annotated training examples, which are often unavailable for clinical and health care tasks. The labeling costs for medical images are very high, especially in medical image segmentation, which typically requires intensive pixel/voxel-wise labeling. Therefore, the strong capability of learning and generalizing from limited supervision, including a limited amount of annotations, sparse annotations, and inaccurate annotations, is crucial for the successful applicatio…
Imperial College London（帝国理工学院）；NIHR Imperial Biomedical Research Centre（国家健康研究所帝国生物医学研究中心）
Machine learning has been widely adopted for medical image analysis in recent years given its promising performance in image segmentation and classification tasks. The success of machine learning, in particular supervised learning, depends on the availability of manually annotated datasets. For medical imaging applications, such annotated datasets are not easy to acquire, it takes a substantial amount of time and resource to curate an annotated medical image set. In this paper, we propose an efficient annotation framework for brain MR images that can suggest informative sample images for human…
Imperial College London（帝国理工学院）；University of Oxford（牛津大学）
Machine learning has been widely adopted for medical image analysis in recent years given its promising performance in image segmentation and classification tasks. As a data-driven science, the success of machine learning, in particular supervised learning, largely depends on the availability of manually annotated datasets. For medical imaging applications, such annotated datasets are not easy to acquire. It takes a substantial amount of time and resource to curate an annotated medical image set. In this paper, we propose an efficient annotation framework for brain tumour images that is able t…
Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability
University College London（伦敦大学学院）；Queen Mary University of London（伦敦玛丽女王大学）；University of Oxford（牛津大学）；University College London Hospitals NHS Trust（伦敦大学学院医院NHS信托基金会）；Imperial College Healthcare（帝国理工学院医疗保健）
Towards Collective Intelligence: Uncertainty-aware SAM Adaptation for Ambiguous Medical Image Segmentation
Department of Computer Science, The University of Alabama at Birmingham（阿拉巴马大学伯明翰分校计算机科学系）；Doctoral Training Centre, University of Oxford（牛津大学博士培训中心）；Department of Biomedical Engineering and Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学生物医学工程系和电气与计算机工程系）；Computational Biology Department, Carnegie Mellon University（卡内基梅隆大学计算生物学系）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
