Supervised learning has proved effective for medical image analysis. However, it can utilize only the small labeled portion of data; it fails to leverage the large amounts of unlabeled data that is often available in medical image datasets. Supervised models are further handicapped by domain shifts, when the labeled dataset, despite being large enough, fails to cover different protocols or ethnicities. In this paper, we introduce \emph{extreme consistency}, which overcomes the above limitations, by maximally leveraging unlabeled data from the same or a different domain in a teacher-student semi-supervised paradigm. Extreme consistency is the process of sending an extreme transformation of a given image to the student network and then constraining its prediction to be consistent with the teacher network's prediction for the untransformed image. The extreme nature of our consistency loss distinguishes our method from related works that yield suboptimal performance by exercising only mild prediction consistency. Our method is 1) auto-didactic, as it requires no extra expert annotations; 2) versatile, as it handles both domain shift and limited annotation problems; 3) generic, as it is readily applicable to classification, segmentation, and detection tasks; and 4) simple to implement, as it requires no adversarial training. We evaluate our method for the tasks of lesion and retinal vessel segmentation in skin and fundus images. Our experiments demonstrate a significant performance gain over both modern supervised networks and recent semi-supervised models. This performance is attributed to the strong regularization enforced by extreme consistency, which enables the student network to learn how to handle extreme variants of both labeled and unlabeled images. This enhances the network's ability to tackle the inevitable same- and cross-domain data variability during inference.
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科技大学）；Imsight Technology, Co., Ltd.（深圳视见科技有限公司）；The University of Hong Kong（香港大学）；Shenzhen People's Hospital（深圳市人民医院）；Guangdong Provincial People's Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）；Queen Marry Hospital（玛丽医院）；Hospital Authority（医院管理局）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Two DL models were developed using radiograph-level annotations (yes or no disease) and fine-grained lesion-level annotations (lesion bounding boxes), respectively named CheXNet and CheXDet. The models' internal classification performance and lesion localization performance were compared on a testing set (n=2,922), external classification performance was compared on NIH-Google (n=4,376) and PadChest (n=24,536) datasets, and external lesion localization performance was compared on NIH-ChestX-ray14 dataset (n=880). The models were also compared to radiologists on a subset of the internal testing…
Addressing data annotation scarcity in Brain Tumor Segmentation on 3D MRI scan Using a Semi-Supervised Teacher-Student Framework
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics（人工智能学院，南京航空航天大学）；Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（脑机智能技术重点实验室，教育部）；Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics（深圳研究院，南京航空航天大学）
Due to the high human cost of annotation, it is non-trivial to curate a large-scale medical dataset that is fully labeled for all classes of interest. Instead, it would be convenient to collect multiple small partially labeled datasets from different matching sources, where the medical images may have only been annotated for a subset of classes of interest. This paper offers an empirical understanding of an under-explored problem, namely partially supervised multi-label classification (PSMLC), where a multi-label classifier is trained with only partially labeled medical images. In contrast to …
Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model
Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学研究所）；Institute of Image Analysis and Computer Vision, Faculty of Informatics and Data Science, University of Regensburg（雷根斯堡大学信息学与数据科学学院图像分析与计算机视觉研究所）；Institute for Pathology, Hannover Medical School（汉诺威医学院病理学研究所）；Institute for Neuropathology, University Clinic Freiburg（弗莱堡大学医院神经病理学研究所）；Institute for Experimental Molecular Imaging, RWTH Aachen University（亚琛工业大学实验分子影像学研究所）
Cycle-Consistent Multi-Graph Matching for Self-Supervised Annotation of C.Elegans
Max-Delbrueck-Center for Molecular Medicine in the Helmholtz Association（马克斯·德尔布鲁克医学研究中心（马克斯·德尔布鲁克医学研究中心隶属于马克斯·德尔布鲁克协会））；Helmholtz Imaging（海德堡影像中心）；University of Potsdam（波茨坦大学）；Heidelberg University（海德堡大学）
AnyCXR: Human Anatomy Segmentation of Chest X-ray at Any Acquisition Position using Multi-stage Domain Randomized Synthetic Data with Imperfect Annotations and Conditional Joint Annotation Regularization Learning
Department of Radiology, Northwestern University, Chicago, IL, USA（西北大学放射科）；Data Science Institute, Vanderbilt University, Nashville, TN, USA（范德比尔特大学数据科学研究院）；Department of Orthopedics, The Second Hospital of Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二医院骨科）；Second Clinical Medical College, Shanxi Medical University, Taiyuan, Shanxi, P.R. China（山西医科大学第二临床医学院）；School of Computer Science, University of Sydney, Sydney, NSW, Australia（悉尼大学计算机科学学院）
Weed Detection in Challenging Field Conditions: A Semi-Supervised Framework for Overcoming Shadow Bias and Data Scarcity
College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；Agriculture Technology and Adoption Centre, James Cook University（詹姆斯库克大学农业技术与采纳中心）；ARC Training Centre in Plant Biosecurity（植物生物安全培训中心）
Jordan University of Science and Technology（约旦科技大学）
The scarcity of high-quality annotated medical imaging datasets is a major problem that collides with machine learning applications in the field of medical imaging analysis and impedes its advancement. Self-supervised learning is a recent training paradigm that enables learning robust representations without the need for human annotation which can be considered an effective solution for the scarcity of annotated medical data. This article reviews the state-of-the-art research directions in self-supervised learning approaches for image data with a concentration on their applications in the fiel…
Addressing Annotation Scarcity in Hyperspectral Brain Image Segmentation with Unsupervised Domain Adaptation
Technical University of Munich, Munich, Germany（慕尼黑技术大学）；Department of Computing, Imperial College London, London, UK（伦敦帝国学院计算机系）；Munich Center for Machine Learning (MCML), Munich, Germany（慕尼黑机器学习中心（MCML））
Active Label Cleaning for Reliable Detection of Electron Dense Deposits in Transmission Electron Microscopy Images
School of Biomedical Engineering（生物医学工程学院）；Southern Medical University（南方医科大学）；College of Letters and Science（文理学院）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；School of Basic Medical Sciences（基础医学学院）；Department of Nephrology（肾内科）；Nanfang Hospital, Southern Medical University（南方医科大学南芳医院）
Handling Supervision Scarcity in Chest X-ray Classification: Long-Tailed and Zero-Shot Learning
University of Technology, Vietnam（越南技术大学）；National University of Singapore（新加坡国立大学）；University of California, San Diego（加州大学圣地亚哥分校）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
This article discusses how the language of causality can shed new light on the major challenges in machine learning for medical imaging: 1) data scarcity, which is the limited availability of high-quality annotations, and 2) data mismatch, whereby a trained algorithm may fail to generalize in clinical practice. Looking at these challenges through the lens of causality allows decisions about data collection, annotation procedures, and learning strategies to be made (and scrutinized) more transparently. We discuss how causal relationships between images and annotations can not only have profound…
Overcoming Annotation Bottlenecks in Underwater Fish Segmentation: A Robust Self-Supervised Learning Approach
College of Science and Engineering, James Cook University（詹姆斯库克大学科学与工程学院）；ARC Research Hub for Supercharging Tropical Aquaculture through Genetic Solutions, James Cook University（詹姆斯库克大学ARC热带水产养殖基因解决方案研究中心）
A Lung Nodule Dataset with Histopathology-based Cancer Type Annotation
School of Computer Science and Technology, Shandong University of Finance and Economics（山东财经大学计算机科学与技术学院）；School of Information Science and Technology, Linyi University（临沂大学信息科学与技术学院）；Thoracic Surgery Department of Linyi Central Hospital（临沂市中心医院胸外科）；Personnel Department of Linyi Central Hospital（临沂市中心医院人事科）
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；University of Science and Technology of China（中国科学技术大学）
Most Visual Question Answering (VQA) models suffer from the language prior problem, which is caused by inherent data biases. Specifically, VQA models tend to answer questions (e.g., what color is the banana?) based on the high-frequency answers (e.g., yellow) ignoring image contents. Existing approaches tackle this problem by creating delicate models or introducing additional visual annotations to reduce question dependency while strengthening image dependency. However, they are still subject to the language prior problem since the data biases have not been even alleviated. In this paper, we i…
Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）
The success of deep convolutional neural networks (DCNNs) benefits from high volumes of annotated data. However, annotating medical images is laborious, expensive, and requires human expertise, which induces the label scarcity problem. Especially when encountering the domain shift, the problem becomes more serious. Although deep unsupervised domain adaptation (UDA) can leverage well-established source domain annotations and abundant target domain data to facilitate cross-modality image segmentation and also mitigate the label paucity problem on the target domain, the conventional UDA methods s…
Montreal Institute for Learning Algorithms（蒙特利尔学习算法研究所）
The use of remote sensing in humanitarian crisis response missions is well-established and has proven relevant repeatedly. One of the problems is obtaining gold annotations as it is costly and time consuming which makes it almost impossible to fine-tune models to new regions affected by the crisis. Where time is critical, resources are limited and environment is constantly changing, models has to evolve and provide flexible ways to adapt to a new situation. The question that we want to answer is if prioritization of samples provide better results in fine-tuning vs other classical sampling meth…
Imperial College London（帝国理工学院）；The Institute of Cancer Research（英国癌症研究院）；The Royal Marsden NHS Foundation Trust（皇家马斯登国民保健服务基金会信托）
Scarcity of high quality annotated images remains a limiting factor for training accurate image segmentation models. While more and more annotated datasets become publicly available, the number of samples in each individual database is often small. Combining different databases to create larger amounts of training data is appealing yet challenging due to the heterogeneity as a result of differences in data acquisition and annotation processes, often yielding incompatible or even conflicting information. In this paper, we investigate and propose several strategies for learning from partially ov…
In Heterogeneous Face Recognition (HFR), the objective is to match faces across two different domains such as visible and thermal. Large domain discrepancy makes HFR a difficult problem. Recent methods attempting to fill the gap via synthesis have achieved promising results, but their performance is still limited by the scarcity of paired training data. In practice, large-scale heterogeneous face data are often inaccessible due to the high cost of acquisition and annotation process as well as privacy regulations. In this paper, we propose a new face hallucination paradigm for HFR, which not on…
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；University of São Paulo（圣保罗大学）；Federal University of Mato Grosso do Sul（南马托格罗索联邦大学）
Semantic segmentation is a classic computer vision task with multiple applications, which includes medical and remote sensing image analysis. Despite recent advances with deep-based approaches, labeling samples (pixels) for training models is laborious and, in some cases, unfeasible. In this paper, we present two novel meta learning methods, named WeaSeL and ProtoSeg, for the few-shot semantic segmentation task with sparse annotations. We conducted extensive evaluation of the proposed methods in different applications (12 datasets) in medical imaging and agricultural remote sensing, which are …
School of Electronics & Information Engineering（电子与信息工程学院）；Shanghai University of Electric Power（上海电力大学）
To address the problem of scarcity and high annotation costs of rotated image table detection datasets, this paper proposes a method for building a rotated image table detection dataset. Based on the ICDAR2019MTD modern table detection dataset, we refer to the annotation format of the DOTA dataset to create the TRR360D rotated table detection dataset. The training set contains 600 rotated images and 977 annotated instances, and the test set contains 240 rotated images and 499 annotated instances. The AP50(T<90) evaluation metric is defined, and this dataset is available for future researchers …
Diffusion-Based Tumor Inpainting for Renal Segmentation under Clinical Data Scarcity
University of Tartu（塔尔图大学）；University of Bern（伯尔尼大学）；McGill University（麦吉尔大学）；Mila – Québec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）；Better Medicine OÜ（Better Medicine 公司）；STACC OÜ（STACC 公司）
Crucial-Diff: A Unified Diffusion Model for Crucial Image and Annotation Synthesis in Data-scarce Scenarios
School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学先进科技学院）；School of Computer Science and Technology, Soochow University（苏州大学计算机科学与技术学院）；School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机科学学院）；Faculty of Information Technology and Communication Sciences, Tampere University（塔尔库大学信息科技与通信科学学院）
University of Toronto（多伦多大学）；University of British Columbia（不列颠哥伦比亚大学）；Sunnybrook Health Science Center（森尼布鲁克健康科学中心）；Sunnybrook Research Institute（森尼布鲁克研究所）
Whole slide images (WSIs) pose unique challenges when training deep learning models. They are very large which makes it necessary to break each image down into smaller patches for analysis, image features have to be extracted at multiple scales in order to capture both detail and context, and extreme class imbalances may exist. Significant progress has been made in the analysis of these images, thanks largely due to the availability of public annotated datasets. We postulate, however, that even if a method scores well on a challenge task, this success may not translate to good performance in a…
Shandong University（山东大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Illinois Institute of Technology（伊利诺伊理工学院）
Several studies have recently pointed that existing Visual Question Answering (VQA) models heavily suffer from the language prior problem, which refers to capturing superficial statistical correlations between the question type and the answer whereas ignoring the image contents. Numerous efforts have been dedicated to strengthen the image dependency by creating the delicate models or introducing the extra visual annotations. However, these methods cannot sufficiently explore how the visual cues explicitly affect the learned answer representation, which is vital for language reliance alleviatio…
The Chinese University of Hong Kong（香港中文大学）；Stanford University（斯坦福大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Medical image annotations are prohibitively time-consuming and expensive to obtain. To alleviate annotation scarcity, many approaches have been developed to efficiently utilize extra information, e.g.,semi-supervised learning further exploring plentiful unlabeled data, domain adaptation including multi-modality learning and unsupervised domain adaptation resorting to the prior knowledge from additional modality. In this paper, we aim to investigate the feasibility of simultaneously leveraging abundant unlabeled data and well-established cross-modality data for annotation-efficient medical imag…
SuoiAI: Building a Dataset for Aquatic Invertebrates in Vietnam
Nuoc Solutions；Microsoft（微软公司）；University of Oslo（奥斯陆大学）；Bowdoin College（博德因学院）；Fulbright University Vietnam（富布赖特大学越南分校）；Westerdijk Fungal Biodiversity Institute（Westerdijk真菌多样性研究所）
National Heart and Lung Institute, Imperial College London（帝国理工学院国家心肺研究所）；Cardiovascular Research Centre, Royal Brompton Hospital（皇家布朗普顿医院心血管研究中心）；Department of Biomedical and Engineering, Imperial College London（帝国理工学院生物医学工程系）；Edinburgh Centre for Robotics, Heriot-Watt University（赫瑞-瓦特大学爱丁堡机器人中心）；School of Biomedical Engineering, Sun Yat-sen University（中山大学生物医学工程学院）
The destitution of image data and corresponding expert annotations limit the training capacities of AI diagnostic models and potentially inhibit their performance. To address such a problem of data and label scarcity, generative models have been developed to augment the training datasets. Previously proposed generative models usually require manually adjusted annotations (e.g., segmentation masks) or need pre-labeling. However, studies have found that these pre-labeling based methods can induce hallucinating artifacts, which might mislead the downstream clinical tasks, while manual adjustment …
PartImageNet++ Dataset: Enhancing Visual Models with High-Quality Part Annotations
Department of Computer Science and Technology, BNRist, Institute for Artificial Intelligence, IDG/McGovern Institute for Brain Research, Tsinghua University（计算机科学与技术系，BNRist，人工智能研究所，IDG/McGovern脑科学研究所，清华大学）；School of Computer Science and Technology, Shandong University（计算机科学与技术学院，山东大学）；Harbin Institute of Technology（哈尔滨工业大学）；Duke University（杜克大学）；Beijing Institute of Technology（北京理工大学）；Chinese Institute for Brain Research（中国脑科学研究所）
University of Edinburgh（爱丁堡大学）；University of California（加利福尼亚大学）；Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of Essex（埃塞克斯大学）；The Alan Turing Institute（艾伦·图灵研究所）
Automated pathology segmentation remains a valuable diagnostic tool in clinical practice. However, collecting training data is challenging. Semi-supervised approaches by combining labelled and unlabelled data can offer a solution to data scarcity. An approach to semi-supervised learning relies on reconstruction objectives (as self-supervision objectives) that learns in a joint fashion suitable representations for the task. Here, we propose Anatomy-Pathology Disentanglement Network (APD-Net), a pathology segmentation model that attempts to learn jointly for the first time: disentanglement of an…
Concordia University（康考迪亚大学）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；Western University（韦仕敦大学）；St. Olavs Hospital（圣奥拉夫医院）；Trondheim University Hospital（特隆赫姆大学医院）；Norwegian University of Science and Technology (NTNU)（挪威科技大学）；SINTEF Digital（SINTEF数字研究所）
Purpose: Registration and segmentation of magnetic resonance (MR) and ultrasound (US) images play an essential role in surgical planning and resection of brain tumors. However, validating these techniques is challenging due to the scarcity of publicly accessible sources with high-quality ground truth information. To this end, we propose a unique annotation dataset of tumor tissues and resection cavities from the previously published RESECT dataset (Xiao et al. 2017) to encourage a more rigorous assessments of image processing techniques. Acquisition and validation methods: The RESECT database …
National University of Defense Technology（国防科技大学）；National University of Singapore（新加坡国立大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The performance of deep learning based semantic segmentation models heavily depends on sufficient data with careful annotations. However, even the largest public datasets only provide samples with pixel-level annotations for rather limited semantic categories. Such data scarcity critically limits scalability and applicability of semantic segmentation models in real applications. In this paper, we propose a novel transferable semi-supervised semantic segmentation model that can transfer the learned segmentation knowledge from a few strong categories with pixel-level annotations to unseen weak c…
Dual Agreement Consistency Learning for Semi-Supervised Fetal Ultrasound Segmentation
Research Ireland Centre for Research Training in Machine Learning（爱尔兰研究机器学习研究中心）；School of Medicine, University College Dublin（都柏林大学医学院）；School of Computer Science, University College Dublin（都柏林大学计算机科学学院）；School of Computer Science and Digital Technologies, Aston University（阿斯顿大学计算机科学与数字技术学院）
