Few-Shot Medical Image Segmentation with High-Fidelity Prototypes
University of Shanghai for Science and Technology（上海理工大学）；Universität Hamburg（汉堡大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Electronic Science and Technology of China（电子科技大学）；University of Surrey（萨里大学）
Song Tang, Shaxu Yan, Xiaozhi Qi, Jianxin Gao, Mao Ye, Jianwei Zhang, Xiatian Zhu
Few-shot Semantic Segmentation (FSS) aims to adapt a pretrained model to new classes with as few as a single labelled training sample per class. Despite the prototype based approaches have achieved substantial success, existing models are limited to the imaging scenarios with considerably distinct objects and not highly complex background, e.g., natural images. This makes such models suboptimal for medical imaging with both conditions invalid. To address this problem, we propose a novel Detail Self-refined Prototype Network (DSPNet) to constructing high-fidelity prototypes representing the object foreground and the background more comprehensively. Specifically, to construct global semantics while maintaining the captured detail semantics, we learn the foreground prototypes by modelling the multi-modal structures with clustering and then fusing each in a channel-wise manner. Considering that the background often has no apparent semantic relation in the spatial dimensions, we integrate channel-specific structural information under sparse channel-aware regulation. Extensive experiments on three challenging medical image benchmarks show the superiority of DSPNet over previous state-of-the-art methods.
EPL: Evidential Prototype Learning for Semi-supervised Medical Image Segmentation
Key Laboratory of High Confidence Software Technologies (Peking University)（高可信软件技术重点实验室（北京大学））；School of Computer Science, Peking University（北京大学计算机学院）
Shanghai Jiao Tong University（上海交通大学）；VoxelCloud, Inc.（体素云公司）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Despite the tremendous success of deep neural networks in medical image segmentation, they typically require a large amount of costly, expert-level annotated data. Few-shot segmentation approaches address this issue by learning to transfer knowledge from limited quantities of labeled examples. Incorporating appropriate prior knowledge is critical in designing high-performance few-shot segmentation algorithms. Since strong spatial priors exist in many medical imaging modalities, we propose a prototype-based method -- namely, the location-sensitive local prototype network -- that leverages spati…
Prototype Contrastive Consistency Learning for Semi-Supervised Medical Image Segmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Computer Vision Institute, College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院计算机视觉研究所）；Baoan Central Hospital of Shenzhen（深圳市宝安中心医院）
SPENet: Self-guided Prototype Enhancement Network for Few-shot Medical Image Segmentation
School of Computer Science, Beijing University of Technology, Beijing, China（北京理工大学计算机科学学院）；Institute of Automation Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Multimedia Department Xiaomi Inc, Beijing, China（小米公司多媒体部门）；Capital Medical University Affiliated Beijing Friendship Hospital, Beijing, China（首都医科大学附属北京友谊医院）
Efficient Prototype Consistency Learning in Medical Image Segmentation via Joint Uncertainty and Data Augmentation
Faculty of Science and Technology（科技学院）；University of Macau（澳门大学）；Key Laboratory of High Confidence Software Technologies (MOE)（高可信软件技术重点实验室）；School of Computer Science（计算机学院）；Peking University（北京大学）
Semi-supervised learning methods have been explored in medical image segmentation tasks due to the scarcity of pixel-level annotation in the real scenario. Proto-type alignment based consistency constraint is an intuitional and plausible solu-tion to explore the useful information in the unlabeled data. In this paper, we propose a mutual- and self- prototype alignment (MSPA) framework to better utilize the unlabeled data. In specific, mutual-prototype alignment enhances the information interaction between labeled and unlabeled data. The mutual-prototype alignment imposes two consistency constr…
Prototype Correlation Matching and Class-Relation Reasoning for Few-Shot Medical Image Segmentation
The Hong Kong Polytechnic University（香港理工大学）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；Durham University（杜伦大学）；Jarvis Research Center, Tencent YouTu Lab（腾讯优图实验室贾维斯研究中心）
Beyond Euclidean Prototypes: Spectral Disentanglement and Geodesic Matching for Few-Shot Medical Image Segmentation
School of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Tied Prototype Model for Few-Shot Medical Image Segmentation
Department of Physics and Technology, UiT - The Arctic University of Norway（物理与技术系，UiT - 北极大学）；Norwegian Centre for Clinical Artificial Intelligence (SPKI)（挪威临床人工智能中心）；University Hospital of North-Norway（北挪威大学医院）；Norwegian Computing Center（挪威计算中心）
Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation
National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；Medical Artificial Intelligence Research Institute of Renmin Hospital, Wuhan University（武汉大学人民医院医学人工智能研究院）；School of Computer Science, Central China Normal University（华中师范大学计算机学院）
Style-Aware Blending and Prototype-Based Cross-Contrast Consistency for Semi-Supervised Medical Image Segmentation
School of Information Science and Engineering（信息科学与工程学院）；Yunnan University（云南大学）；Yunnan Key Laboratory of Intelligent Systems and Computing（智能系统与计算云南省重点实验室）
ProCNS: Progressive Prototype Calibration and Noise Suppression for Weakly-Supervised Medical Image Segmentation
Southern University of Science and Technology（南方科技大学）；University of Hong Kong（香港大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）
Rethinking Medical Landmark Localization with Prototype Learning-based Progressive Offset Correction
Shenzhen University（深圳大学）；Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；School of Biomedical Engineering and Informatics, Nanjing Medical University（南京医科大学生物医学工程与信息学院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学大数据系统计算技术国家工程实验室）
Double Banking on Knowledge: Customized Modulation and Prototypes for Multi-Modality Semi-supervised Medical Image Segmentation
College of Computer Science, Sichuan University（四川大学计算机学院）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；College of Electrical Engineering, Sichuan University（四川大学电气工程学院）；School of Cyber Science and Engineering, Sichuan University（四川大学网络空间安全学院）
Leveraging Labelled Data Knowledge: A Cooperative Rectification Learning Network for Semi-supervised 3D Medical Image Segmentation
School of Mechanical Engineering and Automation, Northeastern University（东北大学机械工程与自动化学院）；Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey（萨里大学视觉、语音与信号处理中心）
Embracing Intra-Class Heterogeneity for Semi-Supervised Medical Image Segmentation: From Diversity to Precision
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Artificial Intelligence Institute, Shanghai University（上海大学人工智能研究院）；Department of Computer and Data Sciences, Case Western Reserve University（凯斯西储大学计算机与数据科学系）；Department of Biomedical Engineering, Case Western Reserve University（凯斯西储大学生物医学工程系）
Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentation
School of Biomedical Engineering (Suzhou), Division of Life Science and Medicine, University of Science and Technology of China, Hefei, China（生物医学工程学院（苏州），生命科学与医学系，中国科学技术大学，合肥，中国）；Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, China（苏州生物医学工程与技术研究所，中国科学院，苏州，中国）；Shanghai Innovation Institute, Shanghai, China（上海创新研究院，上海，中国）；Medical School of Tianjin University, Tianjin, China（天津大学医学院，天津，中国）；Jinan Guoke Medical and Technology Development Co., Ltd., Pharmaceutical Valley New Drug Creation Platform, Jinan, China（济南国科医药科技发展有限公司，药谷新药创制平台，济南，中国）
Medical image annotation typically requires expert knowledge and hence incurs time-consuming and expensive data annotation costs. To alleviate this burden, we propose a novel learning scenario, Exemplar Learning (EL), to explore automated learning processes for medical image segmentation with a single annotated image example. This innovative learning task is particularly suitable for medical image segmentation, where all categories of organs can be presented in one single image and annotated all at once. To address this challenging EL task, we propose an Exemplar Learning-based Synthesis Net (…
Imperial College London（帝国理工学院）；BioMedIA Group, Department of Computing（计算系BioMedIA小组）
Few-shot semantic segmentation (FSS) has great potential for medical imaging applications. Most of the existing FSS techniques require abundant annotated semantic classes for training. However, these methods may not be applicable for medical images due to the lack of annotations. To address this problem we make several contributions: (1) A novel self-supervised FSS framework for medical images in order to eliminate the requirement for annotations during training. Additionally, superpixel-based pseudo-labels are generated to provide supervision; (2) An adaptive local prototype pooling module pl…
Neural networks pre-trained on a self-supervision scheme have become the standard when operating in data rich environments with scarce annotations. As such, fine-tuning a model to a downstream task in a parameter-efficient but effective way, e.g. for a new set of classes in the case of semantic segmentation, is of increasing importance. In this work, we propose and investigate several contributions to achieve a parameter-efficient but effective adaptation for semantic segmentation on two medical imaging datasets. Relying on the recently popularized prompt tuning approach, we provide a prompt-a…
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University（北京交通大学先进信息科学与网络技术北京市重点实验室）；Department of Computer Science and Engineering, Michigan State University（密歇根州立大学计算机科学与工程系）
To bridge the gap between the source and target domains in unsupervised domain adaptation (UDA), the most common strategy puts focus on matching the marginal distributions in the feature space through adversarial learning. However, such category-agnostic global alignment lacks of exploiting the class-level joint distributions, causing the aligned distribution less discriminative. To address this issue, we propose in this paper a novel margin preserving self-paced contrastive Learning (MPSCL) model for cross-modal medical image segmentation. Unlike the conventional construction of contrastive p…
Illinois Institute of Technology（伊利诺伊理工学院）；University of Michigan（密歇根大学）
Recently, due to the increasing requirements of medical imaging applications and the professional requirements of annotating medical images, few-shot learning has gained increasing attention in the medical image semantic segmentation field. To perform segmentation with limited number of labeled medical images, most existing studies use Proto-typical Networks (PN) and have obtained compelling success. However, these approaches overlook the query image features extracted from the proposed representation network, failing to preserving the spatial connection between query and support images. In th…
DMAF-Net: An Effective Modality Rebalancing Framework for Incomplete Multi-Modal Medical Image Segmentation
College of Computer Science and Engineering, Chongqing University of Technology（重庆理工大学计算机科学与工程学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）
Xiaohe Healthcare, ByteDance（字节跳动小荷健康）；The University of Hong Kong（香港大学）
Medical image classification has been widely adopted in medical image analysis. However, due to the difficulty of collecting and labeling data in the medical area, medical image datasets are usually highly-imbalanced. To address this problem, previous works utilized class samples as prior for re-weighting or re-sampling but the feature representation is usually still not discriminative enough. In this paper, we adopt the contrastive learning to tackle the long-tailed medical imbalance problem. Specifically, we first propose the category prototype and adversarial proto-instance to generate repr…
Interpretable Medical Image Classification using Prototype Learning and Privileged Information
Experimental Radiology, University Hospital Ulm, Germany（实验放射学，乌尔姆大学医院，德国）；Department of Diagnostic and Interventional Radiology, University Hospital Ulm, Germany（诊断与介入放射学系，乌尔姆大学医院，德国）；i2SouI - Innovative Imaging in Surgical Oncology Ulm, University Hospital Ulm, Germany（i2SouI - 精准影像学在手术肿瘤学中的创新应用，乌尔姆大学医院，德国）
Diverse Normal Prototypes-Guided Contrastive Reconstruction for Medical Anomaly Detection
School of Mathematics, Shandong University（山东大学数学学院）；Department of Interventional Medicine and Minimally Invasive Oncology, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院介入医学与微创肿瘤科）；Department of Neurosurgery, The Second Qilu Hospital of Shandong University（山东大学第二齐鲁医院神经外科）
Prototype-Enhanced Confidence Modeling for Cross-Modal Medical Image-Report Retrieval
School of Computer Science, The University of Nottingham, NG8 1BB Nottingham, U.K.（计算机科学学院，诺丁汉大学）；Department of Intelligent Science, Xi’an Jiaotong-Liverpool University, China, 215123.（智能科学系，西安交通大学利物浦大学）
Due to the lack of properly annotated medical data, exploring the generalization capability of the deep model is becoming a public concern. Zero-shot learning (ZSL) has emerged in recent years to equip the deep model with the ability to recognize unseen classes. However, existing studies mainly focus on natural images, which utilize linguistic models to extract auxiliary information for ZSL. It is impractical to apply the natural image ZSL solutions directly to medical images, since the medical terminology is very domain-specific, and it is not easy to acquire linguistic models for the medical…
Decouple, Purify and Unite: Semantic-Structural Prototype Learning for Federated Medical Segmentation
Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（北京协和医院，中国医学科学院北京协和医学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Nanyang Technological University（南洋理工大学）；Ant Group（蚂蚁集团）；Shanghai Innovation Institute（上海创新研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Fourth Military Medical University（中国人民解放军空军军医大学）；Macao Polytechnic University（澳门理工大学）；Wuhan University（武汉大学）；The Hong Kong Polytechnic University（香港理工大学）
EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Models
Jilin University（吉林大学）；School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；ByteDance（字节跳动）；Institute of Translational Medicine, Shanghai Jiao Tong University（上海交通大学转化医学研究院）
