The popular methods for semi-supervised semantic segmentation mostly adopt a unitary network model using convolutional neural networks (CNNs) and enforce consistency of the model's predictions over perturbations applied to the inputs or model. However, such a learning paradigm suffers from two critical limitations: a) learning the discriminative features for the unlabeled data; b) learning both global and local information from the whole image. In this paper, we propose a novel Semi-supervised Learning (SSL) approach, called Transformer-CNN Cohort (TCC), that consists of two students with one based on the vision transformer (ViT) and the other based on the CNN. Our method subtly incorporates the multi-level consistency regularization on the predictions and the heterogeneous feature spaces via pseudo-labeling for the unlabeled data. First, as the inputs of the ViT student are image patches, the feature maps extracted encode crucial class-wise statistics. To this end, we propose class-aware feature consistency distillation (CFCD) that first leverages the outputs of each student as the pseudo labels and generates class-aware feature (CF) maps for knowledge transfer between the two students. Second, as the ViT student has more uniform representations for all layers, we propose consistency-aware cross distillation (CCD) to transfer knowledge between the pixel-wise predictions from the cohort. We validate the TCC framework on Cityscapes and Pascal VOC 2012 datasets, which outperforms existing SSL methods by a large margin.
Intrapartum Ultrasound Image Segmentation of Pubic Symphysis and Fetal Head Using Dual Student-Teacher Framework with CNN-ViT Collaborative Learning
College of Information Science and Technology, Jinan University（暨南大学信息科学技术学院）；Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Information Technology, Jinan University（暨南大学广东省中医信息技术重点实验室）；Auckland Bioengineering Institute, University of Auckland（奥克兰大学生物工程研究所）；Universitat de Barcelona（巴塞罗那大学）；Institució Catalana de Recerca i Estudis Avançats (ICREA)（加泰罗尼亚研究与高级研究学院（ICREA））
Convolutional neural network based deep-learning architecture for intraprostatic tumour contouring on PSMA PET images in patients with primary prostate cancer
University of Freiburg（弗莱堡大学）；Medical Center – University of Freiburg（弗莱堡大学医学中心）；Faculty of Medicine, University of Freiburg（弗莱堡大学医学院）；German Cancer Consortium (DKTK)（德国癌症联盟）；Medical School of Nanjing University（南京大学医学院）；Affiliated Drum Tower Hospital（鼓楼医院）；Hannover Medical School（汉诺威医学院）；Institute for Surgical Pathology, Medical Center – University of Freiburg（弗莱堡大学医学中心外科病理研究所）
Cross-Cohort Generalizability of Deep and Conventional Machine Learning for MRI-based Diagnosis and Prediction of Alzheimer's Disease
Erasmus MC（伊拉斯谟医学中心）；Maastricht University Medical Center（马斯特里赫特大学医学中心）；University Medical Center Groningen（格罗宁根大学医学中心）；University Medical Center Utrecht（乌得勒支大学医学中心）；Radboud University Medical Center（拉德堡德大学医学中心）；Leiden University Medical Center（莱顿大学医学中心）；Leiden University（莱顿大学）
ATMS-KD: Adaptive Temperature and Mixed Sample Knowledge Distillation for a Lightweight Residual CNN in Agricultural Embedded Systems
TEDAEEP Research Group, FPL, Abdelmalek saadi University（TEDAEEP研究组，FPL，阿德莱尔马萨迪大学）；Smart System Laboratory (SSL), ENSIAS, Mohammed V University（智能系统实验室（SSL），ENSIAS，穆罕默德五世大学）
Semi-Supervised Brain Lesion Segmentation with an Adapted Mean Teacher Model
Peng Cheng Laboratory（鹏城实验室）；Xidian University（西安电子科技大学）；Beijing Institute of Technology（北京理工大学）；Deepwise AI Lab（深度智耀人工智能实验室）；The People’s Hospital of Nantong（南通市人民医院）；Peking University Third Hospital（北京大学第三医院）
Low-resolution Face Recognition in the Wild via Selective Knowledge Distillation
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beihang University（北京航空航天大学）
Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition
University Hospital of Strasbourg（斯特拉斯堡大学医院）；IRCAD（斯特拉斯堡 IRCAD 研究所）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；ICube（斯特拉斯堡大学 ICube 实验室）；University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）
ViTKD: Practical Guidelines for ViT feature knowledge distillation
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；International Digital Economy Academy (IDEA)（国际数字经济研究院（IDEA））；Beihang University（北京航空航天大学）
Predicting Lung Nodule Malignancies by Combining Deep Convolutional Neural Network and Handcrafted Features
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）；School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；School of Traditional Chinese Medicine, Southern Medical University（南方医科大学中医学院）
