Confidence-Aware Paced-Curriculum Learning by Label Smoothing for Surgical Scene Understanding
National University of Singapore（新加坡国立大学）；The Chinese University of Hong Kong（香港中文大学）；National University of Singapore (Suzhou) Research Institute（新加坡国立大学苏州研究院）；Imperial College London（帝国理工学院）
Mengya Xu, Mobarakol Islam, Ben Glocker, Hongliang Ren
备注：
12 pages
发表信息：
IEEE Transactions on Automation Science and Engineering, 2022
Curriculum learning and self-paced learning are the training strategies that gradually feed the samples from easy to more complex. They have captivated increasing attention due to their excellent performance in robotic vision. Most recent works focus on designing curricula based on difficulty levels in input samples or smoothing the feature maps. However, smoothing labels to control the learning utility in a curriculum manner is still unexplored. In this work, we design a paced curriculum by label smoothing (P-CBLS) using paced learning with uniform label smoothing (ULS) for classification tasks and fuse uniform and spatially varying label smoothing (SVLS) for semantic segmentation tasks in a curriculum manner. In ULS and SVLS, a bigger smoothing factor value enforces a heavy smoothing penalty in the true label and limits learning less information. Therefore, we design the curriculum by label smoothing (CBLS). We set a bigger smoothing value at the beginning of training and gradually decreased it to zero to control the model learning utility from lower to higher. We also designed a confidence-aware pacing function and combined it with our CBLS to investigate the benefits of various curricula. The proposed techniques are validated on four robotic surgery datasets of multi-class, multi-label classification, captioning, and segmentation tasks. We also investigate the robustness of our method by corrupting validation data into different severity levels. Our extensive analysis shows that the proposed method improves prediction accuracy and robustness.
Imperial College London（帝国理工学院）；National University of Singapore（新加坡国立大学）；National Institute of Technology Tiruchirappalli（蒂鲁吉拉伯利国家理工学院）；The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
Purpose: In curriculum learning, the idea is to train on easier samples first and gradually increase the difficulty, while in self-paced learning, a pacing function defines the speed to adapt the training progress. While both methods heavily rely on the ability to score the difficulty of data samples, an optimal scoring function is still under exploration. Methodology: Distillation is a knowledge transfer approach where a teacher network guides a student network by feeding a sequence of random samples. We argue that guiding student networks with an efficient curriculum strategy can improve mod…
School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）
Weakly-supervised object detection (WOD) is a challenging problems in computer vision. The key problem is to simultaneously infer the exact object locations in the training images and train the object detectors, given only the training images with weak image-level labels. Intuitively, by simulating the selective attention mechanism of human visual system, saliency detection technique can select attractive objects in scenes and thus is a potential way to provide useful priors for WOD. However, the way to adopt saliency detection in WOD is not trivial since the detected saliency region might be …
University of Bucharest（布加勒斯特大学）；University of Trento（特伦托大学）；Romanian Young Academy（罗马尼亚青年科学院）；SecurifAI
Training (source) domain bias affects state-of-the-art object detectors, such as Faster R-CNN, when applied to new (target) domains. To alleviate this problem, researchers proposed various domain adaptation methods to improve object detection results in the cross-domain setting, e.g. by translating images with ground-truth labels from the source domain to the target domain using Cycle-GAN. On top of combining Cycle-GAN transformations and self-paced learning in a smart and efficient way, in this paper, we propose a novel self-paced algorithm that learns from easy to hard. Our method is simple …
Intel Corporation（英特尔公司）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
We propose to learn a curriculum or a syllabus for supervised learning and deep reinforcement learning with deep neural networks by an attachable deep neural network, called ScreenerNet. Specifically, we learn a weight for each sample by jointly training the ScreenerNet and the main network in an end-to-end self-paced fashion. The ScreenerNet neither has sampling bias nor requires to remember the past learning history. We show the networks augmented with the ScreenerNet achieve early convergence with better accuracy than the state-of-the-art curricular learning methods in extensive experiments…
Heuristic Self-Paced Learning for Domain Adaptive Semantic Segmentation under Adverse Conditions
National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, School of Computer Science, Wuhan University（国家多媒体软件工程研究中心、人工智能研究院、计算机科学学院、武汉大学）；Hubei Key Laboratory of Multimedia and Network Communication Engineering（湖北省多媒体与网络通信工程重点实验室）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）；School of Computer Science, Wuhan University, Wuhan, China（武汉大学计算机科学学院，武汉，中国）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
University of Science and Technology of China（中国科学技术大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）
Self-paced curriculum learning (SCL) has demonstrated its great potential in computer vision, natural language processing, etc. During training, it implements easy-to-hard sampling based on online estimation of data difficulty. Most SCL methods commonly adopt a loss-based strategy of estimating data difficulty and deweighting the `hard' samples in the early training stage. While achieving success in a variety of applications, SCL stills confront two challenges in a medical image analysis task, such as universal lesion detection, featuring insufficient and highly class-imbalanced data: (i) the …
Hierarchical Curriculum Learning for Surgical Video Generation: Focusing on Bilateral Axillo-Breast Approach Robotic Thyroidectomy Datasets
Seoul National University College of Medicine（首尔国立大学医学院）；Seoul National University Hospital（首尔国立大学医院）；YONSEI WITH-YOU Surgery Clinic（延世WITH-YOU外科诊所）；Institute of Convergence Medicine with Innovative Technology, Seoul National University Hospital（首尔国立大学医院创新技术融合医学研究所）
CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding
Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；Peng Cheng Laboratory (PCL)（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
Adaptive Mix for Semi-Supervised Medical Image Segmentation
School of Computer Science and Engineering, Northeastern University（计算机科学与工程学院，东北大学）；Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education（医学图像智能计算重点实验室，教育部）；Alberta Machine Intelligence Institute, University of Alberta（阿尔伯塔人工智能研究所，阿尔伯塔大学）
University of Electronic Science and Technology of China（电子科技大学）；Tsinghua University（清华大学）；Sichuan University（四川大学）；University of Florida（佛罗里达大学）
Noisy labels composed of correct and corrupted ones are pervasive in practice. They might significantly deteriorate the performance of convolutional neural networks (CNNs), because CNNs are easily overfitted on corrupted labels. To address this issue, inspired by an observation, deep neural networks might first memorize the probably correct-label data and then corrupt-label samples, we propose a novel yet simple self-paced resistance framework to resist corrupted labels, without using any clean validation data. The proposed framework first utilizes the memorization effect of CNNs to learn a cu…
Sun Yat-sen University（中山大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）
This paper aims to develop a novel cost-effective framework for face identification, which progressively maintains a batch of classifiers with the increasing face images of different individuals. By naturally combining two recently rising techniques: active learning (AL) and self-paced learning (SPL), our framework is capable of automatically annotating new instances and incorporating them into training under weak expert re-certification. We first initialize the classifier using a few annotated samples for each individual, and extract image features using the convolutional neural nets. Then, a…
Universitat Pompeu Fabra（庞培法布拉大学）；ICREA（加泰罗尼亚高级研究学院）；Ecole Centrale de Nantes（南特中央理工学院）；Technische Universität München（慕尼黑工业大学）；LMU München（慕尼黑大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
An adequate classification of proximal femur fractures from X-ray images is crucial for the treatment choice and the patients' clinical outcome. We rely on the commonly used AO system, which describes a hierarchical knowledge tree classifying the images into types and subtypes according to the fracture's location and complexity. In this paper, we propose a method for the automatic classification of proximal femur fractures into 3 and 7 AO classes based on a Convolutional Neural Network (CNN). As it is known, CNNs need large and representative datasets with reliable labels, which are hard to co…
Answering from Sure to Uncertain: Uncertainty-Aware Curriculum Learning for Video Question Answering
School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Computer Science & Software Engineering, University of Western Australia（西澳大学计算机科学与软件工程系）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算与通讯学院）；Department of Data Science & AI, Monash University（莫纳什大学数据科学与人工智能系）
CurConMix+: A Unified Spatio-Temporal Framework for Hierarchical Surgical Workflow Understanding
Department of Medical Device Management and Research, Samsung Advanced Institute for Health Sciences & Technology (SAIHST), Sungkyunkwan University（医学院设备管理与研究系，三星健康科学与技术研究院（SAIHST），全北大学）；Clinical Robotics and Embodied AI Research Center, Smart Healthcare Research Institute, Research Institute for Future Medicine, Samsung Medical Center（临床机器人与具身人工智能研究中心，智能医疗研究机构，未来医学研究所，三星医疗中心）；Department of Surgery, Samsung Medical Center（外科系，三星医疗中心）
National University of Singapore（新加坡国立大学）；NUSRI Suzhou（新加坡国立大学苏州研究院）；Singapore General Hospital（新加坡中央医院）；The Chinese University of Hong Kong（香港中文大学）；Imperial College London（伦敦帝国理工学院）
Generating surgical reports aimed at surgical scene understanding in robot-assisted surgery can contribute to documenting entry tasks and post-operative analysis. Despite the impressive outcome, the deep learning model degrades the performance when applied to different domains encountering domain shifts. In addition, there are new instruments and variations in surgical tissues appeared in robotic surgery. In this work, we propose class-incremental domain adaptation (CIDA) with a multi-layer transformer-based model to tackle the new classes and domain shift in the target domain to generate surg…
National University of Singapore（新加坡国立大学）；Imperial College London（伦敦帝国理工学院）；Singapore General Hospital（新加坡中央医院）；The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
Purpose: Surgery scene understanding with tool-tissue interaction recognition and automatic report generation can play an important role in intra-operative guidance, decision-making and postoperative analysis in robotic surgery. However, domain shifts between different surgeries with inter and intra-patient variation and novel instruments' appearance degrade the performance of model prediction. Moreover, it requires output from multiple models, which can be computationally expensive and affect real-time performance. Methodology: A multi-task learning (MTL) model is proposed for surgical report…
