Calibrating Segmentation Networks with Margin-based Label Smoothing
ÉTS Montréal（蒙特利尔高等工程技术学院）；International Laboratory on Learning Systems (ILLS)（国际学习系统实验室）；MILA（米拉研究所）；CNRS（法国国家科学研究中心）；Université Paris-Saclay（巴黎萨克雷大学）；CentraleSupélec（中央理工-高等电力学院）；Universitat Pompeu Fabra（庞培法布拉大学）；Centre de Recherche du Centre Hospitalier de l’Université de Montréal (CRCHUM)（蒙特利尔大学医院研究中心）
Balamurali Murugesan, Bingyuan Liu, Adrian Galdran, Ismail Ben Ayed, Jose Dolz
Despite the undeniable progress in visual recognition tasks fueled by deep neural networks, there exists recent evidence showing that these models are poorly calibrated, resulting in over-confident predictions. The standard practices of minimizing the cross entropy loss during training promote the predicted softmax probabilities to match the one-hot label assignments. Nevertheless, this yields a pre-softmax activation of the correct class that is significantly larger than the remaining activations, which exacerbates the miscalibration problem. Recent observations from the classification literature suggest that loss functions that embed implicit or explicit maximization of the entropy of predictions yield state-of-the-art calibration performances. Despite these findings, the impact of these losses in the relevant task of calibrating medical image segmentation networks remains unexplored. In this work, we provide a unifying constrained-optimization perspective of current state-of-the-art calibration losses. Specifically, these losses could be viewed as approximations of a linear penalty (or a Lagrangian term) imposing equality constraints on logit distances. This points to an important limitation of such underlying equality constraints, whose ensuing gradients constantly push towards a non-informative solution, which might prevent from reaching the best compromise between the discriminative performance and calibration of the model during gradient-based optimization. Following our observations, we propose a simple and flexible generalization based on inequality constraints, which imposes a controllable margin on logit distances. Comprehensive experiments on a variety of public medical image segmentation benchmarks demonstrate that our method sets novel state-of-the-art results on these tasks in terms of network calibration, whereas the discriminative performance is also improved.
Calibrating Undisciplined Over-Smoothing in Transformer for Weakly Supervised Semantic Segmentation
School of Computer and Information Science, Hefei University of Technology（合肥工业大学计算机与信息科学学院）；Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, School of Artificial Intelligence, Xidian University（教育部智能感知与图像理解重点实验室，西安电子科技大学人工智能学院）；Brain and Artificial Intelligence Laboratory, School of Automation, Northwestern Polytechnical University（脑与人工智能实验室，西北工业大学自动化学院）
The task of image segmentation is inherently noisy due to ambiguities regarding the exact location of boundaries between anatomical structures. We argue that this information can be extracted from the expert annotations at no extra cost, and when integrated into state-of-the-art neural networks, it can lead to improved calibration between soft probabilistic predictions and the underlying uncertainty. We built upon label smoothing (LS) where a network is trained on 'blurred' versions of the ground truth labels which has been shown to be effective for calibrating output predictions. However, LS …
Pattern Recognition and Intelligent Vision Lab, Beijing University of Posts and Telecommunications（北京邮电大学模式识别与智能视觉实验室）
In object detection, offset-guided and point-guided regression dominate anchor-based and anchor-free method separately. Recently, point-guided approach is introduced to anchor-based method. However, we observe points predicted by this way are misaligned with matched region of proposals and score of localization, causing a notable gap in performance. In this paper, we propose CPM R-CNN which contains three efficient modules to optimize anchor-based point-guided method. According to sufficient evaluations on the COCO dataset, CPM R-CNN is demonstrated efficient to improve the localization accura…
University of Central Florida（中佛罗里达大学）；Google Research（谷歌研究院）
We study the problem of semantic segmentation calibration. Lots of solutions have been proposed to approach model miscalibration of confidence in image classification. However, to date, confidence calibration research on semantic segmentation is still limited. We provide a systematic study on the calibration of semantic segmentation models and propose a simple yet effective approach. First, we find that model capacity, crop size, multi-scale testing, and prediction correctness have impact on calibration. Among them, prediction correctness, especially misprediction, is more important to miscali…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics（南京航空航天大学深圳研究院）；Hong Kong Polytechnic University（香港理工大学）；Lingnan University（岭南大学）；Hong Kong Metropolitan University（香港都会大学）
Image smoothing is a fundamental low-level vision task that aims to preserve salient structures of an image while removing insignificant details. Deep learning has been explored in image smoothing to deal with the complex entanglement of semantic structures and trivial details. However, current methods neglect two important facts in smoothing: 1) naive pixel-level regression supervised by the limited number of high-quality smoothing ground-truth could lead to domain shift and cause generalization problems towards real-world images; 2) texture appearance is closely related to object semantics, …
The classification of histopathology images fundamentally differs from traditional image classification tasks because histopathology images naturally exhibit a range of diagnostic features, resulting in a diverse range of annotator agreement levels. However, examples with high annotator disagreement are often either assigned the majority label or discarded entirely when training histopathology image classifiers. This widespread practice often yields classifiers that do not account for example difficulty and exhibit poor model calibration. In this paper, we ask: can we improve model calibration…
Imperial College London（伦敦帝国理工学院）；National University of Singapore（新加坡国立大学）
Despite impressive accuracy, deep neural networks are often miscalibrated and tend to overly confident predictions. Recent techniques like temperature scaling (TS) and label smoothing (LS) show effectiveness in obtaining a well-calibrated model by smoothing logits and hard labels with scalar factors, respectively. However, the use of uniform TS or LS factor may not be optimal for calibrating models trained on a long-tailed dataset where the model produces overly confident probabilities for high-frequency classes. In this study, we propose class-distribution-aware TS (CDA-TS) and LS (CDA-LS) by…
Long-Tailed Recognition on Binary Networks by Calibrating A Pre-trained Model
Minds and Company（明兹公司）；Upstage AI（上阶段人工智能公司）；Gwangju Institute of Science and Technology (GIST)（光州科学技术院）；Alsemy（阿尔塞米公司）；Seoul National University（首尔大学）
Shandong University（山东大学）；Microsoft Research（微软研究院）；Shenzhen Research Institute, Shandong University（山东大学深圳研究院）
This paper proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering. Unlike most other deep learning strategies applied in this context, our approach tackles these challenging problems by estimating edges and reconstructing images using only cascaded convolutional layers arranged such that no handcrafted or application-specific image-processing components are required. We apply the resulting transferrable pipeline to two different problem domains that are both sensitive to edges, nam…
SCMM: Calibrating Cross-modal Representations for Text-Based Person Search
College of Future Information Technology, Fudan University（复旦大学未来信息技术学院）；Department of Electrical and Computer Engineering, The University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）；College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；MEGVII Technology（MEGVII技术）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of AI and CS, Nantong University（南通大学人工智能与计算机科学学院）；Academy of Artificial Intelligence, SMBU（SMBU人工智能学院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）
Beyond scalar losses: calibrating segmentation models via gradient vector field surgery
School of Computation, Information and Technology, TUM（慕尼黑工业大学计算、信息与技术学院）；Munich Center for Machine Learning（慕尼黑机器学习中心）；Department of Radiology, Weill Cornell Medicine（威尔康乃尔医学院放射科）；School of Medicine and Health, TUM University Hospital（慕尼黑工业大学医院医学与健康学院）；Cornell Tech（康奈尔科技学院）；Department of Computing, Imperial College London（伦敦帝国理工学院计算系）
Mesh based segmentation for automated margin line generation on incisors receiving crown treatment
King Fahd University of Petroleum and Minerals（法赫德国王石油与矿物大学）；Polytechnique Montréal（蒙特利尔理工学院）；Centre d'intelligence artificielle appliquée (JACOBB)（应用人工智能中心 (JACOBB)）；Intellident Dentaire Inc.（Intellident Dentaire 公司）
