Uncertainty-Aware Deep Calibrated Salient Object Detection
Australian National University（澳大利亚国立大学）；Northwestern Polytechnical University（西北工业大学）；ACRV（澳大利亚机器人视觉中心）；Data61；University of Technology Sydney（悉尼科技大学）；Monash University（莫纳什大学）
Jing Zhang, Yuchao Dai, Xin Yu, Mehrtash Harandi, Nick Barnes, Richard Hartley
Existing deep neural network based salient object detection (SOD) methods mainly focus on pursuing high network accuracy. However, those methods overlook the gap between network accuracy and prediction confidence, known as the confidence uncalibration problem. Thus, state-of-the-art SOD networks are prone to be overconfident. In other words, the predicted confidence of the networks does not reflect the real probability of correctness of salient object detection, which significantly hinder their real-world applicability. In this paper, we introduce an uncertaintyaware deep SOD network, and propose two strategies from different perspectives to prevent deep SOD networks from being overconfident. The first strategy, namely Boundary Distribution Smoothing (BDS), generates continuous labels by smoothing the original binary ground-truth with respect to pixel-wise uncertainty. The second strategy, namely Uncertainty-Aware Temperature Scaling (UATS), exploits a relaxed Sigmoid function during both training and testing with spatially-variant temperature scaling to produce softened output. Both strategies can be incorporated into existing deep SOD networks with minimal efforts. Moreover, we propose a new saliency evaluation metric, namely dense calibration measure C, to measure how the model is calibrated on a given dataset. Extensive experimental results on seven benchmark datasets demonstrate that our solutions can not only better calibrate SOD models, but also improve the network accuracy.
Weakly-supervised salient object detection (WSOD) aims to develop saliency models using image-level annotations. Despite of the success of previous works, explorations on an effective training strategy for the saliency network and accurate matches between image-level annotations and salient objects are still inadequate. In this work, 1) we propose a self-calibrated training strategy by explicitly establishing a mutual calibration loop between pseudo labels and network predictions, liberating the saliency network from error-prone propagation caused by pseudo labels. 2) we prove that even a much…
Northwestern Polytechnical University（西北工业大学）；Australian National University（澳大利亚国立大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；EPFL（瑞士洛桑联邦理工学院）
Visual salient object detection (SOD) aims at finding the salient object(s) that attract human attention, while camouflaged object detection (COD) on the contrary intends to discover the camouflaged object(s) that hidden in the surrounding. In this paper, we propose a paradigm of leveraging the contradictory information to enhance the detection ability of both salient object detection and camouflaged object detection. We start by exploiting the easy positive samples in the COD dataset to serve as hard positive samples in the SOD task to improve the robustness of the SOD model. Then, we introdu…
Graph-Based Uncertainty Modeling and Multimodal Fusion for Salient Object Detection
Guangdong Key Laboratory of Intelligent Information Processing（广东智能信息处理重点实验室）；College of Electronics and Information Engineering（电子信息工程学院）；Shenzhen University（深圳大学）；Furong Laboratory（芙蓉实验室）；Central South University（中南大学）
Sun Yat-sen University（中山大学）；ByteDance Inc.（字节跳动公司）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Recent advances in deep learning significantly boost the performance of salient object detection (SOD) at the expense of labeling larger-scale per-pixel annotations. To relieve the burden of labor-intensive labeling, deep unsupervised SOD methods have been proposed to exploit noisy labels generated by handcrafted saliency methods. However, it is still difficult to learn accurate saliency details from rough noisy labels. In this paper, we propose to learn saliency from synthetic but clean labels, which naturally has higher pixel-labeling quality without the effort of manual annotations. Specifi…
UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）
Towards Reliable Medical Image Segmentation by Modeling Evidential Calibrated Uncertainty
National Key Laboratory of Fundamental Science on Synthetic Vision, the College of Computer Science, Sichuan University（国家基础科学合成视觉重点实验室、计算机科学学院、四川大学）；Department of Radiology, West China Hospital, Sichuan University（放射科、西昌医院、四川大学）；Saw Swee Hock School of Public Health, National University of Singapore（Saw Swee Hock 公共卫生学院、新加坡国立大学）；Department of Mathematics, Sichuan University（数学系、四川大学）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（高性能计算研究所（IHPC）、科学、技术与研究局（A*STAR））；Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore（创新与精准眼健康中心、 Yong Loo Lin 医学院、新加坡国立大学）；Singapore Eye Research Institute, Singapore National Eye Centre（新加坡眼研究所、新加坡国家眼科中心）
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；School of Transportation Science and Engineering, Beihang University（北京航空航天大学交通科学与工程学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；SenseTime Research（商汤科技研究院）；China University of Petroleum (East China)（中国石油大学（华东））；Peng Cheng Laboratory（鹏城实验室）；Case Western Reserve University（凯斯西储大学）
Although weakly-supervised techniques can reduce the labeling effort, it is unclear whether a saliency model trained with weakly-supervised data (e.g., point annotation) can achieve the equivalent performance of its fully-supervised version. This paper attempts to answer this unexplored question by proving a hypothesis: there is a point-labeled dataset where saliency models trained on it can achieve equivalent performance when trained on the densely annotated dataset. To prove this conjecture, we proposed a novel yet effective adversarial trajectory-ensemble active learning (ATAL). Our contrib…
Aiming at discovering and locating most distinctive objects from visual scenes, salient object detection (SOD) plays an essential role in various computer vision systems. Coming to the era of high resolution, SOD methods are facing new challenges. The major limitation of previous methods is that they try to identify the salient regions and estimate the accurate objects boundaries simultaneously with a single regression task at low-resolution. This practice ignores the inherent difference between the two difficult problems, resulting in poor detection quality. In this paper, we propose a novel …
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Leicester（莱斯特大学）
Due to the extreme complexity of scale and shape as well as the uncertainty of the predicted location, salient object detection in optical remote sensing images (RSI-SOD) is a very difficult task. The existing SOD methods can satisfy the detection performance for natural scene images, but they are not well adapted to RSI-SOD due to the above-mentioned image characteristics in remote sensing images. In this paper, we propose a novel Attention Guided Network (AGNet) for SOD in optical RSIs, including position enhancement stage and detail refinement stage. Specifically, the position enhancement s…
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Information Technology University（信息技术大学）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) causes coronavirus disease 2019 (COVID-19). Imaging tests such as chest X-ray (CXR) and computed tomography (CT) can provide useful information to clinical staff for facilitating a diagnosis of COVID-19 in a more efficient and comprehensive manner. As a breakthrough of artificial intelligence (AI), deep learning has been applied to perform COVID-19 infection region segmentation and disease classification by analyzing CXR and CT data. However, prediction uncertainty of deep learning models for these tasks, which is very important to s…
Pseudo-D: Informing Multi-View Uncertainty Estimation with Calibrated Neural Training Dynamics
Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, Canada（电气与计算机工程系，不列颠哥伦比亚大学，温哥华，加拿大）；Division of Cardiology, Vancouver General Hospital, Vancouver, Canada（心血管科，温哥华总医院，温哥华，加拿大）
University of Tehran（德黑兰大学）；Deakin University（迪肯大学）；University of Calgary（卡尔加里大学）
Uncertainty quantification of machine learning and deep learning methods plays an important role in enhancing trust to the obtained result. In recent years, a numerous number of uncertainty quantification methods have been introduced. Monte Carlo dropout (MC-Dropout) is one of the most well-known techniques to quantify uncertainty in deep learning methods. In this study, we propose two new loss functions by combining cross entropy with Expected Calibration Error (ECE) and Predictive Entropy (PE). The obtained results clearly show that the new proposed loss functions lead to having a calibrated…
Cornell University（康奈尔大学）；Cornell Tech（康奈尔理工学院）；Shutterstock（Shutterstock公司）；Rochester Institute of Technology（罗切斯特理工学院）
For many applications it is critical to know the uncertainty of a neural network's predictions. While a variety of neural network parameter estimation methods have been proposed for uncertainty estimation, they have not been rigorously compared across uncertainty measures. We assess four of these parameter estimation methods to calibrate uncertainty estimation using four different uncertainty measures: entropy, mutual information, aleatoric uncertainty and epistemic uncertainty. We evaluate the calibration of these parameter estimation methods using expected calibration error. Additionally, we…
Estimated uncertainty by approximate posteriors in Bayesian neural networks are prone to miscalibration, which leads to overconfident predictions in critical tasks that have a clear asymmetric cost or significant losses. Here, we extend the approximate inference for the loss-calibrated Bayesian framework to dropweights based Bayesian neural networks by maximising expected utility over a model posterior to calibrate uncertainty in deep learning. Furthermore, we show that decisions informed by loss-calibrated uncertainty can improve diagnostic performance to a greater extent than straightforward…
University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
High-quality calibrated uncertainty estimates are crucial for numerous real-world applications, especially for deep learning-based deployed ML systems. While Bayesian deep learning techniques allow uncertainty estimation, training them with large-scale datasets is an expensive process that does not always yield models competitive with non-Bayesian counterparts. Moreover, many of the high-performing deep learning models that are already trained and deployed are non-Bayesian in nature and do not provide uncertainty estimates. To address these issues, we propose BayesCap that learns a Bayesian id…
Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture
Shriners Children’s Lexington（夏皮尔儿童医院莱克星顿分院）；Dept. of Physical Therapy, University of Kentucky（肯塔基大学物理治疗系）；Shirley Ryan AbilityLab（希拉里·瑞安能力实验室）；Northwestern University（西北大学）；University of Göttingen（哥廷根大学）；Lower Saxony Center for AI & Causal Methods in Medicine（下萨克森人工智能与因果方法医学中心）；Shriners Children’s Philadelphia（夏皮尔儿童医院费城分院）
Technical University of Munich（慕尼黑工业大学）；OpenMined；Imperial College London（帝国理工学院）
We show that differentially private stochastic gradient descent (DP-SGD) can yield poorly calibrated, overconfident deep learning models. This represents a serious issue for safety-critical applications, e.g. in medical diagnosis. We highlight and exploit parallels between stochastic gradient Langevin dynamics, a scalable Bayesian inference technique for training deep neural networks, and DP-SGD, in order to train differentially private, Bayesian neural networks with minor adjustments to the original (DP-SGD) algorithm. Our approach provides considerably more reliable uncertainty estimates tha…
Equipping predicted segmentation with calibrated uncertainty is essential for safety-critical applications. In this work, we focus on capturing the data-inherent uncertainty (aka aleatoric uncertainty) in segmentation, typically when ambiguities exist in input images. Due to the high-dimensional output space and potential multiple modes in segmenting ambiguous images, it remains challenging to predict well-calibrated uncertainty for segmentation. To tackle this problem, we propose a novel mixture of stochastic experts (MoSE) model, where each expert network estimates a distinct mode of the ale…
Montreal Robotics and Embodied AI Lab（蒙特利尔机器人与具身人工智能实验室）；Mila（米拉人工智能研究所）；Université de Montréal（蒙特利尔大学）；Denso（电装公司）
While modern deep neural networks are performant perception modules, performance (accuracy) alone is insufficient, particularly for safety-critical robotic applications such as self-driving vehicles. Robot autonomy stacks also require these otherwise blackbox models to produce reliable and calibrated measures of confidence on their predictions. Existing approaches estimate uncertainty from these neural network perception stacks by modifying network architectures, inference procedure, or loss functions. However, in general, these methods lack calibration, meaning that the predictive uncertainti…
Uncertainty-Calibrated Explainable Artificial Intelligence for Fetal Ultrasound Plane Classification: A Systematic Review
Department of Life Sciences and Medicine, Faculty of Science, Technology and Medicine, University of Luxembourg（卢森堡大学生命科学与医学系，科学、技术与医学学院）；Department of Biostatistics and Medical Informatics, Institute of Health Sciences, Ege University（伊兹密尔大学健康科学学院生物统计学与医学信息学系）
Manipal Institute of Technology（马尼帕尔理工学院）；Manipal Academy of Higher Education（马尼帕尔高等教育学院）
Deep Ensemble Convolutional Neural Networks has become a methodology of choice for analyzing medical images with a diagnostic performance comparable to a physician, including the diagnosis of Diabetic Retinopathy. However, commonly used techniques are deterministic and are therefore unable to provide any estimate of predictive uncertainty. Quantifying model uncertainty is crucial for reducing the risk of misdiagnosis. A reliable architecture should be well-calibrated to avoid over-confident predictions. To address this, we propose a UATTA-ENS: Uncertainty-Aware Test-Time Augmented Ensemble Tec…
Hamburg University of Technology（汉堡工业大学）；Leibniz Universität Hannover（莱布尼茨汉诺威大学）；Hannover Medical School（汉诺威医学院）；The Hospital for Sick Children（多伦多病童医院）；University of Toronto Mississauga（多伦多大学密西沙加校区）
The consideration of predictive uncertainty in medical imaging with deep learning is of utmost importance. We apply estimation of both aleatoric and epistemic uncertainty by variational Bayesian inference with Monte Carlo dropout to regression tasks and show that predictive uncertainty is systematically underestimated. We apply $ σ$ scaling with a single scalar value; a simple, yet effective calibration method for both types of uncertainty. The performance of our approach is evaluated on a variety of common medical regression data sets using different state-of-the-art convolutional network arc…
End-to-end learners for autonomous driving are deep neural networks that predict the instantaneous steering angle directly from images of the ahead-lying street. These learners must provide reliable uncertainty estimates for their predictions in order to meet safety requirements and initiate a switch to manual control in areas of high uncertainty. Yet end-to-end learners typically only deliver point predictions, since distributional predictions are associated with large increases in training time or additional computational resources during prediction. To address this shortcoming we investigat…
