When designing a semantic segmentation module for a practical application, such as autonomous driving, it is crucial to understand the robustness of the module with respect to a wide range of image corruptions. While there are recent robustness studies for full-image classification, we are the first to present an exhaustive study for semantic segmentation, based on the state-of-the-art model DeepLabv3+. To increase the realism of our study, we utilize almost 400,000 images generated from Cityscapes, PASCAL VOC 2012, and ADE20K. Based on the benchmark study, we gain several new insights. Firstly, contrary to full-image classification, model robustness increases with model performance, in most cases. Secondly, some architecture properties affect robustness significantly, such as a Dense Prediction Cell, which was designed to maximize performance on clean data only.
Medical image segmentation is one of the most challenging tasks in medical image analysis and has been widely developed for many clinical applications. Most of the existing metrics have been first designed for natural images and then extended to medical images. While object surface plays an important role in medical segmentation and quantitative analysis i.e. analyze brain tumor surface, measure gray matter volume, most of the existing metrics are limited when it comes to analyzing the object surface, especially to tell about surface smoothness or roughness of a given volumetric object or to a…
Comparative Benchmarking of Failure Detection Methods in Medical Image Segmentation: Unveiling the Role of Confidence Aggregation
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Medical Faculty Heidelberg, Heidelberg University（海德堡大学海德堡医学院）；Helmholtz Imaging（亥姆霍兹成像）；Heidelberg University Hospital（海德堡大学医院）
Semantic segmentation is a scene understanding task at the heart of safety-critical applications where robustness to corrupted inputs is essential. Implicit Background Estimation (IBE) has demonstrated to be a promising technique to improve the robustness to out-of-distribution inputs for semantic segmentation models for little to no cost. In this paper, we provide analysis comparing the structures learned as a result of optimization objectives that use Softmax, IBE, and Sigmoid in order to improve understanding their relationship to robustness. As a result of this analysis, we propose combini…
Robustness Evaluation of a Foundation Segmentation Model Under Simulated Domain Shifts in Abdominal CT: Implications for Health Digital Twin Deployment
Healthcare Informatics, University of Illinois Springfield（医疗信息学，伊利诺伊大学斯普林菲尔德分校）
Benchmarking Deep Learning Models for Aerial LiDAR Point Cloud Semantic Segmentation under Real Acquisition Conditions: A Case Study in Navarre
Department of Statistics, Computer Science and Mathematics and Institute of Smart Cities (ISC), Public University of Navarre (UPNA)（统计、计算机科学与数学系和智能城市研究所（ISC），纳瓦拉公共大学（UPNA））；Tracasa Instrumental
SemSegBench & DetecBench: Benchmarking Reliability and Generalization Beyond Classification
Data and Web Science Group, University of Mannheim（曼海姆大学数据与网络科学小组）；Max-Planck-Institute for Informatics, Saarland Informatics Campus（马克斯·普朗克信息研究所，萨尔兰信息校园）
University of Alberta（阿尔伯塔大学）；Valeo Vision Systems（法雷奥视觉系统公司）；Cairo University（开罗大学）
Semantic segmentation benefits robotics related applications especially autonomous driving. Most of the research on semantic segmentation is only on increasing the accuracy of segmentation models with little attention to computationally efficient solutions. The few work conducted in this direction does not provide principled methods to evaluate the different design choices for segmentation. In this paper, we address this gap by presenting a real-time semantic segmentation benchmarking framework with a decoupled design for feature extraction and decoding methods. The framework is comprised of d…
While current approaches for neural network training often aim at improving performance, less focus is put on training methods aiming at robustness towards varying noise conditions or directed attacks by adversarial examples. In this paper, we propose to improve robustness by a multi-task training, which extends supervised semantic segmentation by a self-supervised monocular depth estimation on unlabeled videos. This additional task is only performed during training to improve the semantic segmentation model's robustness at test time under several input perturbations. Moreover, we even find th…
There are two critical sensors for 3D perception in autonomous driving, the camera and the LiDAR. The camera provides rich semantic information such as color, texture, and the LiDAR reflects the 3D shape and locations of surrounding objects. People discover that fusing these two modalities can significantly boost the performance of 3D perception models as each modality has complementary information to the other. However, we observe that current datasets are captured from expensive vehicles that are explicitly designed for data collection purposes, and cannot truly reflect the realistic data di…
Deep Neural Networks (DNNs) have demonstrated exceptional performance on most recognition tasks such as image classification and segmentation. However, they have also been shown to be vulnerable to adversarial examples. This phenomenon has recently attracted a lot of attention but it has not been extensively studied on multiple, large-scale datasets and structured prediction tasks such as semantic segmentation which often require more specialised networks with additional components such as CRFs, dilated convolutions, skip-connections and multiscale processing. In this paper, we present what to…
The robustness of object detection algorithms plays a prominent role in real-world applications, especially in uncontrolled environments due to distortions during image acquisition. It has been proven that the performance of object detection methods suffers from in-capture distortions. In this study, we present a performance evaluation framework for the state-of-the-art object detection methods using a dedicated dataset containing images with various distortions at different levels of severity. Furthermore, we propose an original strategy of image distortion generation applied to the MS-COCO d…
To evaluate the robustness of non-classifier models, we propose probabilistic local equivalence, based on the notion of randomized smoothing, as a way to quantitatively evaluate the robustness of an arbitrary function. In addition, to understand the effect of adversarial training on non-classifiers and to investigate the level of robustness that can be obtained without degrading performance on the training distribution, we apply Fast is Better than Free adversarial training together with the TRADES robust loss to the training of an instance segmentation network. In this direction, we were able…
University of Toronto（多伦多大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）；Western University（西部大学）
Deep artificial neural networks (DNNs) have moved to the forefront of medical image analysis due to their success in classification, segmentation, and detection challenges. A principal challenge in large-scale deployment of DNNs in neuroimage analysis is the potential for shifts in signal-to-noise ratio, contrast, resolution, and presence of artifacts from site to site due to variances in scanners and acquisition protocols. DNNs are famously susceptible to these distribution shifts in computer vision. Currently, there are no benchmarking platforms or frameworks to assess the robustness of new …
Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging
College of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）；Department of Naval, Electrical, Electronics and Telecommunications Engineering, University of Genoa（海军、电气、电子与电信工程部门，热那亚大学）
The Chinese University of Hong Kong（香港中文大学）；Southern University of Science and Technology（南方科技大学）
Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention, since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and fully autonomous inter-floor navigation is still not available. As the very first step of inter-floor navigation, elevator button segmentation and recognition hold an important position. Therefore, we release the first large-scale publicly available elevator panel dataset in this work, containing 3,718…
Egocentric segmentation has attracted recent interest in the computer vision community due to their potential in Mixed Reality (MR) applications. While most previous works have been focused on segmenting egocentric human body parts (mainly hands), little attention has been given to egocentric objects. Due to the lack of datasets of pixel-wise annotations of egocentric objects, in this paper we contribute with a semantic-wise labeling of a subset of 2124 images from the RGB-D THU-READ Dataset. We also report benchmarking results using Thundernet, a real-time semantic segmentation network, that …
Institute of Imaging and Computer Vision, RWTH Aachen University（亚琛工业大学成像与计算机视觉研究所）
Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotatio…
Open-Source Manually Annotated Vocal Tract Database for Automatic Segmentation from 3D MRI Using Deep Learning: Benchmarking 2D and 3D Convolutional and Transformer Networks
University of Iowa（爱荷华大学）；Shenandoah University（谢南多厄大学）
Benchmarking Retinal Blood Vessel Segmentation Models for Cross-Dataset and Cross-Disease Generalization
University of Ibadan（伊巴丹大学）；African Institute for Mathematical Sciences（非洲数学科学研究所）；Hertie Institute for AI in Brain Health（赫蒂脑健康人工智能研究所）；University of Tübingen（蒂宾根大学）；University of Bern（伯尔尼大学）；University College London（伦敦大学学院）；Tübingen AI Center（蒂宾根人工智能中心）
Polyp-E: Benchmarking the Robustness of Deep Segmentation Models via Polyp Editing
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；The Fifth Medical Center, Chinese PLA General Hospital（中国人民解放军总医院第五医学中心）；Children’s Hospital of Zhejiang University School of Medicine（浙江大学医学院附属儿童医院）
More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgery
The Chinese University of Hong Kong（香港中文大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Technical University of Munich（慕尼黑技术大学）
Glioma C6: A Novel Dataset for Training and Benchmarking Cell Segmentation
Pavlov Institute of Physiology, Russian academy of science（俄罗斯科学院帕夫洛夫生理研究所）；Saint-Petersburg State University of Aerospace Instrumentation（圣彼得堡航空航天仪器州大学）；Institute of Physiology, NAS of Belarus（白俄罗斯国家科学院生理研究所）
Federal University of Bahia（巴伊亚联邦大学）；Southeast State University of Bahia（巴伊亚东南州立大学）
This review presents an in-depth study of the literature on segmentation methods applied in dental imaging. Ten segmentation methods were studied and categorized according to the type of the segmentation method (region-based, threshold-based, cluster-based, boundary-based or watershed-based), type of X-ray images used (intra-oral or extra-oral) and characteristics of the dataset used to evaluate the methods in the state-of-the-art works. We found that the literature has primarily focused on threshold-based segmentation methods (54%). 80% of the reviewed papers have used intra-oral X-ray images…
A Multicentric Dataset for Training and Benchmarking Breast Cancer Segmentation in H&E Slides
Department of Pathology, Radboud University Medical Center（病理学系，拉德堡德大学医学中心）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；Department of Cellular, Computational and Integrative Biology, University of Trento（细胞、计算与整合生物学系，特伦托大学）；Biopticka Laboratory Ltd.（Biopticka实验室有限公司）；Department of Pathology, Faculty of Medicine in Pilsen, Charles University（病理学系，皮尔森医学院，查尔斯大学）；University of Washington Medical Center（华盛顿大学医学中心）；Department of Surgical Pathology, Santa Chiara Hospital, APSS（外科病理学系，圣查理医院，APSS）；CISMed - Centre for Medical Sciences, University of Trento（CISMed医学研究中心，特伦托大学）；Pathology Unit, University and Hospital Trust of Verona（病理学部，韦罗纳大学与医院信托）；IRCCS Sacro Cuore Don Calabria Hospital（IRCCS圣心多玛利亚医院）
Despite advances in deep learning, robustness under domain shift remains a major bottleneck in medical imaging settings. Findings on natural images suggest that deep neural models can show a strong textural bias when carrying out image classification tasks. In this thorough empirical study, we draw inspiration from findings on natural images and investigate ways in which addressing the textural bias phenomenon could bring up the robustness of deep segmentation models when applied to three-dimensional (3D) medical data. To achieve this, publicly available MRI scans from the Developing Human Con…
