Quickshift is a popular algorithm for image segmentation, used as a preprocessing step in many applications. Unfortunately, it is quite challenging to understand the hyperparameters' influence on the number and shape of superpixels produced by the method. In this paper, we study theoretically a slightly modified version of the quickshift algorithm, with a particular emphasis on homogeneous image patches with i.i.d. pixel noise and sharp boundaries between such patches. Leveraging this analysis, we derive a simple heuristic to scale quickshift hyperparameters with respect to the image size, which we check empirically.
The extent to which the arbitrarily selected L2 regularization hyperparameter value affects the outcome of semantic segmentation with deep learning is demonstrated. Demonstrations rely on training U-net on small LGE-MRI datasets using the arbitrarily selected L2 regularization values. The remaining hyperparameters are to be manually adjusted or tuned only when 10 % of all epochs are reached before the training validation accuracy reaches 90%. Semantic segmentation with deep learning outcomes are objectively and subjectively evaluated against the manual ground truth segmentation.
Johns Hopkins University（约翰斯·霍普金斯大学）；NVIDIA（英伟达）；University of Pittsburgh（匹兹堡大学）；National Cancer Institute（美国国家癌症研究所）；National Institutes of Health（美国国立卫生研究院）；ASST Santi Paolo e Carlo（ASST圣保禄和圣卡洛医院）；University of Milan（米兰大学）
Federated learning (FL) is a distributed machine learning technique that enables collaborative model training while avoiding explicit data sharing. The inherent privacy-preserving property of FL algorithms makes them especially attractive to the medical field. However, in case of heterogeneous client data distributions, standard FL methods are unstable and require intensive hyperparameter tuning to achieve optimal performance. Conventional hyperparameter optimization algorithms are impractical in real-world FL applications as they involve numerous training trials, which are often not affordabl…
This paper presents a Bayesian image segmentation model based on Potts prior and loopy belief propagation. The proposed Bayesian model involves several terms, including the pairwise interactions of Potts models, and the average vectors and covariant matrices of Gauss distributions in color image modeling. These terms are often referred to as hyperparameters in statistical machine learning theory. In order to determine these hyperparameters, we propose a new scheme for hyperparameter estimation based on conditional maximization of entropy in the Potts prior. The algorithm is given based on loop…
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep network architecture automatically. We specifically focus on designing neural architectures for medical image segmentation task. Our proposed method is based on a policy gradient reinforcement learning for which the reward function is assigned a segmentation evaluation utility (i.…
Toyota Technological Institute at Chicago（丰田工业大学芝加哥分校）
We propose an approach for learning category-level semantic segmentation purely from image-level classification tags indicating presence of categories. It exploits localization cues that emerge from training classification-tasked convolutional networks, to drive a "self-supervision" process that automatically labels a sparse, diverse training set of points likely to belong to classes of interest. Our approach has almost no hyperparameters, is modular, and allows for very fast training of segmentation in less than 3 minutes. It obtains competitive results on the VOC 2012 segmentation benchmark.…
IBM Research – Almaden Research Center（IBM研究院——阿尔马登研究中心）
Deep learning has largely reduced the need for manual feature selection in image segmentation. Nevertheless, network architecture optimization and hyperparameter tuning are mostly manual and time consuming. Although there are increasing research efforts on network architecture search in computer vision, most works concentrate on image classification but not segmentation, and there are very limited efforts on medical image segmentation especially in 3D. To remedy this, here we propose a framework, SegNAS3D, for network architecture search of 3D image segmentation. In this framework, a network a…
Auto-nnU-Net: Towards Automated Medical Image Segmentation
Institute of AI, Leibniz University Hannover（莱比锡大学人工智能研究所）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；Peter L. Reichertz Institute for Medical Informatics, Hannover Medical School（汉诺威医学院彼得·L·里赫特茨医学信息学研究所）；CAIMed: Lower Saxony Center for AI & Causal Methods in Medicine（下萨克森人工智能与因果方法医学中心）；L3S Research Center（L3S研究中心）
High-fidelity semantic segmentation of magnetic resonance volumes is critical for estimating tissue morphometry and relaxation parameters in both clinical and research applications. While manual segmentation is accepted as the gold-standard, recent advances in deep learning and convolutional neural networks (CNNs) have shown promise for efficient automatic segmentation of soft tissues. However, due to the stochastic nature of deep learning and the multitude of hyperparameters in training networks, predicting network behavior is challenging. In this paper, we quantify the impact of three factor…
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；MIT（麻省理工学院）
Obtaining precise instance segmentation masks is of high importance in many modern applications such as robotic manipulation and autonomous driving. Currently, many state of the art models are based on the Mask R-CNN framework which, while very powerful, outputs masks at low resolutions which could result in imprecise boundaries. On the other hand, classic variational methods for segmentation impose desirable global and local data and geometry constraints on the masks by optimizing an energy functional. While mathematically elegant, their direct dependence on good initialization, non-robust im…
Erasmus MC（伊拉斯姆斯医学中心）；University of Copenhagen（哥本哈根大学）
We propose a novel semi-supervised image segmentation method that simultaneously optimizes a supervised segmentation and an unsupervised reconstruction objectives. The reconstruction objective uses an attention mechanism that separates the reconstruction of image areas corresponding to different classes. The proposed approach was evaluated on two applications: brain tumor and white matter hyperintensities segmentation. Our method, trained on unlabeled and a small number of labeled images, outperformed supervised CNNs trained with the same number of images and CNNs pre-trained on unlabeled data…
MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation
Faculty of Electrical Engineering and Computing, University of Zagreb（萨格勒布大学电气工程与计算学院）；Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）；Fundamental AI Lab, University of Technology Nuremberg（纽伦堡工业大学基础人工智能实验室）
Challenging computer vision tasks, in particular semantic image segmentation, require large training sets of annotated images. While obtaining the actual images is often unproblematic, creating the necessary annotation is a tedious and costly process. Therefore, one often has to work with unreliable annotation sources, such as Amazon Mechanical Turk or (semi-)automatic algorithmic techniques. In this work, we present a Gaussian process (GP) based technique for simultaneously identifying which images of a training set have unreliable annotation and learning a segmentation model in which the neg…
This report describes the winning solution to the Robust Vision Challenge (RVC) semantic segmentation track at ECCV 2022. Our method adopts the FAN-B-Hybrid model as the encoder and uses SegFormer as the segmentation framework. The model is trained on a composite dataset consisting of images from 9 datasets (ADE20K, Cityscapes, Mapillary Vistas, ScanNet, VIPER, WildDash 2, IDD, BDD, and COCO) with a simple dataset balancing strategy. All the original labels are projected to a 256-class unified label space, and the model is trained using a cross-entropy loss. Without significant hyperparameter …
Current 3D instance segmentation models generally use multi-stage methods to extract instance objects, including clustering, feature extraction, and post-processing processes. However, these multi-stage approaches rely on hyperparameter settings and hand-crafted processes, which restrict the inference speed of the model. In this paper, we propose a new 3D point cloud instance segmentation network, named OSIS. OSIS is a one-stage network, which directly segments instances from 3D point cloud data using neural network. To segment instances directly from the network, we propose an instance decode…
Innersight Labs（英纳赛特实验室）；Royal Free Hospital（皇家自由医院）；Kings College London（伦敦国王学院）
Training deep neural networks on large and sparse datasets is still challenging and can require large amounts of computation and memory. In this work, we address the task of performing semantic segmentation on large volumetric data sets, such as CT scans. Our contribution is threefold: 1) We propose a boosted sampling scheme that uses a-posterior error maps, generated throughout training, to focus sampling on difficult regions, resulting in a more informative loss. This results in a significant training speed up and improves learning performance for image segmentation. 2) We propose a novel al…
We extend first-order model agnostic meta-learning algorithms (including FOMAML and Reptile) to image segmentation, present a novel neural network architecture built for fast learning which we call EfficientLab, and leverage a formal definition of the test error of meta-learning algorithms to decrease error on out of distribution tasks. We show state of the art results on the FSS-1000 dataset by meta-training EfficientLab with FOMAML and using Bayesian optimization to infer the optimal test-time adaptation routine hyperparameters. We also construct a small benchmark dataset, FP-k, for the empi…
MSA2-Net: Utilizing Self-Adaptive Convolution Module to Extract Multi-Scale Information in Medical Image Segmentation
School of Artificial Intelligence, Nanning Normal University, Nanning, Guangxi, 530100, People's Republic of China（南宁师范学院人工智能学院）；School of Artificial Intelligence, Guangxi Minzu University, Nanning, Guangxi, 530300, People’s Republic of China（广西民族大学人工智能学院）
University of British Columbia（不列颠哥伦比亚大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）
Semi-weakly supervised semantic segmentation (SWSSS) aims to train a model to identify objects in images based on a small number of images with pixel-level labels, and many more images with only image-level labels. Most existing SWSSS algorithms extract pixel-level pseudo-labels from an image classifier - a very difficult task to do well, hence requiring complicated architectures and extensive hyperparameter tuning on fully-supervised validation sets. We propose a method called prediction filtering, which instead of extracting pseudo-labels, just uses the classifier as a classifier: it ignores…
University of Toulouse（图卢兹大学）；IRIT/INP-ENSEEIHT（图卢兹国立综合理工学院国立高等电子技术、电子学、计算机、水力学院/图卢兹信息学研究所）；IRIT, CNRS UMR 5505（法国国家科学研究中心联合研究单位5505 图卢兹信息学研究所）
This paper proposes a joint segmentation and deconvolution Bayesian method for medical ultrasound (US) images. Contrary to piecewise homogeneous images, US images exhibit heavy characteristic speckle patterns correlated with the tissue structures. The generalized Gaussian distribution (GGD) has been shown to be one of the most relevant distributions for characterizing the speckle in US images. Thus, we propose a GGD-Potts model defined by a label map coupling US image segmentation and deconvolution. The Bayesian estimators of the unknown model parameters, including the US image, the label map …
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
Although deep convolutional neural networks(CNNs) have achieved remarkable results on object detection and segmentation, pre- and post-processing steps such as region proposals and non-maximum suppression(NMS), have been required. These steps result in high computational complexity and sensitivity to hyperparameters, e.g. thresholds for NMS. In this work, we propose a novel end-to-end trainable deep neural network architecture, which consists of convolutional and recurrent layers, that generates the correct number of object instances and their bounding boxes (or segmentation masks) given an im…
Sun Yat-Sen University（中山大学）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Shanghai Jiao Tong University（上海交通大学）
Panoptic segmentation that unifies instance segmentation and semantic segmentation has recently attracted increasing attention. While most existing methods focus on designing novel architectures, we steer toward a different perspective: performing automated multi-loss adaptation (named Ada-Segment) on the fly to flexibly adjust multiple training losses over the course of training using a controller trained to capture the learning dynamics. This offers a few advantages: it bypasses manual tuning of the sensitive loss combination, a decisive factor for panoptic segmentation; it allows to explici…
We present SHRED, a method for 3D SHape REgion Decomposition. SHRED takes a 3D point cloud as input and uses learned local operations to produce a segmentation that approximates fine-grained part instances. We endow SHRED with three decomposition operations: splitting regions, fixing the boundaries between regions, and merging regions together. Modules are trained independently and locally, allowing SHRED to generate high-quality segmentations for categories not seen during training. We train and evaluate SHRED with fine-grained segmentations from PartNet; using its merge-threshold hyperparame…
Friedrich Miescher Institute for Biomedical Research (FMI)（弗里德里希·米歇勒生物医学研究所（FMI））；Faculty of Sciences, University of Basel（巴塞尔大学理学院）；University of Basel（巴塞尔大学）
Instance segmentation is a key step for quantitative microscopy. While several machine learning based methods have been proposed for this problem, most of them rely on computationally complex models that are trained on surrogate tasks. Building on recent developments towards end-to-end trainable instance segmentation, we propose a minimalist recurrent network called recurrent dilated convolutional network (RDCNet), consisting of a shared stacked dilated convolution (sSDC) layer that iteratively refines its output and thereby generates interpretable intermediate predictions. It is light-weight …
University of Freiburg（弗莱堡大学）；Fraunhofer Institute for Physical Measurement Techniques IPM（弗劳恩霍夫物理测量技术研究所）
Data augmentation methods enrich datasets with augmented data to improve the performance of neural networks. Recently, automated data augmentation methods have emerged, which automatically design augmentation strategies. Existing work focuses on image classification and object detection, whereas we provide the first study on semantic image segmentation and introduce two new approaches: \textit{SmartAugment} and \textit{SmartSamplingAugment}. SmartAugment uses Bayesian Optimization to search over a rich space of augmentation strategies and achieves a new state-of-the-art performance in all sema…
University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Air Force Research Laboratory（空军研究实验室）
Superpixel-based Higher-order Conditional Random Fields (CRFs) are effective in enforcing long-range consistency in pixel-wise labeling problems, such as semantic segmentation. However, their major short coming is considerably longer time to learn higher-order potentials and extra hyperparameters and/or weights compared with pairwise models. This paper proposes a superpixel-enhanced pairwise CRF framework that consists of the conventional pairwise as well as our proposed superpixel-enhanced pairwise (SP-Pairwise) potentials. SP-Pairwise potentials incorporate the superpixel-based higher-order …
A Labeled Array Distance Metric for Measuring Image Segmentation Quality
SEE-Insight Research Lab, Michigan State University, East Lansing, MI, USA（密歇根州立大学SEE-Insight研究实验室）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Smith College（史密斯学院）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Cornell University（康奈尔大学）
We present a new framework for semantic segmentation without annotations via clustering. Off-the-shelf clustering methods are limited to curated, single-label, and object-centric images yet real-world data are dominantly uncurated, multi-label, and scene-centric. We extend clustering from images to pixels and assign separate cluster membership to different instances within each image. However, solely relying on pixel-wise feature similarity fails to learn high-level semantic concepts and overfits to low-level visual cues. We propose a method to incorporate geometric consistency as an inductive…
Politecnico di Milano（米兰理工大学）；Montreal Institute for Learning Algorithms (MILA)（蒙特利尔学习算法研究所（MILA））；University of Montreal（蒙特利尔大学）；Center for Data Science, New York University（纽约大学数据科学中心）；New York University（纽约大学）
We propose a structured prediction architecture, which exploits the local generic features extracted by Convolutional Neural Networks and the capacity of Recurrent Neural Networks (RNN) to retrieve distant dependencies. The proposed architecture, called ReSeg, is based on the recently introduced ReNet model for image classification. We modify and extend it to perform the more challenging task of semantic segmentation. Each ReNet layer is composed of four RNN that sweep the image horizontally and vertically in both directions, encoding patches or activations, and providing relevant global infor…
DyMorph-B2I: Dynamic and Morphology-Guided Binary-to-Instance Segmentation for Renal Pathology
Southern University of Science and Technology（南方科技大学）；Vanderbilt University（范德比尔特大学）；Vanderbilt University Medical Center（范德比尔特大学医学中心）；Weill Cornell Medicine（韦氏 Cornell 医学院）；Cornell Tech（康奈尔科技）；Northwell Health（北well 健康）
Deep Convolutional Neural Networks Meet Variational Shape Compactness Priors for Image Segmentation
Hong Kong Baptist University（香港浸会大学）；Hong Kong Center for Cerebro-Cardiovascular Health Engineering（香港脑心健康工程中心）；Zhejiang Normal University（浙江师范大学）；Norwegian Research Center（挪威研究中心）
Video object segmentation (VOS) describes the task of segmenting a set of objects in each frame of a video. In the semi-supervised setting, the first mask of each object is provided at test time. Following the one-shot principle, fine-tuning VOS methods train a segmentation model separately on each given object mask. However, recently the VOS community has deemed such a test time optimization and its impact on the test runtime as unfeasible. To mitigate the inefficiencies of previous fine-tuning approaches, we present efficient One-Shot Video Object Segmentation (e-OSVOS). In contrast to most …
IBM Research – Almaden Research Center（IBM研究院–阿尔马登研究中心）
The convolutional layer and loss function are two fundamental components in deep learning. Because of the success of conventional deep learning kernels, the less versatile Gabor kernels become less popular despite the fact that they can provide abundant features at different frequencies, orientations, and scales with much fewer parameters. For existing loss functions for multi-class image segmentation, there is usually a tradeoff among accuracy, robustness to hyperparameters, and manual weight selections for combining different losses. Therefore, to gain the benefits of using Gabor kernels whi…
STARS: Shared-specific Translation and Alignment for missing-modality Remote Sensing Semantic Segmentation
State Key Laboratory of information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（测绘遥感信息工程国家重点实验室，武汉大学）；Electronic Information School, Wuhan University（电子信息学院，武汉大学）；Hubei FreerTech Co. Ltd（湖北弗瑞特科技有限公司）
Harmonizing output imbalance for defect segmentation on extremely-imbalanced photovoltaic module cells images
Shenzhen University（深圳大学）；Guangdong Artificial Intelligence and Digital Economy Laboratory (Shenzhen)（广东人工智能与数字经济实验室（深圳））；Shenzhen Institute of Technology（深圳技术学院）
