In this paper, we propose PASS3D to achieve point-wise semantic segmentation for 3D point cloud. Our framework combines the efficiency of traditional geometric methods with robustness of deep learning methods, consisting of two stages: At stage-1, our accelerated cluster proposal algorithm will generate refined cluster proposals by segmenting point clouds without ground, capable of generating less redundant proposals with higher recall in an extremely short time; stage-2 we will amplify and further process these proposals by a neural network to estimate semantic label for each point and meanwhile propose a novel data augmentation method to enhance the network's recognition capability for all categories especially for non-rigid objects. Evaluated on KITTI raw dataset, PASS3D stands out against the state-of-the-art on some results, making itself competent to 3D perception in autonomous driving system. Our source code will be open-sourced. A video demonstration is available at https://www.youtube.com/watch?v=cukEqDuP_Qw.
Accelerated Rotation-Invariant Convolution for UAV Image Segmentation
Drone Systems Lab, School of Computing, Engineering, and Physical Sciences, University of the West of Scotland（无人机系统实验室，计算、工程与物理科学学院，西方苏格兰大学）；Centre for Robotics & Intelligent Systems, Department of Electronic & Computer Engineering, University of Limerick（机器人与智能系统中心，电子与计算机工程系，利默里克大学）
An Accelerated Pipeline for Multi-label Renal Pathology Image Segmentation at the Whole Slide Image Level
Vanderbilt University（范德堡大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Vanderbilt University Medical Center（范德堡大学医学中心）；Georgia State University（佐治亚州立大学）
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Temporal action detection (TAD) is an important yet challenging task in video analysis. Most existing works draw inspiration from image object detection and tend to reformulate it as a proposal generation - classification problem. However, there are two caveats with this paradigm. First, proposals are not equipped with annotated labels, which have to be empirically compiled, thus the information in the annotations is not necessarily precisely employed in the model training process. Second, there are large variations in the temporal scale of actions, and neglecting this fact may lead to deficie…
Progressive Growing of Patch Size: Curriculum Learning for Accelerated and Improved Medical Image Segmentation
School of Computation, Information and Technology, Technical University Munich（技术大学慕尼黑计算、信息与技术学院）；Department of Radiation Oncology, TUM School of Medicine, TUM University Hospital rechts der Isar, Technical University of Munich（肿瘤放射治疗科，TUM医学院，TUM大学医院rechts der Isar，技术大学慕尼黑）；Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich（生物医学影像中的机器学习研究所，海德堡慕尼黑）；Institute of Radiation Medicine, Helmholtz Center Munich（放射医学研究所，海德堡慕尼黑）；Munich Center of Machine Learning (MCML)（慕尼黑机器学习中心）；School of Biomedical Engineering and Imaging Sciences, King’s College London（生物医学工程与成像科学学院，伦敦国王学院）；Departament de Matemàtiques i Informàtica, Barcelona Artificial Intelligence in Medicine Lab (BCN-AIM), Universitat de Barcelona（数学与计算机科学系，巴塞罗那人工智能在医学实验室（BCN-AIM），巴塞罗那大学）
In this paper, we focus on improving binary 2D instance segmentation to assist humans in labeling ground truth datasets with polygons. Humans labeler just have to draw boxes around objects, and polygons are generated automatically. To be useful, our system has to run on CPUs in real-time. The most usual approach for binary instance segmentation involves encoder-decoder networks. This report evaluates state-of-the-art encoder-decoder networks and proposes a method for improving instance segmentation quality using these networks. Alongside network architecture improvements, our proposed method r…
BEFUnet: A Hybrid CNN-Transformer Architecture for Precise Medical Image Segmentation
School of Electrical Engineering, Iran University of Science and Technology（伊朗科学技术大学电气工程学院）；University College of Nabi Akram（纳比·阿克拉姆大学学院）；Department of Electrical Engineering, Qazvin Branch, Iran（伊朗加兹温分校电气工程系）
Class-agnostic object proposal generation is an important first step in many object detection pipelines. However, object proposals of modern systems are rather inaccurate in terms of segmentation and only roughly adhere to object boundaries. Since typical refinement steps are usually not applicable to thousands of proposals, we propose a superpixel-based refinement system for object proposal generation systems. Utilizing precise superpixels and superpixel pooling on deep features, we refine initial coarse proposals in an end-to-end learned system. Furthermore, we propose a novel DeepFH segment…
We consider the problem of localizing and segmenting individual teeth inside 3D Cone-Beam Computed Tomography (CBCT) images. To handle large image sizes we approach this task with a coarse-to-fine framework, where the whole volume is first analyzed as a 33-class semantic segmentation (adults have up to 32 teeth) in coarse resolution, followed by binary semantic segmentation of the cropped region of interest in original resolution. To improve the performance of the challenging 33-class segmentation, we first train the Coarse step model on a large weakly labeled dataset, then fine-tune it on a s…
Huazhong University of Science and Technology（华中科技大学）；VIVO Inc.（维沃移动通信有限公司）
Humans have a strong class-agnostic object segmentation ability and can outline boundaries of unknown objects precisely, which motivates us to propose a box-supervised class-agnostic object segmentation (BoxCaseg) based solution for weakly-supervised instance segmentation. The BoxCaseg model is jointly trained using box-supervised images and salient images in a multi-task learning manner. The fine-annotated salient images provide class-agnostic and precise object localization guidance for box-supervised images. The object masks predicted by a pretrained BoxCaseg model are refined via a novel m…
Lobachevsky State University（罗巴切夫斯基国立大学）；Intel（英特尔）
MRI analysis takes central position in brain tumor diagnosis and treatment, thus it's precise evaluation is crucially important. However, it's 3D nature imposes several challenges, so the analysis is often performed on 2D projections that reduces the complexity, but increases bias. On the other hand, time consuming 3D evaluation, like, segmentation, is able to provide precise estimation of a number of valuable spatial characteristics, giving us understanding about the course of the disease.\newline Recent studies, focusing on the segmentation task, report superior performance of Deep Learning …
Semantic segmentation of road scenes is one of the key technologies for realizing autonomous driving scene perception, and the effectiveness of deep Convolutional Neural Networks(CNNs) for this task has been demonstrated. State-of-art CNNs for semantic segmentation suffer from excessive computations as well as large-scale training data requirement. Inspired by the ideas of Fine-tuning-based Transfer Learning (FTT) and feature-based knowledge distillation, we propose a new knowledge distillation method for cross-domain knowledge transference and efficient data-insufficient network training, nam…
Large Language Model Evaluated Stand-alone Attention-Assisted Graph Neural Network with Spatial and Structural Information Interaction for Precise Endoscopic Image Segmentation
Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）；Department of Electronic and Computer Engineering, University of Massachusetts, Lowell（马萨诸塞大学低罗分校电子与计算机工程系）；Department of Radiology, Northwestern University（西北大学放射科）
The segmentation of synthetic aperture radar (SAR) images is a longstanding yet challenging task, not only because of the presence of speckle, but also due to the variations of surface backscattering properties in the images. Tremendous investigations have been made to eliminate the speckle effects for the segmentation of SAR images, while few work devotes to dealing with the variations of backscattering coefficients in the images. In order to overcome both the two difficulties, this paper presents a novel SAR image segmentation method by exploiting a multi-scale active contour model based on …
University of Ostrava（奥斯特拉发大学）；Varroc Lighting Systems（瓦罗克照明系统公司）
We present a new version of YOLO with better performance and extended with instance segmentation called Poly-YOLO. Poly-YOLO builds on the original ideas of YOLOv3 and removes two of its weaknesses: a large amount of rewritten labels and inefficient distribution of anchors. Poly-YOLO reduces the issues by aggregating features from a light SE-Darknet-53 backbone with a hypercolumn technique, using stairstep upsampling, and produces a single scale output with high resolution. In comparison with YOLOv3, Poly-YOLO has only 60% of its trainable parameters but improves mAP by a relative 40%. We also…
Fully convolutional networks (FCNs), including UNet and VNet, are widely-used network architectures for semantic segmentation in recent studies. However, conventional FCN is typically trained by the cross-entropy or Dice loss, which only calculates the error between predictions and ground-truth labels for pixels individually. This often results in non-smooth neighborhoods in the predicted segmentation. To address this problem, we propose a two-stage framework, with the first stage to quickly localize the prostate region and the second stage to precisely segment the prostate by a multi-task UNe…
The key to a successful cascade architecture for precise instance segmentation is to fully leverage the relationship between bounding box detection and mask segmentation across multiple stages. Although modern instance segmentation cascades achieve leading performance, they mainly make use of a unidirectional relationship, i.e., mask segmentation can benefit from iteratively refined bounding box detection. In this paper, we investigate an alternative direction, i.e., how to take the advantage of precise mask segmentation for bounding box detection in a cascade architecture. We propose a Deeply…
Boundary feature fusion network for tooth image segmentation
College of Information Engineering, China Jiliang University（中国计量大学信息工程学院）；College of Optoelectronics, China Jiliang University（中国计量大学光电学院）；College of Computer and Control Engineering, Qiqihar University（齐齐哈尔大学计算机与控制工程学院）
针对牙齿与相邻组织边界模糊问题，本文提出融合边界特征提取与特征交叉融合模块的牙齿分割网络，在STS Data Challenge中获得0.91分，显著优于现有方法。
Mask Factory: Towards High-quality Synthetic Data Generation for Dichotomous Image Segmentation
State Key Lab of CAD&CG, Zhejiang University（浙江大学CAD&CG国家重点实验室）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；Linköping University（林雪平大学）；VCIP&CS, Nankai University（南开大学VCIP&CS）
Fully convolutional neural networks give accurate, per-pixel prediction for input images and have applications like semantic segmentation. However, a typical FCN usually requires lots of floating point computation and large run-time memory, which effectively limits its usability. We propose a method to train Bit Fully Convolution Network (BFCN), a fully convolutional neural network that has low bit-width weights and activations. Because most of its computation-intensive convolutions are accomplished between low bit-width numbers, a BFCN can be accelerated by an efficient bit-convolution implem…
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Weakly supervised object detection aims at learning precise object detectors, given image category labels. In recent prevailing works, this problem is generally formulated as a multiple instance learning module guided by an image classification loss. The object bounding box is assumed to be the one contributing most to the classification among all proposals. However, the region contributing most is also likely to be a crucial part or the supporting context of an object. To obtain a more accurate detector, in this work we propose a novel end-to-end weakly supervised detection approach, where a …
Visual Computing Institute RWTH Aachen University（亚琛工业大学视觉计算研究所）
Semantic image segmentation is an essential component of modern autonomous driving systems, as an accurate understanding of the surrounding scene is crucial to navigation and action planning. Current state-of-the-art approaches in semantic image segmentation rely on pre-trained networks that were initially developed for classifying images as a whole. While these networks exhibit outstanding recognition performance (i.e., what is visible?), they lack localization accuracy (i.e., where precisely is something located?). Therefore, additional processing steps have to be performed in order to obtai…
In this paper, we propose a novel medical image segmentation using iterative deep learning framework. We have combined an iterative learning approach and an encoder-decoder network to improve segmentation results, which enables to precisely localize the regions of interest (ROIs) including complex shapes or detailed textures of medical images in an iterative manner. The proposed iterative deep convolutional encoder-decoder network consists of two main paths: convolutional encoder path and convolutional decoder path with iterative learning. Experimental results show that the proposed iterative …
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；United Imaging Intelligence（联影智能）
The goal of click-based interactive image segmentation is to obtain precise object segmentation masks with limited user interaction, i.e., by a minimal number of user clicks. Existing methods require users to provide all the clicks: by first inspecting the segmentation mask and then providing points on mislabeled regions, iteratively. We ask the question: can our model directly predict where to click, so as to further reduce the user interaction cost? To this end, we propose {\PseudoClick}, a generic framework that enables existing segmentation networks to propose candidate next clicks. These …
This paper proposes a Genetic Algorithm based segmentation method that can automatically segment gray-scale images. The proposed method mainly consists of spatial unsupervised grayscale image segmentation that divides an image into regions. The aim of this algorithm is to produce precise segmentation of images using intensity information along with neighborhood relationships. In this paper, Fuzzy Hopfield Neural Network (FHNN) clustering helps in generating the population of Genetic algorithm which there by automatically segments the image. This technique is a powerful method for image segment…
The way features propagate in Fully Convolutional Networks is of momentous importance to capture multi-scale contexts for obtaining precise segmentation masks. This paper proposes a novel series-parallel hybrid paradigm called the Chained Context Aggregation Module (CAM) to diversify feature propagation. CAM gains features of various spatial scales through chain-connected ladder-style information flows and fuses them in a two-stage process, namely pre-fusion and re-fusion. The serial flow continuously increases receptive fields of output neurons and those in parallel encode different region-ba…
Uber Advanced Technologies Group（优步先进技术集团）；University of Toronto（多伦多大学）；MIT（麻省理工学院）
In this paper, we propose PolyTransform, a novel instance segmentation algorithm that produces precise, geometry-preserving masks by combining the strengths of prevailing segmentation approaches and modern polygon-based methods. In particular, we first exploit a segmentation network to generate instance masks. We then convert the masks into a set of polygons that are then fed to a deforming network that transforms the polygons such that they better fit the object boundaries. Our experiments on the challenging Cityscapes dataset show that our PolyTransform significantly improves the performance…
Advancing Marine Research: UWSAM Framework and UIIS10K Dataset for Precise Underwater Instance Segmentation
School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；School of Computer Science and Artificial Intelligence, Zhengzhou University（郑州大学计算机科学与人工智能学院）；Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）
针对 SAM 在水下实例分割中的领域适配与计算开销问题，本文构建 UIIS10K 数据集，并提出结合 MG-UKD 与 EUPG 的高效 UWSAM，在多个数据集上显著优于现有方法。
Universidad de Sevilla（塞维利亚大学）；Universidad de Cádiz（加的斯大学）；School of Engineering, Universidad de Cádiz（加的斯大学工程学院）
Medical image segmentation can be implemented using Deep Learning methods with fast and efficient segmentation networks. Single-board computers (SBCs) are difficult to use to train deep networks due to their memory and processing limitations. Specific hardware such as Google's Edge TPU makes them suitable for real time predictions using complex pre-trained networks. In this work, we study the performance of two SBCs, with and without hardware acceleration for fundus image segmentation, though the conclusions of this study can be applied to the segmentation by deep neural networks of other type…
Silesian University of Technology（西里西亚工业大学）；KP Labs（KP实验室）
Hyperspectral satellite imaging attracts enormous research attention in the remote sensing community, hence automated approaches for precise segmentation of such imagery are being rapidly developed. In this letter, we share our observations on the strategy for validating hyperspectral image segmentation algorithms currently followed in the literature, and show that it can lead to over-optimistic experimental insights. We introduce a new routine for generating segmentation benchmarks, and use it to elaborate ready-to-use hyperspectral training-test data partitions. They can be utilized for fair…
Deep Neural Networks (DNNs) suffer from the performance degradation when image appearance shift occurs, especially in ultrasound (US) image segmentation. In this paper, we propose a novel and intuitive framework to remove the appearance shift, and hence improve the generalization ability of DNNs. Our work has three highlights. First, we follow the spirit of universal style transfer to remove appearance shifts, which was not explored before for US images. Without sacrificing image structure details, it enables the arbitrary style-content transfer. Second, accelerated with Adaptive Instance Norm…
The instance segmentation problem intends to precisely detect and delineate objects in images. Most of the current solutions rely on deep convolutional neural networks but despite this fact proposed solutions are very diverse. Some solutions approach the problem as a network problem, where they use several networks or specialize a single network to solve several tasks. A different approach tries to solve the problem as an annotation problem, where the instance information is encoded in a mathematical representation. This work proposes a solution based in the DCME technique to solve the instanc…
Lung cancer has emerged as a severe disease that threatens human life and health. The precise segmentation of lung regions is a crucial prerequisite for localizing tumors, which can provide accurate information for lung image analysis. In this work, we first propose a lung image segmentation model using the NASNet-Large as an encoder and then followed by a decoder architecture, which is one of the most commonly used architectures in deep learning for image segmentation. The proposed NASNet-Large-decoder architecture can extract high-level information and expand the feature map to recover the s…
Universidade de São Paulo（圣保罗大学）；Universidade Federal do Tocantins（托坎廷斯联邦大学）
Image segmentation has many applications which range from machine learning to medical diagnosis. In this paper, we propose a framework for the segmentation of images based on super-pixels and algorithms for community identification in graphs. The super-pixel pre-segmentation step reduces the number of nodes in the graph, rendering the method the ability to process large images. Moreover, community detection algorithms provide more accurate segmentation than traditional approaches, such as those based on spectral graph partition. We also compare our method with two algorithms: a) the graph-base…
Fraunhofer Heinrich-Hertz-Institute（弗劳恩霍夫海因里希·赫兹研究所）；Technische Universität Berlin（柏林工业大学）；Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
Applying traditional post-hoc attribution methods to segmentation or object detection predictors offers only limited insights, as the obtained feature attribution maps at input level typically resemble the models' predicted segmentation mask or bounding box. In this work, we address the need for more informative explanations for these predictors by proposing the post-hoc eXplainable Artificial Intelligence method L-CRP to generate explanations that automatically identify and visualize relevant concepts learned, recognized and used by the model during inference as well as precisely locate them …
