社区检测是复杂网络分析领域的一项强大工具,已在多个研究领域得到应用。例如,多种图像分割方法将社区检测算法作为黑盒工具,用于计算欠分割结果,即少量代表图像感兴趣区域的区域。然而据我们所知,这类方法在超像素任务中的效率一直被忽视——超像素旨在以更细的粒度表示图像,同时尽可能保留原始信息。唯一的相关研究似乎是Liu等人(IET Image Processing, 2022)的工作,他们利用所谓的模块度最大化方法开发了一种超像素算法,取得了不错的结果。我们延续这一研究方向,研究在4连通像素图(即像素网格)上,用前沿社区检测算法计算超像素的效率。我们首先在这类图上检测社区,随后应用简单的合并流程,以得到目标数量的超像素。正如我们将展示的,定性和定量实验(基于多种广泛使用的、依赖真值对比或仅基于超像素本身的指标)均表明,这类方法能生成高质量的超像素。我们观察到,社区检测算法的选择对社区数量有很大影响,进而影响合并流程。同样,像素网格的微小变化也会从定性和定量两方面带来不同结果。为保证研究完整性,我们将所得结果与Stutz等人(Computer Vision and Image Understanding, 2018)实现的多种前沿超像素算法的结果进行了对比。
英文摘要:
Community detection is a powerful tool from complex networks analysis that finds applications in various research areas. Several image segmentation methods rely for instance on community detection algorithms as a black box in order to compute undersegmentations, i.e. a small number of regions that represent areas of interest of the image. However, to the best of our knowledge, the efficiency of such an approach w.r.t. superpixels, that aim at representing the image at a smaller level while preserving as much as possible original information, has been neglected so far. The only related work seems to be the one by Liu et. al. (IET Image Processing, 2022) that developed a superpixels algorithm using a so-called modularity maximization approach, leading to relevant results. We follow this line of research by studying the efficiency of superpixels computed by state-of-the-art community detection algorithms on a 4-connected pixel graph, so-called pixel-grid. We first detect communities on such a graph and then apply a simple merging procedure that allows to obtain the desired number of superpixels. As we shall see, such methods result in the computation of relevant superpixels as emphasized by both qualitative and quantitative experiments, according to different widely-used metrics based on ground-truth comparison or on superpixels only. We observe that the choice of the community detection algorithm has a great impact on the number of communities and hence on the merging procedure. Similarly, small variations on the pixel-grid may provide different results from both qualitative and quantitative viewpoints. For the sake of completeness, we compare our results with those of several state-of-the-art superpixels algorithms as computed by Stutz et al. (Computer Vision and Image Understanding, 2018).
Detecting carried objects is one of the requirements for developing systems to reason about activities involving people and objects. We present an approach to detect carried objects from a single video frame with a novel method that incorporates features from multiple scales. Initially, a foreground mask in a video frame is segmented into multi-scale superpixels. Then the human-like regions in the segmented area are identified by matching a set of extracted features from superpixels against learned features in a codebook. A carried object probability map is generated using the complement of th…
Synthetic Aperture Radar (SAR) is the main instrument utilized for the detection of oil slicks on the ocean surface. In SAR images, some areas affected by ocean phenomena, such as rain cells, upwellings, and internal waves, or discharge from oil spills appear as dark spots on images. Dark spot detection is the first step in the detection of oil spills, which then become oil slick candidates. The accuracy of dark spot segmentation ultimately affects the accuracy of oil slick identification. Although some advanced deep learning methods that use pixels as processing units perform well in remote s…
STENet: Superpixel Token Enhancing Network for RGB-D Salient Object Detection
School of Communication and Information Engineering, Shanghai University(上海大学通信与信息工程学院);Yunnan Key Laboratory of Service Computing, Yunnan University of Finance and Economics(云南财经大学服务计算重点实验室);Innovation Academy for Microsatellites of Chinese Academy of Science(中国科学院微卫星创新院);Institute for Urban Governance, Shanghai University(上海大学城市治理研究院)
RGB-D salient object detection (SOD) has been in the spotlight recently because it is an important preprocessing operation for various vision tasks. However, despite advances in deep learning-based methods, RGB-D SOD is still challenging due to the large domain gap between an RGB image and the depth map and low-quality depth maps. To solve this problem, we propose a novel superpixel prototype sampling network (SPSN) architecture. The proposed model splits the input RGB image and depth map into component superpixels to generate component prototypes. We design a prototype sampling network so tha…
Saliency Object Detection (SOD) has several applications in image analysis. The methods have evolved from image-intrinsic to object-inspired (deep-learning-based) models. When a model fail, however, there is no alternative to enhance its saliency map. We fill this gap by introducing a hybrid approach, named \textit{Iterative Saliency Enhancement over Superpixel Similarity} (ISESS), that iteratively generates enhanced saliency maps by executing two operations alternately: object-based superpixel segmentation and superpixel-based saliency estimation -- cycling operations never exploited. ISESS e…
Class-agnostic object proposal generation is an important first step in many object detection pipelines. However, object proposals of modern systems are rather inaccurate in terms of segmentation and only roughly adhere to object boundaries. Since typical refinement steps are usually not applicable to thousands of proposals, we propose a superpixel-based refinement system for object proposal generation systems. Utilizing precise superpixels and superpixel pooling on deep features, we refine initial coarse proposals in an end-to-end learned system. Furthermore, we propose a novel DeepFH segment…
Computer vision algorithms with pixel-wise labeling tasks, such as semantic segmentation and salient object detection, have gone through a significant accuracy increase with the incorporation of deep learning. Deep segmentation methods slightly modify and fine-tune pre-trained networks that have hundreds of millions of parameters. In this work, we question the need to have such memory demanding networks for the specific task of salient object segmentation. To this end, we propose a way to learn a memory-efficient network from scratch by training it only on salient object detection datasets. Ou…
The effortless detection of salient objects by humans has been the subject of research in several fields, including computer vision as it has many applications. However, salient object detection remains a challenge for many computer models dealing with color and textured images. Herein, we propose a novel and efficient strategy, through a simple model, almost without internal parameters, which generates a robust saliency map for a natural image. This strategy consists of integrating color information into local textural patterns to characterize a color micro-texture. Most models in the literat…
Saliency object detection estimates the objects that most stand out in an image. The available unsupervised saliency estimators rely on a pre-determined set of assumptions of how humans perceive saliency to create discriminating features. By fixing the pre-selected assumptions as an integral part of their models, these methods cannot be easily extended for specific settings and different image domains. We then propose a superpixel-based ITerative Saliency Estimation fLexible Framework (ITSELF) that allows any user-defined assumptions to be added to the model when required. Thanks to recent adv…
While robotic manipulation of rigid objects is quite straightforward, coping with deformable objects is an open issue. More specifically, tasks like tying a knot, wiring a connector or even surgical suturing deal with the domain of Deformable Linear Objects (DLOs). In particular the detection of a DLO is a non-trivial problem especially under clutter and occlusions (as well as self-occlusions). The pose estimation of a DLO results into the identification of its parameters related to a designed model, e.g. a basis spline. It follows that the stand-alone segmentation of a DLO might not be suffic…
Hainan University(海南大学);Wuhan University of Technology(武汉理工大学);Beijing Jiaotong University(北京交通大学);Shenzhen University(深圳大学);City University of Hong Kong(香港城市大学);City University of Hong Kong Shenzhen Research Institute(香港城市大学深圳研究院)
Stereo superpixel segmentation aims at grouping the discretizing pixels into perceptual regions through left and right views more collaboratively and efficiently. Existing superpixel segmentation algorithms mostly utilize color and spatial features as input, which may impose strong constraints on spatial information while utilizing the disparity information in terms of stereo image pairs. To alleviate this issue, we propose a stereo superpixel segmentation method with a decoupling mechanism of spatial information in this work. To decouple stereo disparity information and spatial information, t…
College of Microelectronics and Communication Engineering, Chongqing University(重庆大学微电子与通信工程学院);Chongqing Key Laboratory of Space Information Network and Intelligent Information Fusion(重庆空间信息网络与智能信息融合重点实验室);Lancaster Environment Centre, Lancaster University(兰卡斯特大学兰卡斯特环境中心);UK Centre for Ecology & Hydrology(英国生态与水文中心)
Change detection is one of the fundamental applications of synthetic aperture radar (SAR) images. However, speckle noise presented in SAR images has a much negative effect on change detection. In this research, a novel two-phase object-based deep learning approach is proposed for multi-temporal SAR image change detection. Compared with traditional methods, the proposed approach brings two main innovations. One is to classify all pixels into three categories rather than two categories: unchanged pixels, changed pixels caused by strong speckle (false changes), and changed pixels formed by real t…
University of Pittsburgh(匹兹堡大学);Intelinair Inc.(Intelinair公司);University of Illinois at Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校)
Advances in remote sensing technology have led to the capture of massive amounts of data. Increased image resolution, more frequent revisit times, and additional spectral channels have created an explosion in the amount of data that is available to provide analyses and intelligence across domains, including agriculture. However, the processing of this data comes with a cost in terms of computation time and money, both of which must be considered when the goal of an algorithm is to provide real-time intelligence to improve efficiencies. Specifically, we seek to identify nutrient deficient areas…
In this paper, an stereo-based traversability analysis approach for all terrains in off-road mobile robotics, e.g. Unmanned Ground Vehicles (UGVs) is proposed. This approach reformulates the problem of terrain traversability analysis into two main problems: (1) 3D terrain reconstruction and (2) terrain all surfaces detection and analysis. The proposed approach is using stereo camera for perception and 3D reconstruction of the terrain. In order to detect all the existing surfaces in the 3D reconstructed terrain as superpixel surfaces (i.e. segments), an image segmentation technique is applied u…
China University of Geosciences(中国地质大学);City University of Hong Kong(香港城市大学);Tongji University(同济大学)
Superpixel is widely used in image processing. And among the methods for superpixel generation, clustering-based methods have a high speed and a good performance at the same time. However, most clustering-based superpixel methods are sensitive to noise. To solve these problems, in this paper, we first analyze the features of noise. Then according to the statistical features of noise, we propose a novel centroid update approach to enhance the robustness of clustering-based superpixel methods. Besides, we propose a novel superpixel-based edge detection method. The experiments on BSD500 dataset s…
Nebula AI Group, School of Computer Science, Fudan University(复旦大学计算机科学技术学院星云人工智能组);Shanghai Key Lab of Intelligent Information Processing(上海市智能信息处理重点实验室);Department of Computer Science, The University of Hong Kong(香港大学计算机科学系)
Feature pyramids have been proven powerful in image understanding tasks that require multi-scale features. State-of-the-art methods for multi-scale feature learning focus on performing feature interactions across space and scales using neural networks with a fixed topology. In this paper, we propose graph feature pyramid networks that are capable of adapting their topological structures to varying intrinsic image structures and supporting simultaneous feature interactions across all scales. We first define an image-specific superpixel hierarchy for each input image to represent its intrinsic i…
How to Identify Good Superpixels for Deforestation Detection on Tropical Rainforests
Institute of Science and Technology – Universidade Federal de São Paulo(圣保罗联邦大学科学与技术研究所);Computer Science Department – Pontifícia Universidade Católica de Minas Gerais(米纳斯吉拉斯州天主教大学计算机科学系)
Scene background initialization allows the recovery of a clear image without foreground objects from a video sequence, which is generally the first step in many computer vision and video processing applications. The process may be strongly affected by some challenges such as illumination changes, foreground cluttering, intermittent movement, etc. In this paper, a robust background initialization approach based on superpixel motion detection is proposed. Both spatial and temporal characteristics of frames are adopted to effectively eliminate foreground objects. A subsequence with stable illumin…
Instance segmentation has attracted recent attention in computer vision and existing methods in this domain mostly have an object detection stage. In this paper, we study the intrinsic challenge of the instance segmentation problem, the presence of a quotient space (swapping the labels of different instances leads to the same result), and propose new methods that are object proposal- and object detection- free. We propose three alternative methods, namely pixel-based affinity mapping, superpixel-based affinity learning, and boundary-based component segmentation, all focusing on performing labe…
Scene model construction based on image rendering is an indispensable but challenging technique in computer vision and intelligent transportation systems. In this paper, we propose a framework for constructing 3D corridor-based road scene models. This consists of two successive stages: road detection and scene construction. The road detection is realized by a new superpixel Markov random field (MRF) algorithm. The data fidelity term in the MRF's energy function is jointly computed according to the superpixel features of color, texture and location. The smoothness term is established on the bas…
Salient object detection segments attractive objects in scenes. RGB and thermal modalities provide complementary information and scribble annotations alleviate large amounts of human labor. Based on the above facts, we propose a scribble-supervised RGB-T salient object detection model. By a four-step solution (expansion, prediction, aggregation, and supervision), label-sparse challenge of scribble-supervised method is solved. To expand scribble annotations, we collect the superpixels that foreground scribbles pass through in RGB and thermal images, respectively. The expanded multi-modal labels…
We consider detecting objects in an image by iteratively selecting from a set of arbitrarily shaped candidate regions. Our generic approach, which we term visual chunking, reasons about the locations of multiple object instances in an image while expressively describing object boundaries. We design an optimization criterion for measuring the performance of a list of such detections as a natural extension to a common per-instance metric. We present an efficient algorithm with provable performance for building a high-quality list of detections from any candidate set of region-based proposals. We…
Recent advances in supervised salient object detection has resulted in significant performance on benchmark datasets. Training such models, however, requires expensive pixel-wise annotations of salient objects. Moreover, many existing salient object detection models assume that at least one salient object exists in the input image. Such an assumption often leads to less appealing saliency maps on the background images, which contain no salient object at all. To avoid the requirement of expensive pixel-wise salient region annotations, in this paper, we study weakly supervised learning approache…
McMaster University(麦克马斯特大学);Isfahan University of Technology(伊斯法罕理工大学);University of Michigan(密歇根大学)
Coronary artery disease (CAD) is the leading causes of death around the world. One of the most common imaging methods for diagnosing this disease is X-ray angiography. Diagnosing using these images is usually challenging due to non-uniform illumination, low contrast, presence of other body tissues, presence of catheter etc. These challenges make the diagnoses task of cardiologists tougher and more prone to misdiagnosis. In this paper we propose a new automated framework for coronary arteries segmentation, catheter detection and center-line extraction in x-ray angiography images. Our proposed s…
Quantum Information-Empowered Graph Neural Network for Hyperspectral Change Detection
National Cheng Kung University(成功大学);Inria(法国国家信息与自动化研究所);CNRS(法国国家科学研究中心);Grenoble INP(格勒诺布尔理工学院);LJK(让·昆廷诺实验室);Université Grenoble Alpes(格勒诺布尔阿尔卑斯大学)
Wireless Capsule Endoscopy (WCE) is a relatively new technology to record the entire GI trace, in vivo. The large amounts of frames captured during an examination cause difficulties for physicians to review all these frames. The need for reducing the reviewing time using some intelligent methods has been a challenge. Polyps are considered as growing tissues on the surface of intestinal tract not inside of an organ. Most polyps are not cancerous, but if one becomes larger than a centimeter, it can turn into cancer by great chance. The WCE frames provide the early stage possibility for detection…
Huazhong University of Science and Technology(华中科技大学);Shenzhen Huazhong University of Science and Technology Research Institute(深圳华中科技大学研究院)
Superpixels have been widely used in computer vision tasks due to their representational and computational efficiency. Meanwhile, deep learning and end-to-end framework have made great progress in various fields including computer vision. However, existing superpixel algorithms cannot be integrated into subsequent tasks in an end-to-end way. Traditional algorithms and deep learning-based algorithms are two main streams in superpixel segmentation. The former is non-differentiable and the latter needs a non-differentiable post-processing step to enforce connectivity, which constraints the integr…
The accurate detection of lesion attributes is meaningful for both the computeraid diagnosis system and dermatologists decisions. However, unlike lesion segmentation and melenoma classification, there are few deep learning methods and literatures focusing on this task. Currently, the lesion attribute detection still remains challenging due to the extremely unbalanced class distribution and insufficient samples, as well as large intraclass and low interclass variations. To solve these problems, we propose a deep learning framework named superpixel attention network (SANet). Firstly, we segment …
University of Naples Federico II(那不勒斯费德里科二世大学);University of Salzburg(萨尔茨堡大学)
Capable of automated near real time superpixel detection and quality assessment in an uncalibrated monitor typical red green blue (RGB) image, depicted in either true or false colors, an original low level computer vision (CV) lightweight computer program, called RGB Image Automatic Mapper (RGBIAM), is designed and implemented. Constrained by the Calibration Validation (CalVal) requirements of the Quality Assurance Framework for Earth Observation (QA4EO) guidelines, RGBIAM requires as mandatory an uncalibrated RGB image pre processing first stage, consisting of an automated statistical model b…
Exploring Superpixel Segmentation Methods in the Context of Citizen Science and Deforestation Detection
Institute of Science and Technology, Universidade Federal de São Paulo(圣保罗联邦大学科学与技术学院);Pontifícia Universidade Católica de Minas Gerais(米纳斯吉拉斯州天主教大学);Instituto Superior Tecnico, Universidade de Lisboa(里斯本大学高级技术学院)
Indian Institute of Technology, Gandhinagar(印度理工学院甘地纳加尔分校)
Over-segmentation of an image into superpixels has become a useful tool for solving various problems in image processing and computer vision. Reflection symmetry is quite prevalent in both natural and man-made objects and is an essential cue in understanding and grouping the objects in natural scenes. Existing algorithms for estimating superpixels do not preserve the reflection symmetry of an object which leads to different sizes and shapes of superpixels across the symmetry axis. In this work, we propose an algorithm to over-segment an image through the propagation of reflection symmetry evid…
Unsupervised Skin Lesion Segmentation via Structural Entropy Minimization on Multi-Scale Superpixel Graphs
Beihang University(北京航空航天大学);Zhongguancun Laboratory(中关村实验室);Salesforce AI Research(赛富时人工智能研究院);Didi Chuxing(滴滴出行);University of Illinois Chicago(伊利诺伊大学芝加哥分校);Lehigh University(理海大学)
Università degli Studi di Cassino e del Lazio Meridionale(卡西诺与拉齐奥南部大学)
Identification and segmentation of breast masses in mammograms face complex challenges, owing to the highly variable nature of malignant densities with regards to their shape, contours, texture and orientation. Additionally, classifiers typically suffer from high class imbalance in region candidates, where normal tissue regions vastly outnumber malignant masses. This paper proposes a rigorous segmentation method, supported by morphological enhancement using grayscale linear filters. A novel cascaded ensemble of support vector machines (SVM) is used to effectively tackle the class imbalance and…
Recent anomaly detection algorithms have shown powerful performance by adopting frame predicting autoencoders. However, these methods face two challenging circumstances. First, they are likely to be trained to be excessively powerful, generating even abnormal frames well, which leads to failure in detecting anomalies. Second, they are distracted by the large number of objects captured in both foreground and background. To solve these problems, we propose a novel superpixel-based video data transformation technique named Random Superpixel Erasing on Moving Objects (RandomSEMO) and Moving Object…
Learning Superpixel Ensemble and Hierarchy Graphs for Melanoma Detection
Department of Biomedical Engineering and Systems, Faculty of Engineering, Cairo University(开罗大学工程学院生物医学工程与系统系);School of Information Technology, New Giza University(新吉萨大学信息技术学院);Department of Mathematics, Faculty of Science, Cairo University(开罗大学理学院数学系)
The sheer volume and size of histopathological images (e.g.,10^6 MPixel) underscores the need for faster and more accurate Regions-of-interest (ROI) detection algorithms. In this paper, we propose such an algorithm, which has four main components that help achieve greater accuracy and faster speed: First, while using coarse-to-fine topology preserving segmentation as the baseline, the proposed algorithm uses a superpixel regularity optimization scheme for avoiding irregular and extremely small superpixels. Second, the proposed technique employs a prediction strategy to focus only on important …
Supervoxel methods such as Simple Linear Iterative Clustering (SLIC) are an effective technique for partitioning an image or volume into locally similar regions, and are a common building block for the development of detection, segmentation and analysis methods. We introduce maskSLIC an extension of SLIC to create supervoxels within regions-of-interest, and demonstrate,on examples from 2-dimensions to 4-dimensions, that maskSLIC overcomes issues that affect SLIC within an irregular mask. We highlight the benefits of this method through examples, and show that it is able to better represent und…
Erasmus+ Joint Program in Medical Imaging and Applications, University of Burgundy (France), UNICLAM (Italy) and University of Girona (Spain)(伊拉斯谟+医学影像与应用联合项目,勃艮第大学(法国)、UNICLAM(意大利)与赫罗纳大学(西班牙))
Positron Emission Tomography scan images are extensively used in radiotherapy planning, clinical diagnosis, assessment of growth and treatment of a tumor. These all rely on fidelity and speed of detection and delineation algorithm. Despite intensive research, segmentation remained a challenging problem due to the diverse image content, resolution, shape, and noise. This paper presents a fast positron emission tomography tumor segmentation method in which superpixels are extracted first from the input image. Principal component analysis is then applied on the superpixels and also on their avera…
Automatic visual fire detection is used to complement traditional fire detection sensor systems (smoke/heat). In this work, we investigate different Convolutional Neural Network (CNN) architectures and their variants for the non-temporal real-time bounds detection of fire pixel regions in video (or still) imagery. Two reduced complexity compact CNN architectures (NasNet-A-OnFire and ShuffleNetV2-OnFire) are proposed through experimental analysis to optimise the computational efficiency for this task. The results improve upon the current state-of-the-art solution for fire detection, achieving a…
We test this premise and explore representation spaces from a single deep convolutional network and their visualization to argue for a novel unified feature extraction framework. The objective is to utilize and re-purpose trained feature extractors without the need for network retraining on three remote sensing tasks i.e. superpixel mapping, pixel-level segmentation and semantic based image visualization. By leveraging the same convolutional feature extractors and viewing them as visual information extractors that encode different image representation spaces, we demonstrate a preliminary induc…
UC San Diego(加州大学圣地亚哥分校);Cyprus International University(塞浦路斯国际大学);the Hong Kong University of Science and Technology(香港科技大学);University of Thessaloniki(塞萨洛尼基大学)
Potholes are one of the most common forms of road damage, which can severely affect driving comfort, road safety and vehicle condition. Pothole detection is typically performed by either structural engineers or certified inspectors. This task is, however, not only hazardous for the personnel but also extremely time-consuming. This paper presents an efficient pothole detection algorithm based on road disparity map estimation and segmentation. We first generalize the perspective transformation by incorporating the stereo rig roll angle. The road disparities are then estimated using semi-global m…
We propose a new fast fully unsupervised method to discover semantic patterns. Our algorithm is able to hierarchically find visual categories and produce a segmentation mask where previous methods fail. Through the modeling of what is a visual pattern in an image, we introduce the notion of "semantic levels" and devise a conceptual framework along with measures and a dedicated benchmark dataset for future comparisons. Our algorithm is composed by two phases. A filtering phase, which selects semantical hotsposts by means of an accumulator space, then a clustering phase which propagates the sema…
Image saliency detection is an active research topic in the community of computer vision and multimedia. Fusing complementary RGB and thermal infrared data has been proven to be effective for image saliency detection. In this paper, we propose an effective approach for RGB-T image saliency detection. Our approach relies on a novel collaborative graph learning algorithm. In particular, we take superpixels as graph nodes, and collaboratively use hierarchical deep features to jointly learn graph affinity and node saliency in a unified optimization framework. Moreover, we contribute a more challen…
Individual tree detection and crown delineation (ITDD) are critical in forest inventory management and remote sensing based forest surveys are largely carried out through satellite images. However, most of these surveys only use 2D spectral information which normally has not enough clues for ITDD. To fully explore the satellite images, we propose a ITDD method using the orthophoto and digital surface model (DSM) derived from the multi-view satellite data. Our algorithm utilizes the top-hat morphological operation to efficiently extract the local maxima from DSM as treetops, and then feed them …
In this paper, we propose an improved mechanism for saliency detection. Firstly,based on a neoteric background prior selecting four corners of an image as background,we use color and spatial contrast with each superpixel to obtain a salinecy map(CBP). Inspired by reverse-measurement methods to improve the accuracy of measurement in Engineering,we employ the Objectness labels as foreground prior based on part of information of CBP to construct a map(OFP).Further,an original energy function is applied to optimize both of them respectively and a single-layer saliency map(SLP)is formed by merging …
This paper proposes a novel saliency detection method by combining region-level saliency estimation and pixel-level saliency prediction with CNNs (denoted as CRPSD). For pixel-level saliency prediction, a fully convolutional neural network (called pixel-level CNN) is constructed by modifying the VGGNet architecture to perform multi-scale feature learning, based on which an image-to-image prediction is conducted to accomplish the pixel-level saliency detection. For region-level saliency estimation, an adaptive superpixel based region generation technique is first designed to partition an image …