Scribble-based Boundary-aware Network for Weakly Supervised Salient Object Detection in Remote Sensing Images
University of Electronic Science and Technology of China（电子科技大学）；Inception Institute of Artificial Intelligence (IIAI)（人工智能初始研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Zhou Huang, Tian-Zhu Xiang, Huai-Xin Chen, Hang Dai
Existing CNNs-based salient object detection (SOD) heavily depends on the large-scale pixel-level annotations, which is labor-intensive, time-consuming, and expensive. By contrast, the sparse annotations become appealing to the salient object detection community. However, few efforts are devoted to learning salient object detection from sparse annotations, especially in the remote sensing field. In addition, the sparse annotation usually contains scanty information, which makes it challenging to train a well-performing model, resulting in its performance largely lagging behind the fully-supervised models. Although some SOD methods adopt some prior cues to improve the detection performance, they usually lack targeted discrimination of object boundaries and thus provide saliency maps with poor boundary localization. To this end, in this paper, we propose a novel weakly-supervised salient object detection framework to predict the saliency of remote sensing images from sparse scribble annotations. To implement it, we first construct the scribble-based remote sensing saliency dataset by relabelling an existing large-scale SOD dataset with scribbles, namely S-EOR dataset. After that, we present a novel scribble-based boundary-aware network (SBA-Net) for remote sensing salient object detection. Specifically, we design a boundary-aware module (BAM) to explore the object boundary semantics, which is explicitly supervised by the high-confidence object boundary (pseudo) labels generated by the boundary label generation (BLG) module, forcing the model to learn features that highlight the object structure and thus boosting the boundary localization of objects. Then, the boundary semantics are integrated with high-level features to guide the salient object detection under the supervision of scribble labels.
Salient object detection segments attractive objects in scenes. RGB and thermal modalities provide complementary information and scribble annotations alleviate large amounts of human labor. Based on the above facts, we propose a scribble-supervised RGB-T salient object detection model. By a four-step solution (expansion, prediction, aggregation, and supervision), label-sparse challenge of scribble-supervised method is solved. To expand scribble annotations, we collect the superpixels that foreground scribbles pass through in RGB and thermal images, respectively. The expanded multi-modal labels…
Australian National University（澳大利亚国立大学）；ACRV（澳大利亚机器人视觉中心）；Data61（数据61）；University of Technology Sydney（悉尼科技大学）；Northwestern Polytechnical University（西北工业大学）
Compared with laborious pixel-wise dense labeling, it is much easier to label data by scribbles, which only costs 1$\sim$2 seconds to label one image. However, using scribble labels to learn salient object detection has not been explored. In this paper, we propose a weakly-supervised salient object detection model to learn saliency from such annotations. In doing so, we first relabel an existing large-scale salient object detection dataset with scribbles, namely S-DUTS dataset. Since object structure and detail information is not identified by scribbles, directly training with scribble labels …
Road surface extraction from remote sensing images using deep learning methods has achieved good performance, while most of the existing methods are based on fully supervised learning, which requires a large amount of training data with laborious per-pixel annotation. In this paper, we propose a scribble-based weakly supervised road surface extraction method named ScRoadExtractor, which learns from easily accessible scribbles such as centerlines instead of densely annotated road surface ground-truths. To propagate semantic information from sparse scribbles to unlabeled pixels, we introduce a r…
Weakly-Supervised RGB-D Salient Object Detection via SAM-driven Pseudo Annotation and State Space Interaction-based Diffusion
School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）
School of Advanced Technology, Xi’an Jiaotong Liverpool University（西交利物浦大学先进技术学院）
Sparse labels have been attracting much attention in recent years. However, the performance gap between weakly supervised and fully supervised salient object detection methods is huge, and most previous weakly supervised works adopt complex training methods with many bells and whistles. In this work, we propose a one-round end-to-end training approach for weakly supervised salient object detection via scribble annotations without pre/post-processing operations or extra supervision data. Since scribble labels fail to offer detailed salient regions, we propose a local coherence loss to propagate…
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）；Wageningen University（瓦赫宁根大学）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Training Convolutional Neural Networks (CNNs) for very high resolution images requires a large quantity of high-quality pixel-level annotations, which is extremely labor- and time-consuming to produce. Moreover, professional photo interpreters might have to be involved for guaranteeing the correctness of annotations. To alleviate such a burden, we propose a framework for semantic segmentation of aerial images based on incomplete annotations, where annotators are asked to label a few pixels with easy-to-draw scribbles. To exploit these sparse scribbled annotations, we propose the FEature and Sp…
Northwestern Polytechnical University（西北工业大学）；Australian National University（澳大利亚国立大学）；CSIRO Australia（澳大利亚联邦科学与工业研究组织）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Significant performance improvement has been achieved for fully-supervised video salient object detection with the pixel-wise labeled training datasets, which are time-consuming and expensive to obtain. To relieve the burden of data annotation, we present the first weakly supervised video salient object detection model based on relabeled "fixation guided scribble annotations". Specifically, an "Appearance-motion fusion module" and bidirectional ConvLSTM based framework are proposed to achieve effective multi-modal learning and long-term temporal context modeling based on our new weak annotatio…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Leicester（莱斯特大学）
Due to the more dramatic multi-scale variations and more complicated foregrounds and backgrounds in optical remote sensing images (RSIs), the salient object detection (SOD) for optical RSIs becomes a huge challenge. However, different from natural scene images (NSIs), the discussion on the optical RSI SOD task still remains scarce. In this paper, we propose a multi-scale context network, namely MSCNet, for SOD in optical RSIs. Specifically, a multi-scale context extraction module is adopted to address the scale variation of salient objects by effectively learning multi-scale contextual informa…
City University of Hong Kong（香港城市大学）；Beijing Jiaotong University（北京交通大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Tianjin University（天津大学）
Arising from the various object types and scales, diverse imaging orientations, and cluttered backgrounds in optical remote sensing image (RSI), it is difficult to directly extend the success of salient object detection for nature scene image to the optical RSI. In this paper, we propose an end-to-end deep network called LV-Net based on the shape of network architecture, which detects salient objects from optical RSIs in a purely data-driven fashion. The proposed LV-Net consists of two key modules, i.e., a two-stream pyramid module (L-shaped module) and an encoder-decoder module with nested co…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Leicester（莱斯特大学）
Due to the extreme complexity of scale and shape as well as the uncertainty of the predicted location, salient object detection in optical remote sensing images (RSI-SOD) is a very difficult task. The existing SOD methods can satisfy the detection performance for natural scene images, but they are not well adapted to RSI-SOD due to the above-mentioned image characteristics in remote sensing images. In this paper, we propose a novel Attention Guided Network (AGNet) for SOD in optical RSIs, including position enhancement stage and detail refinement stage. Specifically, the position enhancement s…
Salient object detection (SOD) in optical remote sensing images (RSIs), or RSI-SOD, is an emerging topic in understanding optical RSIs. However, due to the difference between optical RSIs and natural scene images (NSIs), directly applying NSI-SOD methods to optical RSIs fails to achieve satisfactory results. In this paper, we propose a novel Adjacent Context Coordination Network (ACCoNet) to explore the coordination of adjacent features in an encoder-decoder architecture for RSI-SOD. Specifically, ACCoNet consists of three parts: an encoder, Adjacent Context Coordination Modules (ACCoMs), and …
Beijing Jiaotong University（北京交通大学）；City University of Hong Kong（香港城市大学）；Hunan University（湖南大学）；Peng Cheng Laboratory（鹏城实验室）；Sun Yat-sen University（中山大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）
Salient object detection (SOD) for optical remote sensing images (RSIs) aims at locating and extracting visually distinctive objects/regions from the optical RSIs. Despite some saliency models were proposed to solve the intrinsic problem of optical RSIs (such as complex background and scale-variant objects), the accuracy and completeness are still unsatisfactory. To this end, we propose a relational reasoning network with parallel multi-scale attention for SOD in optical RSIs in this paper. The relational reasoning module that integrates the spatial and the channel dimensions is designed to in…
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Beijing Key Laboratory of Advanced Information Science and Network Technology（北京市先进信息科学与网络技术重点实验室）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）
The diverse spatial resolutions, various object types, scales and orientations, and cluttered backgrounds in optical remote sensing images (RSIs) challenge the current salient object detection (SOD) approaches. It is commonly unsatisfactory to directly employ the SOD approaches designed for nature scene images (NSIs) to RSIs. In this paper, we propose a novel Parallel Down-up Fusion network (PDF-Net) for SOD in optical RSIs, which takes full advantage of the in-path low- and high-level features and cross-path multi-resolution features to distinguish diversely scaled salient objects and suppres…
City University of Hong Kong（香港城市大学）；Beijing Jiaotong University（北京交通大学）
Despite the remarkable advances in visual saliency analysis for natural scene images (NSIs), salient object detection (SOD) for optical remote sensing images (RSIs) still remains an open and challenging problem. In this paper, we propose an end-to-end Dense Attention Fluid Network (DAFNet) for SOD in optical RSIs. A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships, and is further embedded in a Dense Attention Fluid (DAF) structure that enables shallow attention cues flow into deep layers to guide the generation of high-leve…
Fudan University（复旦大学）；Yiwu Research Institute of Fudan University（复旦大学义乌研究院）
Video salient object detection models trained on pixel-wise dense annotation have achieved excellent performance, yet obtaining pixel-by-pixel annotated datasets is laborious. Several works attempt to use scribble annotations to mitigate this problem, but point supervision as a more labor-saving annotation method (even the most labor-saving method among manual annotation methods for dense prediction), has not been explored. In this paper, we propose a strong baseline model based on point supervision. To infer saliency maps with temporal information, we mine inter-frame complementary informatio…
G2HFNet: GeoGran-Aware Hierarchical Feature Fusion Network for Salient Object Detection in Optical Remote Sensing Images
School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）；School of Automation, Hangzhou Dianzi University（自动化学院，杭州电子科技大学）；School of information engineering, Huzhou University（信息工程学院，湖州大学）；School of Data Science, Lingnan University（数据科学学院，岭南大学）
University of Electronic Science and Technology of China（电子科技大学）；Inception Institute of Artificial Intelligence (IIAI)（起源人工智能研究院）；CETC Special Mission Aircraft System Engineering Co.Ltd（中国电子科技集团特种任务飞机系统工程技术有限公司）
Object detection is an important task in remote sensing image analysis. To reduce the computational complexity of redundant information and improve the efficiency of image processing, visual saliency models have been widely applied in this field. In this paper, a novel saliency detection model based on Contrast-weighted Dictionary Learning (CDL) is proposed for remote sensing images. Specifically, the proposed CDL learns salient and non-salient atoms from positive and negative samples to construct a discriminant dictionary, in which a contrast-weighted term is proposed to encourage the contras…
SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection
Key Laboratory of Intelligent Computing and Signal Processing of Ministry of Education, School of Computer Science and Technology, Anhui University（教育部智能计算与信号处理重点实验室，计算机科学与技术学院，安徽大学）
Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；Shanghai University（上海大学）；Nanyang Technological University（南洋理工大学）；Stony Brook University（石溪大学）
Salient object detection in optical remote sensing images (ORSI-SOD) has been widely explored for understanding ORSIs. However, previous methods focus mainly on improving the detection accuracy while neglecting the cost in memory and computation, which may hinder their real-world applications. In this paper, we propose a novel lightweight ORSI-SOD solution, named CorrNet, to address these issues. In CorrNet, we first lighten the backbone (VGG-16) and build a lightweight subnet for feature extraction. Then, following the coarse-to-fine strategy, we generate an initial coarse saliency map from h…
Beyond Global Scanning: Adaptive Visual State Space Modeling for Salient Object Detection in Optical Remote Sensing Images
School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）
Shanghai Institute for Advanced Communication and Data Science, Shanghai University（上海大学上海先进通信与数据科学研究院）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Department of Computer Science, Stony Brook University（石溪大学计算机科学系）
In the computer vision community, great progresses have been achieved in salient object detection from natural scene images (NSI-SOD); by contrast, salient object detection in optical remote sensing images (RSI-SOD) remains to be a challenging emerging topic. The unique characteristics of optical RSIs, such as scales, illuminations and imaging orientations, bring significant differences between NSI-SOD and RSI-SOD. In this paper, we propose a novel Multi-Content Complementation Network (MCCNet) to explore the complementarity of multiple content for RSI-SOD. Specifically, MCCNet is based on the…
Current state-of-the-art saliency detection models rely heavily on large datasets of accurate pixel-wise annotations, but manually labeling pixels is time-consuming and labor-intensive. There are some weakly supervised methods developed for alleviating the problem, such as image label, bounding box label, and scribble label, while point label still has not been explored in this field. In this paper, we propose a novel weakly-supervised salient object detection method using point supervision. To infer the saliency map, we first design an adaptive masked flood filling algorithm to generate pseud…
RDNet: Region Proportion-Aware Dynamic Adaptive Salient Object Detection Network in Optical Remote Sensing Images
School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；Key Laboratory of Machine Intelligence and System Control, Ministry of Education（教育部智能信息与系统控制重点实验室）；School of Automation, Hangzhou Dianzi University（自动化学院，杭州电子科技大学）；School of Mathematics and Computer Science, Lishui University（数学与计算机科学学院，丽水大学）；School of Data Science, Lingnan University（数据科学学院，岭南大学）
We present a conceptually simple self-supervised method for saliency detection. Our method generates and uses pseudo-ground truth labels for training. The generated pseudo-GT labels don't require any kind of human annotations (e.g., pixel-wise labels or weak labels like scribbles). Recent works show that features extracted from classification tasks provide important saliency cues like structure and semantic information of salient objects in the image. Our method, called 3SD, exploits this idea by adding a branch for a self-supervised classification task in parallel with salient object detectio…
Segmentation using deep learning has shown promising directions in medical imaging as it aids in the analysis and diagnosis of diseases. Nevertheless, a main drawback of deep models is that they require a large amount of pixel-level labels, which are laborious and expensive to obtain. To mitigate this problem, weakly supervised learning has emerged as an efficient alternative, which employs image-level labels, scribbles, points, or bounding boxes as supervision. Among these, image-level labels are easier to obtain. However, since this type of annotation only contains object category informatio…
Boundary-semantic collaborative guidance network with dual-stream feedback mechanism for salient object detection in optical remote sensing imagery
State-Province Joint Engineering Laboratory of Spatial Information Technology of High-Speed Rail Safety, Southwest Jiaotong University（西南交通大学高速铁路安全空间信息技术省部共建工程实验室）
IPDiff: Diffusion-driven ORSI Salient Object Detection with Information Reconstruction and Multi-Prior Guidance
School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；Department of Medical Equipment, the First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院医学装备部）；School of Control Science and Engineering, Shandong University（山东大学控制科学与工程学院）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Shenzhen Key Laboratory of Ubiquitous Data Enabling, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院深圳泛在数据赋能重点实验室）
Texture-Semantic Collaboration Network for ORSI Salient Object Detection
Shanghai University（上海大学）；Shanghai Institute for Advanced Communication and Data Science（上海先进通信与数据科学研究院）；School of Communication and Information Engineering, Shanghai University（上海大学通信与信息工程学院）；Wenzhou Institute of Shanghai University（上海大学温州研究院）
SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensing
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Artificial Intelligence and Automation, Wuhan University of Science and Technology（武汉科技大学人工智能与自动化学院）；School of Electronic Information, Wuhan University of Science and Technology（武汉科技大学电子信息学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
A Multi-Source Data Fusion-based Semantic Segmentation Model for Relic Landslide Detection
Key Lab of Universal Wireless Communication, MOE, Beijing University of Posts and Telecommunications（信息通信领域重点实验室，教育部，北京邮电大学）；China Aero Geophysical Survey and Remote Sensing Center for Natural Resources（中国航空地球物理测绘与遥感自然资源中心）；School of Engineering and Mathematical Sciences, La Trobe University（拉筹伯大学工程与数学科学学院）
State-of-the-art detection systems are generally evaluated on their ability to exhaustively retrieve objects densely distributed in the image, across a wide variety of appearances and semantic categories. Orthogonal to this, many real-life object detection applications, for example in remote sensing, instead require dealing with large images that contain only a few small objects of a single class, scattered heterogeneously across the space. In addition, they are often subject to strict computational constraints, such as limited battery capacity and computing power. To tackle these more practic…
An Iterative Classification and Semantic Segmentation Network for Old Landslide Detection Using High-Resolution Remote Sensing Images
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Institute of Technology（北京理工大学）；China Aero Geophysical Survey and Remote Sensing Center for Natural Resources（中国自然资源航空物探遥感中心）；La Trobe University（乐卓博大学）
Department of Forest Sciences, University of Helsinki（赫尔辛基大学森林科学系）；School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, Wuhan, China（武汉大学计算机学院、国家多媒体软件工程研究中心、人工智能研究院、湖北省多媒体与网络通信工程重点实验室，武汉，中国）
School of Computer and Artificial Intelligence, Zhengzhou University（计算机与人工智能学院，郑州大学）；Wuhan University（武汉大学）；Harbin Institution of Technology（哈尔滨理工大学）；Dalian Maritime University（大连海事大学）；National Tsinghua University（清华大学）
