Salient objects attract human attention and usually stand out clearly from their surroundings. In contrast, camouflaged objects share similar colors or textures with the environment. In this case, salient objects are typically non-camouflaged, and camouflaged objects are usually not salient. Due to this inherent contradictory attribute, we introduce an uncertainty-aware learning pipeline to extensively explore the contradictory information of salient object detection (SOD) and camouflaged object detection (COD) via data-level and task-wise contradiction modeling. We first exploit the dataset correlation of these two tasks and claim that the easy samples in the COD dataset can serve as hard samples for SOD to improve the robustness of the SOD model. Based on the assumption that these two models should lead to activation maps highlighting different regions of the same input image, we further introduce a contrastive module with a joint-task contrastive learning framework to explicitly model the contradictory attributes of these two tasks. Different from conventional intra-task contrastive learning for unsupervised representation learning, our contrastive module is designed to model the task-wise correlation, leading to cross-task representation learning. To better understand the two tasks from the perspective of uncertainty, we extensively investigate the uncertainty estimation techniques for modeling the main uncertainties of the two tasks, namely task uncertainty (for SOD) and data uncertainty (for COD), and aiming to effectively estimate the challenging regions for each task to achieve difficulty-aware learning. Experimental results on benchmark datasets demonstrate that our solution leads to both state-of-the-art performance and informative uncertainty estimation.
Rethinking Detecting Salient and Camouflaged Objects in Unconstrained Scenes
School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；School of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程学院）；School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）
Northwestern Polytechnical University（西北工业大学）；Australian National University（澳大利亚国立大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；EPFL（瑞士洛桑联邦理工学院）
Visual salient object detection (SOD) aims at finding the salient object(s) that attract human attention, while camouflaged object detection (COD) on the contrary intends to discover the camouflaged object(s) that hidden in the surrounding. In this paper, we propose a paradigm of leveraging the contradictory information to enhance the detection ability of both salient object detection and camouflaged object detection. We start by exploiting the easy positive samples in the COD dataset to serve as hard positive samples in the SOD task to improve the robustness of the SOD model. Then, we introdu…
Distribution-Specific Learning for Joint Salient and Camouflaged Object Detection
School of Computing and Information Technology, Great Bay University（计算与信息学院，大湾大学）；Computer Vision and Pattern Recognition Laboratory, School of Engineering Science, Lappeenranta-Lahti University of Technology LUT（计算机视觉与模式识别实验室，工程科学学院，拉佩兰塔-拉赫蒂技术大学）；School of Computer Science and Software Engineering, Shenzhen University（计算机科学与软件工程学院，深圳大学）；School of Software Engineering, Xi’an Jiaotong University（软件工程学院，西安交通大学）；School of Electrical and Information Engineering, Tianjin University（电气与信息工程学院，天津大学）
Seamless Detection: Unifying Salient Object Detection and Camouflaged Object Detection
School of Computer Science and Artificial Intelligence, Changzhou University（常州大学计算机科学与人工智能学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；Department of Automation, Tsinghua University（清华大学自动化系）
VSCode: General Visual Salient and Camouflaged Object Detection with 2D Prompt Learning
Northwestern Polytechnical University（西北工业大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；National University of Singapore（新加坡国立大学）；Nankai University（南开大学）；Nankai International Advanced Research Institute (SHENZHEN FUTIAN)（南开大学深圳福田国际高级研究院）；Linköping University（林雪平大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
Referring Camouflaged Object Detection With Multi-Context Overlapped Windows Cross-Attention
School of Computer and Information Engineering, Shanghai Polytechnic University（计算机与信息工程学院，上海理工大学）；Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University（智能信息处理重点实验室，复旦大学）；Department of Automation, School of Intelligent Manufacturing and Control Engineering, Shanghai Polytechnic University（自动化系，智能制造与控制工程学院，上海理工大学）；Academy for Engineering and Technology, Fudan University（工程与技术学院，复旦大学）；Research and Design Center, Shanghai Insititue Of Computer Technology Company（研发设计中心，上海计算机技术有限公司）
HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection
School of Artificial Intelligence, Jiangxi Normal University（江西师范大学人工智能学院）；Faculty of Business Information, Shanghai Business School（上海商学院商务信息学院）；School of Computer Science and Information Engineering, Shanghai Institute of Technology（上海应用技术大学计算机科学与信息工程学院）
Tencent Youtu Lab（腾讯优图实验室）；ETH Zurich（苏黎世联邦理工学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Sun Yat-sen University（中山大学）
Spotting camouflaged objects that are visually assimilated into the background is tricky for both object detection algorithms and humans who are usually confused or cheated by the perfectly intrinsic similarities between the foreground objects and the background surroundings. To tackle this challenge, we aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries. We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner, essentially a global loop…
Assisted Refinement Network Based on Channel Information Interaction for Camouflaged and Salient Object Detection
Department of Physics, J.K. Institute of Science（物理系，J.K.科学研究院）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；School of Computer Science and Technology, Xinjiang University（计算机科学与技术学院，新疆大学）；Department of Electronic Engineering, Tsinghua University（电子工程系，清华大学）
A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection
School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；School of Computing and Information Technology, Great Bay University（湾区大学计算与信息技术学院）；Computer Vision and Pattern Recognition Laboratory, Lappeenranta-Lahti University of Technology LUT（拉彭兰塔-拉赫蒂理工大学计算机视觉与模式识别实验室）；School of Statistics and Data Science, Nankai University（南开大学统计与数据科学学院）
本文提出基于 ViT 的简单非对称编解码网络 SENet，结合 LICM 与动态加权损失，并解决 COD/SOD 联合训练冲突，在两类检测任务上均取得有竞争力的性能。
School of Computer Science, Nankai University（南开大学计算机学院）；BNRist, Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系BNRist（注：BNRist为北京信息科学与技术国家研究中心的英文缩写））
Monash University（莫纳什大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；ETH Zurich（苏黎世联邦理工学院）；SenseTime Research（商汤科技研究院）；Shanghai AI Laboratory（上海人工智能实验室）；Data61, CSIRO（联邦科学与工业研究组织Data61）；Airdoc Research Australia（澳大利亚鹰瞳研究院）
We propose a new video camouflaged object detection (VCOD) framework that can exploit both short-term dynamics and long-term temporal consistency to detect camouflaged objects from video frames. An essential property of camouflaged objects is that they usually exhibit patterns similar to the background and thus make them hard to identify from still images. Therefore, effectively handling temporal dynamics in videos becomes the key for the VCOD task as the camouflaged objects will be noticeable when they move. However, current VCOD methods often leverage homography or optical flows to represent…
C3Net: Context-Contrast Network for Camouflaged Object Detection
Computer Engineering Department, King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学计算机工程系）；Electronic Systems Engineering, University of Regina（里贾纳大学电子系统工程系）；Department of Computer Science and Software Engineering, The University of Western Australia（西澳大学计算机科学与软件工程系）
UESTC（电子科技大学）；Group 42 (G42)（G42集团）；Qingdao University（青岛大学）；Inception Institute of AI (IIAI)（Inception人工智能研究院）
Automatically detecting/segmenting object(s) that blend in with their surroundings is difficult for current models. A major challenge is that the intrinsic similarities between such foreground objects and background surroundings make the features extracted by deep model indistinguishable. To overcome this challenge, an ideal model should be able to seek valuable, extra clues from the given scene and incorporate them into a joint learning framework for representation co-enhancement. With this inspiration, we design a novel Mutual Graph Learning (MGL) model, which generalizes the idea of convent…
RealCamo: Boosting Real Camouflage Synthesis with Layout Controls and Textual-Visual Guidance
College of Artificial Intelligence, Nankai University, Tianjin, China（人工智能学院，南开大学，天津，中国）；School of Data Science, Fudan University, Shanghai, China（数据科学学院，复旦大学，上海，中国）
DualGazeNet: A Biologically Inspired Dual-Gaze Query Network for Salient Object Detection
School of Computation, Information and Technology, Technical University of Munich（计算信息与技术学院，慕尼黑技术大学）；Department of Informatics, Technical University of Munich（信息学院，慕尼黑技术大学）；University Research and Innovation Center, Obuda University（奥布达大学研究与创新中心）；State Key Laboratory for Novel Software Technology, Nanjing University Suzhou Campus（新型软件技术国家重点实验室，南京大学苏州校区）；School of Science and Technology, Nanjing University Suzhou Campus（科学与技术学院，南京大学苏州校区）；Department of Computer Science and Technology, Tsinghua University（计算机科学与技术系，清华大学）
Benefiting from color independence, illumination invariance and location discrimination attributed by the depth map, it can provide important supplemental information for extracting salient objects in complex environments. However, high-quality depth sensors are expensive and can not be widely applied. While general depth sensors produce the noisy and sparse depth information, which brings the depth-based networks with irreversible interference. In this paper, we propose a novel multi-task and multi-modal filtered transformer (MMFT) network for RGB-D salient object detection (SOD). Specificall…
Sichuan University（四川大学）；Nankai University（南开大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Wuhan University（武汉大学）
This paper proposes a novel joint learning and densely-cooperative fusion (JL-DCF) architecture for RGB-D salient object detection. Existing models usually treat RGB and depth as independent information and design separate networks for feature extraction from each. Such schemes can easily be constrained by a limited amount of training data or over-reliance on an elaborately-designed training process. In contrast, our JL-DCF learns from both RGB and depth inputs through a Siamese network. To this end, we propose two effective components: joint learning (JL), and densely-cooperative fusion (DCF)…
MSRNet: A Multi-Scale Recursive Network for Camouflaged Object Detection
King Fahd University of Petroleum and Minerals（国王法赫德石油与矿物大学）；Ontario Tech University（安大略技术大学）；National University of Computer and Emerging Sciences（国家计算机与新兴科学大学）；The University of Regina（里贾纳大学）；The University of Western Australia（西澳大学）
University of Oulu（奥卢大学）；University of Oxford（牛津大学）；Xi’an Jiaotong University（西安交通大学）；The University of Sydney（悉尼大学）；Xiamen University（厦门大学）；University of Technology Sydney（悉尼科技大学）
Recent saliency models extensively explore to incorporate multi-scale contextual information from Convolutional Neural Networks (CNNs). Besides direct fusion strategies, many approaches introduce message-passing to enhance CNN features or predictions. However, the messages are mainly transmitted in two ways, by feature-to-feature passing, and by prediction-to-prediction passing. In this paper, we add message-passing between features and predictions and propose a deep unified CRF saliency model . We design a novel cascade CRFs architecture with CNN to jointly refine deep features and prediction…
Recent advances in supervised salient object detection has resulted in significant performance on benchmark datasets. Training such models, however, requires expensive pixel-wise annotations of salient objects. Moreover, many existing salient object detection models assume that at least one salient object exists in the input image. Such an assumption often leads to less appealing saliency maps on the background images, which contain no salient object at all. To avoid the requirement of expensive pixel-wise salient region annotations, in this paper, we study weakly supervised learning approache…
Salient object detection is a fundamental topic in computer vision. Previous methods based on RGB-D often suffer from the incompatibility of multi-modal feature fusion and the insufficiency of multi-scale feature aggregation. To tackle these two dilemmas, we propose a novel multi-modal and multi-scale refined network (M2RNet). Three essential components are presented in this network. The nested dual attention module (NDAM) explicitly exploits the combined features of RGB and depth flows. The adjacent interactive aggregation module (AIAM) gradually integrates the neighbor features of high, midd…
Tianjin University（天津大学）；Nanyang Technology University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Cleveland State University（克利夫兰州立大学）；IHPC, A*STAR（新加坡科技研究局高性能计算研究所）；Zhejiang Sci-Tech University（浙江理工大学）；University of South Carolina（南卡罗来纳大学）
Co-salient object detection (CoSOD) has recently achieved significant progress and played a key role in retrieval-related tasks. However, it inevitably poses an entirely new safety and security issue, i.e., highly personal and sensitive content can potentially be extracting by powerful CoSOD methods. In this paper, we address this problem from the perspective of adversarial attacks and identify a novel task: adversarial co-saliency attack. Specially, given an image selected from a group of images containing some common and salient objects, we aim to generate an adversarial version that can mis…
Dalian University of Technology（大连理工大学）；University of Adelaide（阿德莱德大学）；Shanghai Jiao Tong University（上海交通大学）
Salient object detection, which aims to identify and locate the most salient pixels or regions in images, has been attracting more and more interest due to its various real-world applications. However, this vision task is quite challenging, especially under complex image scenes. Inspired by the intrinsic reflection of natural images, in this paper we propose a novel feature learning framework for large-scale salient object detection. Specifically, we design a symmetrical fully convolutional network (SFCN) to learn complementary saliency features under the guidance of lossless feature reflectio…
Huazhong University of Science and Technology（华中科技大学）；VIVO Inc.（维沃移动通信有限公司）
Humans have a strong class-agnostic object segmentation ability and can outline boundaries of unknown objects precisely, which motivates us to propose a box-supervised class-agnostic object segmentation (BoxCaseg) based solution for weakly-supervised instance segmentation. The BoxCaseg model is jointly trained using box-supervised images and salient images in a multi-task learning manner. The fine-annotated salient images provide class-agnostic and precise object localization guidance for box-supervised images. The object masks predicted by a pretrained BoxCaseg model are refined via a novel m…
Debate-Enhanced Pseudo Labeling and Frequency-Aware Progressive Debiasing for Weakly-Supervised Camouflaged Object Detection with Scribble Annotations
School of Cyber Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；Jiangsu Provincial Key Laboratory of Computer Networking Technology（江苏省计算机网络技术重点实验室）；Jiangsu Provincial Key Laboratory of Network and Information Security（江苏省网络与信息安全重点实验室）；Key Laboratory of Computer Network and Information Integration of Ministry of Education of China（教育部计算机网络与信息集成重点实验室）；Nanjing Purple Mountain Laboratories（南京紫金山实验室）；Fintech and Big Data Laboratory of Southeast University（东南大学金融科技与大数据实验室）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空与航空工程系）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Electronics, Electrical Engineering and Computer Science, Queen’s University Belfast（贝尔法斯特女王大学电子、电气与计算机科学学院）
Deep learning solutions of the salient object detection problem have achieved great results in recent years. The majority of these models are based on encoders and decoders, with a different multi-feature combination. In this paper, we show that feature concatenation works better than other combination methods like multiplication or addition. Also, joint feature learning gives better results, because of the information sharing during their processing. We designed a Complementary Extraction Module (CEM) to extract necessary features with edge preservation. Our proposed Excessiveness Loss (EL) f…
Dalian University of Technology（大连理工大学）；Shanghai Jiao Tong University（上海交通大学）；University of Adelaide（阿德莱德大学）
Salient object detection (SOD), which aims to identify and locate the most salient pixels or regions in images, has been attracting more and more interest due to its various real-world applications. However, this vision task is quite challenging, especially under complex image scenes. Inspired by the intrinsic reflection of natural images, in this paper we propose a novel feature learning framework for large-scale salient object detection. Specifically, we design a symmetrical fully convolutional network (SFCN) to effectively learn complementary saliency features under the guidance of lossless…
UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Computer Science, The University of Sydney（悉尼大学计算机学院）；School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）
Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences（中国科学院上海微系统与信息技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；ShanghaiTech University（上海科技大学）
We propose a novel 3D point cloud segmentation framework named SASO, which jointly performs semantic and instance segmentation tasks. For semantic segmentation task, inspired by the inherent correlation among objects in spatial context, we propose a Multi-scale Semantic Association (MSA) module to explore the constructive effects of the semantic context information. For instance segmentation task, different from previous works that utilize clustering only in inference procedure, we propose a Salient Point Clustering Optimization (SPCO) module to introduce a clustering procedure into the traini…
In this paper, we propose a novel deep neural network framework embedded with low-level features (LCNN) for salient object detection in complex images. We utilise the advantage of convolutional neural networks to automatically learn the high-level features that capture the structured information and semantic context in the image. In order to better adapt a CNN model into the saliency task, we redesign the network architecture based on the small-scale datasets. Several low-level features are extracted, which can effectively capture contrast and spatial information in the salient regions, and in…
Moving object detection is critical for automated video analysis in many vision-related tasks, such as surveillance tracking, video compression coding, etc. Robust Principal Component Analysis (RPCA), as one of the most popular moving object modelling methods, aims to separate the temporally varying (i.e., moving) foreground objects from the static background in video, assuming the background frames to be low-rank while the foreground to be spatially sparse. Classic RPCA imposes sparsity of the foreground component using l1-norm, and minimizes the modeling error via 2-norm. We show that such a…
Typically, objects with the same semantics are not always prominent in images containing different backgrounds. Motivated by this observation that accurately salient object detection is related to both foreground and background, we proposed a novel cooperative attention mechanism that jointly considers reciprocal relationships between background and foreground for efficient salient object detection. Concretely, we first aggregate the features at each side-out of traditional dilated FCN to extract the initial foreground and background local responses respectively. Then taking these responses as…
Sichuan University（四川大学）；National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University（四川大学视觉合成图形图像技术国家级重点实验室）；Nankai University（南开大学）；Wuhan University（武汉大学）；Inception Institute of Artificial Intelligence (IIAI)（起源人工智能研究院）；University of Electronic Science and Technology of China（电子科技大学）
Existing RGB-D salient object detection (SOD) models usually treat RGB and depth as independent information and design separate networks for feature extraction from each. Such schemes can easily be constrained by a limited amount of training data or over-reliance on an elaborately designed training process. Inspired by the observation that RGB and depth modalities actually present certain commonality in distinguishing salient objects, a novel joint learning and densely cooperative fusion (JL-DCF) architecture is designed to learn from both RGB and depth inputs through a shared network backbone…
Dalian University of Technology（大连理工大学）；The University of Adelaide（阿德莱德大学）
Salient object detection (SOD), which aims to find the most important region of interest and segment the relevant object/item in that area, is an important yet challenging vision task. This problem is inspired by the fact that human seems to perceive main scene elements with high priorities. Thus, accurate detection of salient objects in complex scenes is critical for human-computer interaction. In this paper, we present a novel feature learning framework for SOD, in which we cast the SOD as a pixel-wise classification problem. The proposed framework utilizes a densely hierarchical feature fus…
We present a systematic study on a new task called dichotomous image segmentation (DIS) , which aims to segment highly accurate objects from natural images. To this end, we collected the first large-scale DIS dataset, called DIS5K, which contains 5,470 high-resolution (e.g., 2K, 4K or larger) images covering camouflaged, salient, or meticulous objects in various backgrounds. DIS is annotated with extremely fine-grained labels. Besides, we introduce a simple intermediate supervision baseline (IS-Net) using both feature-level and mask-level guidance for DIS model training. IS-Net outperforms var…
RDVSv2: A Large-scale Benchmark for RGB-D Video Salient Object Detection
National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University（四川大学合成视觉基础科学国家重点实验室）；College of Computer Science, Sichuan University（四川大学计算机学院）
University of Electronic Science and Technology of China（电子科技大学）；Inception Institute of Artificial Intelligence (IIAI)（起源人工智能研究院）；CETC Special Mission Aircraft System Engineering Co.Ltd（中国电子科技集团特种任务飞机系统工程技术有限公司）
Object detection is an important task in remote sensing image analysis. To reduce the computational complexity of redundant information and improve the efficiency of image processing, visual saliency models have been widely applied in this field. In this paper, a novel saliency detection model based on Contrast-weighted Dictionary Learning (CDL) is proposed for remote sensing images. Specifically, the proposed CDL learns salient and non-salient atoms from positive and negative samples to construct a discriminant dictionary, in which a contrast-weighted term is proposed to encourage the contras…
G2HFNet: GeoGran-Aware Hierarchical Feature Fusion Network for Salient Object Detection in Optical Remote Sensing Images
School of Control Science and Engineering, Shandong University（控制科学与工程学院，山东大学）；State Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）；School of Automation, Hangzhou Dianzi University（自动化学院，杭州电子科技大学）；School of information engineering, Huzhou University（信息工程学院，湖州大学）；School of Data Science, Lingnan University（数据科学学院，岭南大学）
Unified-modal Salient Object Detection via Adaptive Prompt Learning
Anhui University（安徽大学）；School of Computer Science and Technology（计算机科学与技术学院）；School of Artificial Intelligence（人工智能学院）；Institute of Physical Science and Information Technology（物理科学与技术学院）
