CamoSAM2: SAM2-oriented Prompt Auto-Refinement for Video Camouflaged Object Detection
National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University（合成视觉基础科学国家重点实验室，四川大学）；College of Computer Science, Sichuan University（计算机学院，四川大学）
Segment Anything Model 2 (SAM2) 是一种提示引导的视频基础模型，在视频目标分割中表现出色，引起了社区的广泛关注。由于伪装目标与其周围环境高度相似，即使人眼也难以区分，因此SAM2在现实场景中自动分割的应用面临伪装感知和可靠提示生成的挑战。为了解决这些问题，我们提出了CamoSAM2，一个运动-外观提示诱导器（MAPI）和精化框架，用于自动生成和精化SAM2的提示，从而在VCOD任务中实现高质量的自动检测和分割。首先，我们引入了一个提示诱导器，它同时整合运动和外观线索来检测伪装目标，比现有方法提供更准确的初始预测。其次，我们提出了一种针对SAM2的基于视频的自适应多提示精化（AMPR）策略，旨在减轻初始粗糙掩码中的提示错误，并进一步生成良好的提示。具体来说，我们引入了一个新颖的三步过程，通过伪装目标确定、关键提示帧选择和多提示形成来生成可靠的提示。在两个基准数据集上进行的大量实验表明，我们提出的模型CamoSAM2显著优于现有最先进的方法，在mIoU指标上分别提高了8.0%和10.1%。此外，与当前的VCOD模型相比，我们的方法实现了最快的推理速度。
英文摘要：
The Segment Anything Model 2 (SAM2), a prompt-guided video foundation model, has remarkably performed in video object segmentation, drawing significant attention in the community. Due to the high similarity between camouflaged objects and their surroundings, which makes them difficult to distinguish even by the human eye, the application of SAM2 for automated segmentation in real-world scenarios faces challenges in camouflage perception and reliable prompts generation. To address these issues, we propose CamoSAM2, a motion-appearance prompt inducer (MAPI) and refinement framework to automatically generate and refine prompts for SAM2, enabling high-quality automatic detection and segmentation in VCOD task. Initially, we introduce a prompt inducer that simultaneously integrates motion and appearance cues to detect camouflaged objects, delivering more accurate initial predictions than existing methods. Subsequently, we propose a video-based adaptive multi-prompts refinement (AMPR) strategy tailored for SAM2, aimed at mitigating prompt error in initial coarse masks and further producing good prompts. Specifically, we introduce a novel three-step process to generate reliable prompts by camouflaged object determination, pivotal prompt frame selection, and multi-prompts formation. Extensive experiments conducted on two benchmark datasets demonstrate that our proposed model, CamoSAM2, significantly outperforms existing state-of-the-art methods, achieving increases of 8.0% and 10.1% in mIoU metric. Additionally, our method achieves the fastest inference speed compared to current VCOD models.
South China University of Technology（华南理工大学）；The Chinese University of Hong Kong（香港中文大学）；The Open University of Hong Kong（香港公开大学）
Camouflaged object detection is a challenging task that aims to identify objects having similar texture to the surroundings. This paper presents to amplify the subtle texture difference between camouflaged objects and the background for camouflaged object detection by formulating multiple texture-aware refinement modules to learn the texture-aware features in a deep convolutional neural network. The texture-aware refinement module computes the covariance matrices of feature responses to extract the texture information, designs an affinity loss to learn a set of parameter maps that help to sepa…
Toward Realistic Camouflaged Object Detection: Benchmarks and Method
School of Cyber Science and Engineering, Huazhong University of Science and Technology（华中科技大学网络空间科学与工程学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）；School of Computer Science & Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Intelligent Science and Technology Academy of CASIC（中国航天科工集团有限公司智能科技研究院）
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）
This paper focuses on camouflaged object detection (COD), which is a task to detect objects hidden in the background. Most of the current COD models aim to highlight the target object directly while outputting ambiguous camouflaged boundaries. On the other hand, the performance of the models considering edge information is not yet satisfactory. To this end, we propose a new framework that makes full use of multiple visual cues, i.e., saliency as well as edges, to refine the predicted camouflaged map. This framework consists of three key components, i.e., a pseudo-edge generator, a pseudo-map g…
Assisted Refinement Network Based on Channel Information Interaction for Camouflaged and Salient Object Detection
Department of Physics, J.K. Institute of Science（物理系，J.K.科学研究院）；World Scientific University（世界科学大学）；University of Intelligent Studies（智能研究大学）；School of Computer Science and Technology, Xinjiang University（计算机科学与技术学院，新疆大学）；Department of Electronic Engineering, Tsinghua University（电子工程系，清华大学）
DRRNet: Macro-Micro Feature Fusion and Dual Reverse Refinement for Camouflaged Object Detection
School of Computer Science and Engineering, Southeast University（计算机科学与工程学院，东南大学）；Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, China（计算机网络与信息集成重点实验室（东南大学），教育部，中国）
EviRCOD: Evidence-Guided Probabilistic Decoding for Referring Camouflaged Object Detection
West China Hospital, Sichuan University（四川大学华西医院）；Independent Researcher（独立研究员）；Department of Data Science and Artificial Intelligence, Auckland University of Technology（奥克兰理工大学数据科学与人工智能系）
High-Resolution Underwater Camouflaged Object Detection: GBU-UCOD Dataset and Topology-Aware and Frequency-Decoupled Networks
College of Shipbuilding Engineering, Harbin Engineering University（哈尔滨工程大学船舶工程学院）；School of Information Science and Technology, Great Bay University（广东工业大学信息科学与技术学院）
Certainty Is Redundant: Token Sparsification for Efficient Camouflaged Object Detection with Vision Foundation Models
VCIP, College of Computer Science, Nankai University（VCIP，计算机科学学院，南开大学）；Academy for Advanced Interdisciplinary Studies, Nankai University（先进交叉学科研究院，南开大学）；School of Computer Science and Engineering, Tianjin University of Technology（计算机科学与工程学院，天津工业大学）；School of Information and Communication Engineering, UESTC（信息与通信工程学院，电子科技大学）；Nankai International Advanced Research Institute, Shenzhen Futian（南开国际先进研究院，深圳福田）
C3Net: Context-Contrast Network for Camouflaged Object Detection
Computer Engineering Department, King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学计算机工程系）；Electronic Systems Engineering, University of Regina（里贾纳大学电子系统工程系）；Department of Computer Science and Software Engineering, The University of Western Australia（西澳大学计算机科学与软件工程系）
CamoFA: A Learnable Fourier-based Augmentation for Camouflage Segmentation
University of Science, Ho Chi Minh City（胡志明市科学大学）；Vietnam National University, Ho Chi Minh City（胡志明市越南国家大学）；Stony Brook University（石溪大学）；John von Neumann Institute（约翰·冯·诺依曼研究所）；University of Dayton（代顿大学）；Monash University（莫纳什大学）
ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection
Department of CMPS, University of British Columbia（计算机科学与软件工程系，不列颠哥伦比亚大学）；Group of Methane Emission Observation & Warning (MEOW) , Weathon Software（甲烷排放观测与预警小组，Weathon软件）
UESTC（电子科技大学）；Group 42 (G42)（G42集团）；Qingdao University（青岛大学）；Inception Institute of AI (IIAI)（Inception人工智能研究院）
Automatically detecting/segmenting object(s) that blend in with their surroundings is difficult for current models. A major challenge is that the intrinsic similarities between such foreground objects and background surroundings make the features extracted by deep model indistinguishable. To overcome this challenge, an ideal model should be able to seek valuable, extra clues from the given scene and incorporate them into a joint learning framework for representation co-enhancement. With this inspiration, we design a novel Mutual Graph Learning (MGL) model, which generalizes the idea of convent…
SPEGNet: Synergistic Perception-Guided Network for Camouflaged Object Detection
Computer Engineering Department, King Fahd University of Petroleum and Minerals（国王法赫德石油和矿物大学计算机工程系）；Department of Computer Science and Software Engineering, The University of Western Australia（西澳大学计算机科学与软件工程系）；Electronic Systems Engineering, University of Regina（里贾纳大学电子系统工程系）
CF-YOLO: Context-Aware Feature Refinement for Camouflaged Industrial Micro-Defect Detection
School of Information Engineering, Huzhou University（湖州大学信息工程学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）
A Survey of Camouflaged Object Detection and Beyond
Tsinghua University（清华大学）；Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Peking University（北京大学）；School of Electrical and Computer Engineering, Peking University（北京大学电气与计算机工程学院）；Duke University（杜克大学）
SRPR-Net: Semantic and Relational Prompt Refinement for Automated SAM-based Instance Segmentation
Ruijin-XJTLU Intelligent Medicine Institute, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（瑞金-西交利物浦智能医学研究所，瑞金医院，上海交通大学医学院）；School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）
SWNet: A Cross-Spectral Network for Camouflaged Weed Detection
ESPOL Polytechnic University（ESPOL多科技术大学）；Research Center for Information and Communication Technologies (CITIC-UGR)（信息与通信技术研究中心（CITIC-UGR））；University of Granada（格拉纳达大学）；Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）
Brigham Young University（杨百翰大学）；Adobe Research（Adobe 研究院）
As deep neural networks become the state-of-the-art approach in the field of computer vision for dense prediction tasks, many methods have been developed for automatic estimation of the target outputs given the visual inputs. Although the estimation accuracy of the proposed automatic methods continues to improve, interactive refinement is oftentimes necessary for further correction. Recently, feature backpropagating refinement scheme (f-BRS) has been proposed for the task of interactive segmentation, which enables efficient optimization of a small set of auxiliary variables inserted into the p…
ArgusCogito: Chain-of-Thought for Cross-Modal Synergy and Omnidirectional Reasoning in Camouflaged Object Segmentation
University of Chinese Academy of Sciences（中国科学院大学）；Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间利用技术与工程中心）
Camo-M3FD: A New Benchmark Dataset for Cross-Spectral Camouflaged Pedestrian Detection
ESPOL Polytechnic University（ESPOL理工大学）；Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心）；Software Engineering Department, Research Center for Information and Communication Technologies (CITIC-UGR), University of Granada（格拉纳达大学软件工程系，信息与通信技术研究中心（CITIC-UGR））；Università della Svizzera Italiana（瑞士意大利大学）
German Research Center for Artificial Intelligence（德国人工智能研究中心）；University of Kaiserslautern（凯泽斯劳滕大学）
The ability to create an accurate three-dimensional reconstruction of a captured scene draws attention to the principles of light fields. This paper presents an approach for light field camera calibration and rectification, based on pairwise pattern-based parameters extraction. It is followed by a correspondence-based algorithm for camera parameters refinement from arbitrary scenes using the triangulation filter and nonlinear optimization. The effectiveness of our approach is validated on both real and synthetic data.
The Hong Kong University of Science and Technology（香港科技大学）
We propose a 3D object detection system with multi-sensor refinement in the context of autonomous driving. In our framework, the monocular camera serves as the fundamental sensor for 2D object proposal and initial 3D bounding box prediction. While the stereo cameras and LiDAR are treated as adaptive plug-in sensors to refine the 3D box localization performance. For each observed element in the raw measurement domain (e.g., pixels for stereo, 3D points for LiDAR), we model the local geometry as an instance vector representation, which indicates the 3D coordinate of each element respecting to th…
University of Electronic Science and Technology of China（电子科技大学）；King’s College London（伦敦国王学院）；University Hospitals KU Leuven（鲁汶大学附属医院）；University College London（伦敦大学学院）；SenseTime Research（商汤科技研究院）
Segmentation of the fetal brain from stacks of motion-corrupted fetal MRI slices is important for motion correction and high-resolution volume reconstruction. Although Convolutional Neural Networks (CNNs) have been widely used for automatic segmentation of the fetal brain, their results may still benefit from interactive refinement for challenging slices. To improve the efficiency of interactive refinement process, we propose an Uncertainty-Guided Interactive Refinement (UGIR) framework. We first propose a grouped convolution-based CNN to obtain multiple automatic segmentation predictions with…
New York University（纽约大学）；Hunter College High School（亨特学院高中）；Beihang University（北京航空航天大学）；Riverside Research（河滨研究所）；Skirball Institute of Biomolecular Medicine, New York University（纽约大学斯科博尔生物分子医学研究所）
High-frequency ultrasound (HFU) is well suited for imaging embryonic mice due to its noninvasive and real-time characteristics. However, manual segmentation of the brain ventricles (BVs) and body requires substantial time and expertise. This work proposes a novel deep learning based end-to-end auto-context refinement framework, consisting of two stages. The first stage produces a low resolution segmentation of the BV and body simultaneously. The resulting probability map for each object (BV or body) is then used to crop a region of interest (ROI) around the target object in both the original i…
