SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More
KOKONI, Moxin (Huzhou) Tech. Co., LTD（科科尼（湖州）魔芯科技有限公司）；School of Information Engineering, Huzhou University（湖州大学信息工程学院）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Information Systems Technology and Design Pillar, Singapore University of Technology and Design（新加坡科技设计大学信息系统技术与设计支柱学院）
Tianrun Chen, Ankang Lu, Lanyun Zhu, Chaotao Ding, Chunan Yu, Deyi Ji, Zejian Li, Lingyun Sun, Papa Mao, Ying Zang
备注：
arXiv admin note: text overlap with arXiv:2304.09148
The advent of large models, also known as foundation models, has significantly transformed the AI research landscape, with models like Segment Anything (SAM) achieving notable success in diverse image segmentation scenarios. Despite its advancements, SAM encountered limitations in handling some complex low-level segmentation tasks like camouflaged object and medical imaging. In response, in 2023, we introduced SAM-Adapter, which demonstrated improved performance on these challenging tasks. Now, with the release of Segment Anything 2 (SAM2), a successor with enhanced architecture and a larger training corpus, we reassess these challenges. This paper introduces SAM2-Adapter, the first adapter designed to overcome the persistent limitations observed in SAM2 and achieve new state-of-the-art (SOTA) results in specific downstream tasks including medical image segmentation, camouflaged (concealed) object detection, and shadow detection. SAM2-Adapter builds on the SAM-Adapter's strengths, offering enhanced generalizability and composability for diverse applications. We present extensive experimental results demonstrating SAM2-Adapter's effectiveness. We show the potential and encourage the research community to leverage the SAM2 model with our SAM2-Adapter for achieving superior segmentation outcomes. Code, pre-trained models, and data processing protocols are available at http://tianrun-chen.github.io/SAM-Adaptor/
SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentation
KOKONI, Moxin (Huzhou) Tech. Co., LTD（摩西（湖州）科技有限公司）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Information Engineering, Huzhou University（湖州大学信息工程学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（新加坡南洋理工大学电子与电气工程学院）；College of Computing and Data Science, Nanyang Technological University（新加坡南洋理工大学计算与数据科学学院）；School of Information Science and Technology, University of Science and Technology of China（中国科学技术大学信息科学与技术学院）
SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More
KOKONI, Moxin (Huzhou) Tech. Co., LTD（莫西（湖州）科技有限公司KOKONI）；Beihang University（北京航空航天大学）；Zhejiang University（浙江大学）；Huzhou University（湖州师范学院）；Singapore University of Technology and Design（新加坡科技设计大学）
SAM-TTT: Segment Anything Model via Reverse Parameter Configuration and Test-Time Training for Camouflaged Object Detection
College of Computer Science and Artificial Intelligence, Wenzhou University（计算机科学与人工智能学院，温州大学）；College of Computer Science and Technology, Tongji University（计算机科学与技术学院，同济大学）；College of Information Science and Technology, Zhejiang Shuren University（信息科学与技术学院，浙江师大）
本文提出 SAM-TTT 模型，通过反向参数配置模块抑制不利参数，并引入 T-Visioner 模块利用测试时训练层强化有利参数，显著提升了 SAM 在伪装目标检测任务中的语义理解能力，并在多个基准上达到最先进性能。
Frequency-Guided Spatial Adaptation for Camouflaged Object Detection
National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology（空天地一体化大数据应用技术国家工程实验室）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；School of Information and Technology, Northwest University（西北大学信息与技术学院）；Xi’an Key Laboratory of Human-Machine Integration and Control Technology for Intelligent Rehabilitation, Xijing University（西京学院智能康复人机融合控制技术西安市重点实验室）
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
University of Oxford（牛津大学）；National University of Singapore（新加坡国立大学）；University of Alberta（阿尔伯塔大学）；A*STAR（新加坡科技研究局）；Carnegie Mellon University（卡内基梅隆大学）；Singapore Eye Research Institute（新加坡眼科研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Hongkong（香港大学）
本文提出Medical SAM Adapter（Med-SA），通过轻量适配技术将医学知识融入SAM，利用SD-Trans和HyP-Adpt适配2D模型至3D医学图像，在17个任务上以仅2%参数更新超越SOTA方法。
Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection
Information Science and Technology College, Dalian Maritime University（大连海事大学信息科学与技术学院）；Beijing University of Technology（北京理工大学）；Dalian University of Technology（大连理工大学）
Expose Camouflage in the Water: Underwater Camouflaged Instance Segmentation and Dataset
School of Information and Communication Engineering, Hainan University（海南大学信息与通信工程学院）；School of Electronic and Information Engineering, Guangdong Ocean University（广东海洋大学电子与信息工程学院）；School of Computer Science and Technology, Hainan University（海南大学计算机科学与技术学院）；School of Computer Science, Nankai University（南开大学计算机科学学院）；Institute of Software, Chinese Academy of Science（中国科学院软件研究所）；School of Data Science, Lingnan University（岭南大学数据科学学院）
CamoSAM2: SAM2-oriented Prompt Auto-Refinement for Video Camouflaged Object Detection
National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University（合成视觉基础科学国家重点实验室，四川大学）；College of Computer Science, Sichuan University（计算机学院，四川大学）
ZS-VCOS: Zero-Shot Video Camouflaged Object Segmentation By Optical Flow and Open Vocabulary Object Detection
Department of CMPS, University of British Columbia（计算机科学与软件工程系，不列颠哥伦比亚大学）；Group of Methane Emission Observation & Warning (MEOW) , Weathon Software（甲烷排放观测与预警小组，Weathon软件）
Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation
National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；Medical Artificial Intelligence Research Institute of Renmin Hospital, Wuhan University（武汉大学人民医院医学人工智能研究院）；School of Computer Science, Central China Normal University（华中师范大学计算机学院）
SAM3-UNet: Simplified Adaptation of Segment Anything Model 3
Sun Yat-sen University（中山大学）；Jiangxi Normal University（江西师范大学）；Hainan University（海南大学）；Dalian University of Technology（大连理工大学）；Chizhou University（池州学院）；Wuhan University（武汉大学）
Compositional Oil Spill Detection Based on Object Detector and Adapted Segment Anything Model from SAR Images
The Hong Kong Polytechnic University（香港理工大学）；Research Institute for Sustainable Urban Development, The Hong Kong Polytechnic University（香港理工大学可持续城市发展研究院）
SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentation
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Computer and Information Engineering, Jiangxi Normal University（江西师范大学计算机与信息工程学院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Tianjin University（天津大学）；Monash University（墨尔本大学）；Pazhou Lab（琶洲实验室）；Smart Hospital Research Institute, Peking University Shenzhen Hospital（北京大学深圳医院智能医院研究所）
TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentation
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Wuhan University（武汉大学）；Dalian University of Technology（大连理工大学）；University of Chinese Academy of Sciences（中国科学院大学）
Multi-scale Contrastive Adaptor Learning for Segmenting Anything in Underperformed Scenes
School of Automation and Electrical Engineering, University of Science and Technology Beijing（北京科技大学自动化与电气工程学院）；Alibaba DAMO Academy（阿里巴巴达摩院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
CAT-SAM: Conditional Tuning for Few-Shot Adaptation of Segment Anything Model
The University of Tokyo（东京大学）；Nara Institute of Science and Technology（奈良科学技术研究所）；Nanyang Technological University（南洋理工大学）；Waseda University（早稻田大学）；Zhejiang University of Technology（浙江工业大学）；University of Chinese Academy of Sciences（中国科学院大学）；RIKEN AIP（理化学研究所人工智能项目）
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Beihang University（北京航空航天大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
