利用来自 SA-1B 的大量训练数据,Segment Anything Model(SAM)展现出了卓越的泛化能力和零样本能力,在医学图像分割和遥感图像分割等领域引起了广泛关注。然而,其在图像篡改检测领域的性能在很大程度上仍未被探索和证实。将 SAM 应用于图像篡改检测存在两个主要挑战:a)依赖手动提示,以及 b)单一视图信息难以支持跨数据集泛化。为了应对这些挑战,我们开发了一种基于 SAM 的跨视图提示学习范式,称为 IMDPrompter。得益于自动提示的设计,IMDPrompter 不再依赖手动指导,实现了自动化检测与定位。此外,我们提出了跨视图特征感知、最优提示选择和跨视图提示一致性等组件,这些组件促进了跨视图感知学习并引导 SAM 生成准确的掩码。来自五个数据集(CASIA、Columbia、Coverage、IMD2020 和 NIST16)的大量实验结果验证了我们所提出方法的有效性。
英文摘要:
Using extensive training data from SA-1B, the Segment Anything Model (SAM) has demonstrated exceptional generalization and zero-shot capabilities, attracting widespread attention in areas such as medical image segmentation and remote sensing image segmentation. However, its performance in the field of image manipulation detection remains largely unexplored and unconfirmed. There are two main challenges in applying SAM to image manipulation detection: a) reliance on manual prompts, and b) the difficulty of single-view information in supporting cross-dataset generalization. To address these challenges, we develops a cross-view prompt learning paradigm called IMDPrompter based on SAM. Benefiting from the design of automated prompts, IMDPrompter no longer relies on manual guidance, enabling automated detection and localization. Additionally, we propose components such as Cross-view Feature Perception, Optimal Prompt Selection, and Cross-View Prompt Consistency, which facilitate cross-view perceptual learning and guide SAM to generate accurate masks. Extensive experimental results from five datasets (CASIA, Columbia, Coverage, IMD2020, and NIST16) validate the effectiveness of our proposed method.
Scalpel-SAM: A Semi-Supervised Paradigm for Adapting SAM to Infrared Small Object Detection
School of Computer Science, Chengdu University(成都大学计算机科学学院);China University of Geosciences(中国地质大学);Zhejiang University(浙江大学);Intelligent Computing Infrastructure Innovation Center, Zhejiang Lab(浙江实验室智能计算基础设施创新中心)
ASLseg: Adapting SAM in the Loop for Semi-supervised Liver Tumor Segmentation
Southern University of Science and Technology(南方科技大学);The University of Hong Kong(香港大学);Jiaxing Research Institute, Southern University of Science and Technology(南方科技大学嘉兴研究院)
SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, Medical Image Segmentation, and More
KOKONI, Moxin (Huzhou) Tech. Co., LTD(莫西(湖州)科技有限公司KOKONI);Beihang University(北京航空航天大学);Zhejiang University(浙江大学);Huzhou University(湖州师范学院);Singapore University of Technology and Design(新加坡科技设计大学)
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
University of Oxford(牛津大学);National University of Singapore(新加坡国立大学);University of Alberta(阿尔伯塔大学);A*STAR(新加坡科技研究局);Carnegie Mellon University(卡内基梅隆大学);Singapore Eye Research Institute(新加坡眼科研究所);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University of Hongkong(香港大学)
本文提出Medical SAM Adapter(Med-SA),通过轻量适配技术将医学知识融入SAM,利用SD-Trans和HyP-Adpt适配2D模型至3D医学图像,在17个任务上以仅2%参数更新超越SOTA方法。
Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images
Information Engineering University(信息工程大学);Nanjing University of Information Science and Technology(南京信息工程大学);ETH Zurich(苏黎世联邦理工学院);University of Trento(特伦托大学)
SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More
KOKONI, Moxin (Huzhou) Tech. Co., LTD(科科尼(湖州)魔芯科技有限公司);School of Information Engineering, Huzhou University(湖州大学信息工程学院);School of Information Science and Technology, University of Science and Technology of China(中国科学技术大学信息科学技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Information Systems Technology and Design Pillar, Singapore University of Technology and Design(新加坡科技设计大学信息系统技术与设计支柱学院)
Adapting SAM to Nuclei Instance Segmentation and Classification via Cooperative Fine-Grained Refinement
organization= College of Computer;Data Science , addressline= Fuzhou University , city= Fuzhou , country= China;organization= School of Mechanical Engineering;Automation , addressline= Fuzhou University , city= Fuzhou , country= China;organization= College of Physics;Information Engineering , addressline= Fuzhou University , city= Fuzhou , country= China
Adapting SAM with Dynamic Similarity Graphs for Few-Shot Parameter-Efficient Small Dense Object Detection: A Case Study of Chickpea Pods in Field Conditions
McGill University(麦吉尔大学);Michigan State University(密歇根州立大学)
Adapting Medical Vision Foundation Models for Volumetric Medical Image Segmentation via Active Learning and Selective Semi-supervised Fine-tuning
Mallinckrodt Institute of Radiology(马林克罗德放射医学研究所);Institute for Informatics, Data Science and Biostatistics(信息学、数据科学与生物统计学研究所);Department of Radiology, Perelman School of Medicine(放射科,佩尔曼医学院);Center for AI and Data Science for Integrated Diagnostics (AI2D)(集成诊断人工智能与数据科学中心(AI2D))
Adapting SAM 2 for Visual Object Tracking: 1st Place Solution for MMVPR Challenge Multi-Modal Tracking
University of Washington(华盛顿大学);Electronics and Telecommunications Research Institute(电子电信研究所);National Center for High-performance Computing(高性能计算国家中心)
SAMamba3D: adapting Segment Anything for generalizable 3D segmentation of multiphase pore-scale images
Department of Earth Science and Engineering, Imperial College London(帝国理工学院伦敦地球科学与工程系);College of Artificial Intelligence, China University of Petroleum (Beijing)(中国石油大学(北京)人工智能学院);College of Petroleum Engineering, China University of Petroleum (Beijing)(中国石油大学(北京)石油工程学院)
SAMRI-3D: Adapting SAM2 for 3D MRI Segmentation with Global Volume Tokens
School of Electrical Engineering and Computer Science, The University of Queensland(电气工程与计算机科学学院,昆士兰大学);School of Engineering, College of Engineering, Science and Environment, University of Newcastle(工程学院,工程、科学与环境学院,纽卡斯尔大学);School of Human Movement and Nutrition Sciences, The University of Queensland(人类运动与营养科学学院,昆士兰大学)
Adapting a Segmentation Foundation Model for Medical Image Classification
Department of Computer Science, University of Texas Rio Grande Valley(德克萨斯理工大学里奥格兰德分校计算机科学系);Department of Computer Science, Texas Woman’s University(德克萨斯女性大学计算机科学系);Sewickley Academy(塞维克利学院);Department of Mathematical Sciences, The University of Texas at Dallas(德克萨斯大学达拉斯分校数学科学系);Department of Computer Science and Engineering, University of Notre Dame(诺特丹大学计算机科学与工程系)
本文提出将 SAM 图像编码器作为冻结特征提取器,并设计空间局部化通道注意力机制 SLCA,将其集成到深度学习分类模型中,以增强对关键区域的关注,在三个公开医学图像分类数据集上验证了方法的有效性和数据高效性。
MPG-SAM 2: Adapting SAM 2 with Mask Priors and Global Context for Referring Video Object Segmentation
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University(国家多媒体软件工程研究中心,计算机学院,武汉大学);Hong Kong University of Science and Technology(香港科学与技术大学);Horizon Robotics
Adapting Segment Anything Model for Power Transmission Corridor Hazard Segmentation
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence(计算机学院、多媒体软件国家工程研究中心、人工智能研究所);Hubei Key Laboratory of Multimedia(湖北多媒体重点实验室);Network Communication Engineering, Wuhan University, Wuhan, China(网络通信工程、武汉大学、中国武汉);Institute of Power Transmission(输变电研究所);Institute of Power Transmission and Transformation Technology, State Grid Zhejiang Electric Power Co., LTD, Research Institute Hangzhou, China(输变电技术研究所、国家电网浙江电力有限公司、杭州研究院、中国)
Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation in Medical Images: An Experimental Study
The Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心放射肿瘤学系医学人工智能与自动化(MAIA)实验室);Medical Image Processing Group, Department of Radiology, University of Pennsylvania(宾夕法尼亚大学放射学系医学图像处理组);Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心内科学系消化与肝脏疾病科)
RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机学院);School of Computer Science, The University of Sydney(悉尼大学计算机学院)
Enhancing and Adapting in the Clinic: Source-free Unsupervised Domain Adaptation for Medical Image Enhancement
Southern University of Science and Technology(南方科技大学);Agency for Science, Technology and Research(新加坡科技研究局);Beijing Information Science and Technology University(北京信息科技大学);University of Science and Technology Beijing(北京科技大学)