arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.02454cs.CV
收藏

IMDPrompter:通过跨视图自动提示学习将 SAM 适配于图像篡改检测

IMDPrompter: Adapting SAM to Image Manipulation Detection by Cross-View Automated Prompt Learning

Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学)

Quan Zhang, Yuxin Qi, Xi Tang, Jinwei Fang, Xi Lin, Ke Zhang, Chun Yuan

AI总结:

本文提出基于SAM的跨视图自动提示学习范式IMDPrompter,通过设计跨视图特征感知、最优提示选择等组件实现自动化检测与定位,有效解决了SAM在图像篡改检测中依赖手动提示和跨数据集泛化难的问题。

摘要:

利用来自 SA-1B 的大量训练数据,Segment Anything Model(SAM)展现出了卓越的泛化能力和零样本能力,在医学图像分割和遥感图像分割等领域引起了广泛关注。然而,其在图像篡改检测领域的性能在很大程度上仍未被探索和证实。将 SAM 应用于图像篡改检测存在两个主要挑战:a)依赖手动提示,以及 b)单一视图信息难以支持跨数据集泛化。为了应对这些挑战,我们开发了一种基于 SAM 的跨视图提示学习范式,称为 IMDPrompter。得益于自动提示的设计,IMDPrompter 不再依赖手动指导,实现了自动化检测与定位。此外,我们提出了跨视图特征感知、最优提示选择和跨视图提示一致性等组件,这些组件促进了跨视图感知学习并引导 SAM 生成准确的掩码。来自五个数据集(CASIA、Columbia、Coverage、IMD2020 和 NIST16)的大量实验结果验证了我们所提出方法的有效性。

英文摘要:

Using extensive training data from SA-1B, the Segment Anything Model (SAM) has demonstrated exceptional generalization and zero-shot capabilities, attracting widespread attention in areas such as medical image segmentation and remote sensing image segmentation. However, its performance in the field of image manipulation detection remains largely unexplored and unconfirmed. There are two main challenges in applying SAM to image manipulation detection: a) reliance on manual prompts, and b) the difficulty of single-view information in supporting cross-dataset generalization. To address these challenges, we develops a cross-view prompt learning paradigm called IMDPrompter based on SAM. Benefiting from the design of automated prompts, IMDPrompter no longer relies on manual guidance, enabling automated detection and localization. Additionally, we propose components such as Cross-view Feature Perception, Optimal Prompt Selection, and Cross-View Prompt Consistency, which facilitate cross-view perceptual learning and guide SAM to generate accurate masks. Extensive experimental results from five datasets (CASIA, Columbia, Coverage, IMD2020, and NIST16) validate the effectiveness of our proposed method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑