arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.20193cs.CVcs.AIcs.LG

从遮罩到像素与意义:一种新的分类、基准和度量标准用于VLM图像篡改

From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tampering

Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);University College London(伦敦大学学院)

Xinyi Shang, Yi Tang, Jiacheng Cui, Ahmed Elhagry, Salwa K. Al Khatib, Sondos Mahmoud Bsharat, Jiacheng Liu, Xiaohan Zhao, Jing-Hao Xue, Hao Li, Salman Khan, Zhiqiang Shen

AI总结:

本文提出了一种基于像素和意义的VLM图像篡改检测新方法,引入了新的分类体系和基准,提出了基于像素的度量标准,并评估了现有模型在微编辑和非遮罩篡改上的表现。

摘要:

现有的图像篡改检测基准大多依赖于对象遮罩,这与真实的编辑信号严重不一致:许多遮罩内的像素未被修改或仅轻微修改,而遮罩外的微妙但重要的编辑则被视作自然。我们重新定义了VLM图像篡改任务,从粗略的区域标签转向基于像素的、具有意义和语言意识的任务。首先,我们引入了一种涵盖编辑基本原理(替换/删除/拼接/修补/属性/着色等)及其语义类别篡改对象的分类体系,将低层次的变化与高层次的理解联系起来。其次,我们发布了一个新的基准,包含每像素篡改图谱和配对的类别监督,以评估检测和分类在统一协议下的表现。第三,我们提出了一个训练框架和评估指标,通过量化像素级别的正确性并定位以评估置信度或对真实编辑强度的预测,并进一步通过语义感知分类和自然语言描述来衡量篡改意义的理解。我们还重新评估了现有强大的分割/定位基线在最新强大的篡改检测器上的表现,并揭示了使用遮罩-only度量标准时的显著过评分和低估问题,并暴露了在微编辑和非遮罩变化上的失败模式。我们的框架将该领域从遮罩推进到像素、意义和语言描述,建立了篡改定位、语义分类和描述的严格标准。代码和基准数据可在https://github.com/VILA-Lab/PIXAR上获得。

英文摘要:

Existing tampering detection benchmarks largely rely on object masks, which severely misalign with the true edit signal: many pixels inside a mask are untouched or only trivially modified, while subtle yet consequential edits outside the mask are treated as natural. We reformulate VLM image tampering from coarse region labels to a pixel-grounded, meaning and language-aware task. First, we introduce a taxonomy spanning edit primitives (replace/remove/splice/inpaint/attribute/colorization, etc.) and their semantic class of tampered object, linking low-level changes to high-level understanding. Second, we release a new benchmark with per-pixel tamper maps and paired category supervision to evaluate detection and classification within a unified protocol. Third, we propose a training framework and evaluation metrics that quantify pixel-level correctness with localization to assess confidence or prediction on true edit intensity, and further measure tamper meaning understanding via semantics-aware classification and natural language descriptions for the predicted regions. We also re-evaluate the existing strong segmentation/localization baselines on recent strong tamper detectors and reveal substantial over- and under-scoring using mask-only metrics, and expose failure modes on micro-edits and off-mask changes. Our framework advances the field from masks to pixels, meanings and language descriptions, establishing a rigorous standard for tamper localization, semantic classification and description. Code and benchmark data are available at https://github.com/VILA-Lab/PIXAR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑