使用视觉语言模型检测图像中的文本篡改
Detecting Text Manipulation in Images using Vision Language Models
AI总结：
本文研究了视觉语言模型在图像文本篡改检测中的应用，通过对比开源与闭源模型及专用VLMs在多种数据集上的表现，揭示了开源模型的差距及专用模型的泛化问题。
摘要：
近期研究表明，大型视觉语言模型（VLMs或LVLMs）在图像篡改检测中非常有效。然而，这些研究在很大程度上缺失了对文本篡改检测的探讨。我们通过在不同的文本篡改数据集上分析闭源和开源VLMs来弥补这一知识空白。结果表明，开源模型正逐步逼近但仍落后于如GPT-4o等闭源模型。此外，我们针对文本篡改检测对图像篡改检测专用的VLMs进行了基准测试，发现它们存在泛化问题。我们在自然场景文本和幻想身份证上对VLMs进行了篡改基准测试，后者模拟了一种极具挑战性的现实世界滥用场景。
英文摘要：
Recent works have shown the effectiveness of Large Vision Language Models (VLMs or LVLMs) in image manipulation detection. However, text manipulation detection is largely missing in these studies. We bridge this knowledge gap by analyzing closed- and open-source VLMs on different text manipulation datasets. Our results suggest that open-source models are getting closer, but still behind closed-source ones like GPT- 4o. Additionally, we benchmark image manipulation detection-specific VLMs for text manipulation detection and show that they suffer from the generalization problem. We benchmark VLMs for manipulations done on in-the-wild scene texts and on fantasy ID cards, where the latter mimic a challenging real-world misuse.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
