arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2504.04893cs.CVcs.AI
收藏

SCAM:多模态基础模型的现实世界字形鲁棒性评估

SCAM: A Real-World Typographic Robustness Evaluation for Multimodal Foundation Models

BLISS e.V.(BLISS协会);Berliner Hochschule für Technik (BHT)(柏林技术大学);Technische Universität Berlin(柏林技术大学);KI Werkstatt, Hochschule für Technik und Wirtschaft Berlin (HTW)(柏林技术经济学院);Merantix Momentum(Merantix Momentum公司);Fraunhofer Heinrich-Hertz-Institut, Berlin, Germany(柏林弗劳恩霍夫 Heinrich-Hertz 研究所)

Justus Westerhoff, Erblina Purelku, Jakob Hackstein, Jonas Loos, Leo Pinetzki, Erik Rodner, Lorenz Hufe

AI总结:

本文提出SCAM数据集,用于评估多模态基础模型对字形攻击的鲁棒性,发现训练数据和模型架构影响攻击敏感性,且大语言模型基座可降低攻击影响。

摘要:

字形攻击利用文本与视觉内容在多模态基础模型中的相互作用,导致图像中嵌入误导性文本时产生误分类。现有数据集规模和多样性有限,难以研究此类漏洞。本文介绍SCAM,目前最大且最多样化的现实世界字形攻击图像数据集,包含1162张图像,覆盖数百种物体类别和攻击词汇。通过广泛基准测试视觉-语言模型,证明字形攻击显著降低性能,并发现训练数据和模型架构影响攻击敏感性。研究结果表明,字形攻击仍有效对抗最先进的大型视觉-语言模型,尤其是那些固有易受此类攻击的视觉编码器模型。然而,使用更大的大型语言模型基座可降低这种脆弱性,同时增强字形理解。此外,我们证明合成攻击与现实世界(手写)攻击高度相似,验证了其在研究中的使用价值。本文提供全面资源和实证洞察,以促进未来多模态AI系统的鲁棒性和可信度研究。最后,我们公开发布本文介绍的数据集及评估代码,网址为www.bliss.berlin/research/scam。

英文摘要:

Typographic attacks exploit the interplay between text and visual content in multimodal foundation models, causing misclassifications when misleading text is embedded within images. Existing datasets are limited in size and diversity, making it difficult to study such vulnerabilities. In this paper, we introduce SCAM, the largest and most diverse dataset of real-world typographic attack images to date, containing 1162 images across hundreds of object categories and attack words. Through extensive benchmarking of Vision-Language Models on SCAM, we demonstrate that typographic attacks significantly degrade performance, and identify that training data and model architecture influence the susceptibility to these attacks. Our findings indicate that typographic attacks remain effective against state-of-the-art Large Vision-Language Models, especially those employing vision encoders inherently vulnerable to such attacks. However, employing larger Large Language Model backbones reduces this vulnerability while simultaneously enhancing typographic understanding. Additionally, we demonstrate that synthetic attacks closely resemble real-world (handwritten) attacks, validating their use in research. Our work provides a comprehensive resource and empirical insights to facilitate future research toward robust and trustworthy multimodal AI systems. Finally, we publicly release the datasets introduced in this paper, along with the code for evaluations under www.bliss.berlin/research/scam.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑