当有害内容被伪装时:通过CamHarmTI揭示LVLMs的感知失败
When Harmful Content Gets Camouflaged: Unveiling Perception Failure of LVLMs with CamHarmTI
AI总结:
本文提出CamHarmTI基准,揭示LVLMs在识别伪装有害内容时的感知不足,并通过微调提升模型性能。
摘要:
大型视觉-语言模型(LVLMs)越来越多地用于需要检测多模态有害内容的任务,如在线内容审查。然而,现实中的有害内容往往被伪装,依赖于微妙的文本-图像交互,例如隐含恶意文本的图片或漫画,以逃避检测。这引发了一个关键问题:LVLMs能否像人类一样敏感地感知此类伪装的有害内容?在本文中,我们引入了CamHarmTI,一个用于评估LVLMs在文本-图像组合中感知和解释伪装有害内容能力的基准。CamHarmTI包含超过4500个样本,涵盖三种类型的图文帖子。在100名人类用户和12种主流LVLMs上的实验揭示了一个明显的感知差距:人类能够轻松识别此类内容(例如,准确率超过95.75%),而当前的LVLMs经常失败(例如,ChatGPT-4o仅达到2.10%的准确率)。此外,微调实验表明,CamHarmTI为提高模型感知能力提供了有效的资源,使Qwen2.5VL-7B的准确率提高了55.94%。注意力分析和逐层探测进一步揭示,微调主要增强了视觉编码器的早期层,促进更集成的场景理解。这些发现突显了LVLMs固有的感知限制,并为更符合人类视觉推理系统的系统提供了见解。
英文摘要:
Large vision-language models (LVLMs) are increasingly used for tasks where detecting multimodal harmful content is crucial, such as online content moderation. However, real-world harmful content is often camouflaged, relying on nuanced text-image interplay, such as memes or images with embedded malicious text, to evade detection. This raises a key question: \textbf{can LVLMs perceive such camouflaged harmful content as sensitively as humans do?} In this paper, we introduce CamHarmTI, a benchmark for evaluating LVLM ability to perceive and interpret camouflaged harmful content within text-image compositions. CamHarmTI consists of over 4,500 samples across three types of image-text posts. Experiments on 100 human users and 12 mainstream LVLMs reveal a clear perceptual gap: humans easily recognize such content (e.g., over 95.75\% accuracy), whereas current LVLMs often fail (e.g., ChatGPT-4o achieves only 2.10\% accuracy). Moreover, fine-tuning experiments demonstrate that \bench serves as an effective resource for improving model perception, increasing accuracy by 55.94\% for Qwen2.5VL-7B. Attention analysis and layer-wise probing further reveal that fine-tuning enhances sensitivity primarily in the early layers of the vision encoder, promoting a more integrated scene understanding. These findings highlight the inherent perceptual limitations in LVLMs and offer insight into more human-aligned visual reasoning systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。