AI 中文总结
研究针对YouTube等平台误导性缩略图问题，提出用大语言模型的多模态检测管道，构建数据集并评估多个模型，发现Claude 3.5 Sonnet性能突出，通过失败分析为视频平台发展提供方向。
AI 中文摘要
YouTube等平台上误导性视频缩略图是个普遍问题，破坏用户信任和平台诚信。本文提出用大语言模型标记误导性缩略图的多模态检测管道。构建含2843个视频的数据集，整合视频描述、缩略图和字幕进行分析。评估四个前沿大语言模型及两个视觉语言模型，发现大语言模型识别误导性缩略图有效，Claude 3.5 Sonnet性能强劲。还进行失败分析，讨论结果对内容审核等的影响，为视频平台发展提供方向。
英文摘要
Misleading video thumbnails on platforms like YouTube are a pervasive problem, undermining user trust and platform integrity. This paper proposes a novel multi-modal detection pipeline that uses Large Language Models (LLMs) to flag misleading thumbnails. We first construct a comprehensive dataset of 2,843 videos from eight countries, including 1,359 misleading thumbnail videos that collectively amassed over 7.6 billion views, providing a unique cross-cultural perspective on this global issue. Our detection pipeline integrates video-to-text descriptions, thumbnail images, and subtitle transcripts to holistically analyze content and flag misleading thumbnails. Through extensive experimentation and prompt engineering, we evaluate the performance of four frontier-level LLMs, including GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet, and Gemini-1.5 Flash. We further evaluate open-weight vision-language models, LLaVA-v1.5 and Qwen2.5-VL-7B-Instruct, to assess the generalizability of our approach beyond proprietary systems. Our findings show the effectiveness of LLMs in identifying misleading thumbnails, with Claude 3.5 Sonnet consistently showing strong performance, achieving an accuracy of 93.8%, precision over 92%, and recall exceeding 94% in certain scenarios. Beyond evaluating detection performance, we conducted a careful failure analysis to understand when LLMs fail in identifying misleading thumbnails. We discuss the implications of our findings for content moderation, user experience, and the ethical considerations of deploying such systems at scale. Our findings pave the way for more transparent, trustworthy video platforms and stronger content integrity for audiences worldwide.
CommentsAccepted to the 21st International AAAI Conference on Web and Social Media (ICWSM 2027)
Journal refProceedings of the International AAAI Conference on Web and Social Media, 2027
