被谜题难住：当视觉语言模型无法领会暗示时
Puzzled by Puzzles: When Vision-Language Models Can't Take a Hint
AI总结：
本文通过构建手工标注的英语画谜基准，评估了当代视觉语言模型在多模态抽象与符号推理上的能力，发现其虽能解码简单视觉线索，但在抽象推理与视觉隐喻理解上仍存在显著困难。
摘要：
画谜是一种通过图像、空间排列和符号替换来编码语言的视觉谜题，对当前的视觉语言模型（VLMs）构成了独特挑战。与传统的图像描述或问答任务不同，解画谜需要多模态抽象、符号推理，以及对文化、语音和语言双关语的掌握。本文通过构建一个手工生成且带有标注的多样化英语画谜基准（涵盖从简单的象形文字替换到依赖空间的线索，如“head” over “heels”），调查了当代VLMs解释和解决画谜的能力。我们分析了不同VLMs的表现，研究结果表明，尽管VLMs在解码简单视觉线索方面展现出一些令人惊讶的能力，但它们在需要抽象推理、水平思考和视觉隐喻理解的任务上仍明显感到吃力。
英文摘要：
Rebus puzzles, visual riddles that encode language through imagery, spatial arrangement, and symbolic substitution, pose a unique challenge to current vision-language models (VLMs). Unlike traditional image captioning or question answering tasks, rebus solving requires multi-modal abstraction, symbolic reasoning, and a grasp of cultural, phonetic and linguistic puns. In this paper, we investigate the capacity of contemporary VLMs to interpret and solve rebus puzzles by constructing a hand-generated and annotated benchmark of diverse English-language rebus puzzles, ranging from simple pictographic substitutions to spatially-dependent cues ("head" over "heels"). We analyze how different VLMs perform, and our findings reveal that while VLMs exhibit some surprising capabilities in decoding simple visual clues, they struggle significantly with tasks requiring abstract reasoning, lateral thinking, and understanding visual metaphors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
