Eye-Q：一个多语言视觉词谜解决和图像到短语推理的基准
Eye-Q: A Multilingual Benchmark for Visual Word Puzzle Solving and Image-to-Phrase Reasoning
AI总结：
Eye-Q提出一个多语言视觉词谜基准，评估模型在复杂视觉推理中的能力，发现现有模型在抽象和跨语言推理上存在显著差距。
摘要：
视觉语言模型（VLMs）在标准视觉语言基准上取得了强劲表现，但往往依赖于表面识别而非深层推理。我们提出视觉词谜作为具有挑战性的替代方案，因为它们需要发现隐含的视觉线索，生成并修订假设，并将感知证据映射到非字面概念，这些任务难以通过字面接地、OCR密集的捷径或简单的检索式匹配来解决。我们引入了Eye-Q，一个多语言基准，旨在评估这种复杂的视觉理解形式。Eye-Q包含1,343个谜题，其中模型观察到一个概念密集的场景，伴有简短的描述，必须推断出特定的目标词或短语。谜题有意无结构且线索隐含，具有干扰项和上下文关系，要求选择性注意、抽象和联想推理。该基准涵盖英语、波斯语、阿拉伯语和跨语言谜题。我们使用开放式、与人类对齐的协议评估最先进的VLMs，该协议探测在轻量级辅助下假设的形成和修订。结果揭示了显著的性能差距，尤其是在抽象和跨语言谜题上，突显了当前模型在构建和搜索适当概念表示以进行灵活的图像到短语推断方面的局限性；最大准确率达到仅60.27%。
英文摘要：
Vision-Language Models (VLMs) have achieved strong performance on standard vision-language benchmarks, yet often rely on surface-level recognition rather than deeper reasoning. We propose visual word puzzles as a challenging alternative, as they require discovering implicit visual cues, generating and revising hypotheses, and mapping perceptual evidence to non-literal concepts in ways that are difficult to solve via literal grounding, OCR-heavy shortcuts, or simple retrieval-style matching. We introduce Eye-Q, a multilingual benchmark designed to assess this form of complex visual understanding. Eye-Q contains 1,343 puzzles in which a model observes a conceptually dense scene with a brief description and must infer a specific target word or phrase. The puzzles are intentionally unstructured and cue-implicit, with distractors and contextual relationships that demand selective attention, abstraction, and associative inference. The benchmark spans English, Persian, Arabic, and cross-lingual puzzles. We evaluate state-of-the-art VLMs using an open-ended, human-aligned protocol that probes hypothesis formation and revision under lightweight assistance. Results reveal substantial performance gaps, especially on abstract and cross-lingual puzzles, highlighting limitations in current models' ability to construct and search over appropriate conceptual representations for flexible image-to-phrase inference; maximum accuracy reaches only 60.27%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
