仙境中的Bongard：仍令AI抓狂的视觉谜题？
Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?
AI总结：
本文通过经典的Bongard视觉推理谜题评估视觉语言模型，发现其在模式识别与抽象推理上表现不佳，揭示了机器认知与人类视觉推理能力间仍存在显著差距。
摘要：
近期，诸如OpenAI的o1等新开发的视觉语言模型（VLM）不断涌现，似乎展现出了跨文本与图像模态的高级推理能力。然而，这些模型在语言引导感知与抽象推理方面的进展深度仍未被充分探索，且它们能否真正兑现其宏大承诺尚不明确。为评估进展并识别不足，我们进入了Bongard问题的仙境，这是一组经典的视觉推理谜题，需要类似人类的模式识别与抽象推理能力。通过我们广泛的评估设置，我们表明，尽管VLM偶尔能成功识别判别性概念并解决部分问题，但它们经常出错。令人惊讶的是，即便是人类看来可能微不足道的基本概念（如简单的螺旋线），也构成了重大挑战。此外，当被明确要求识别真值概念时，它们依然出错，这表明它们不仅缺乏对这些基本视觉概念的理解，而且无法泛化到未见过的概念。我们将VLM的结果与人类表现进行了比较，观察到人类视觉推理能力与机器认知之间仍存在显著差距。
英文摘要：
Recently, newly developed Vision-Language Models (VLMs), such as OpenAI's o1, have emerged, seemingly demonstrating advanced reasoning capabilities across text and image modalities. However, the depth of these advances in language-guided perception and abstract reasoning remains underexplored, and it is unclear whether these models can truly live up to their ambitious promises. To assess the progress and identify shortcomings, we enter the wonderland of Bongard problems, a set of classic visual reasoning puzzles that require human-like abilities of pattern recognition and abstract reasoning. With our extensive evaluation setup, we show that while VLMs occasionally succeed in identifying discriminative concepts and solving some of the problems, they frequently falter. Surprisingly, even elementary concepts that may seem trivial to humans, such as simple spirals, pose significant challenges. Moreover, when explicitly asked to recognize ground truth concepts, they continue to falter, suggesting not only a lack of understanding of these elementary visual concepts but also an inability to generalize to unseen concepts. We compare the results of VLMs to human performance and observe that a significant gap remains between human visual reasoning capabilities and machine cognition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
