从博弈论视角诊断视觉语言模型的组合知识
Diagnosing the Compositional Knowledge of Vision Language Models from a Game-Theoretic View
AI总结：
本文从博弈论视角提出诊断方法，系统评估视觉语言模型在组合推理（如关系与属性）上的脆弱性，揭示其能力不足的原因，并为未来研究提供可靠指导。
摘要：
组合推理能力通常被认为是表征人类感知的基本技能。近期研究表明，当前的视觉语言模型（VLMs）令人惊讶地缺乏关于此类能力的足够知识。为此，我们提出从博弈论的新视角彻底诊断VLMs编码的组合表示，系统揭示这一弱点的潜在原因。具体而言，我们提出了评估方法，以评估VLMs在组合理解的不同方面（如关系和属性）的脆弱性。大量实验结果验证了关于VLMs在组合推理上能力不足的若干见解，为未来研究提供了有用且可靠的指导。相关成果将在https://vlms-compositionality-gametheory.github.io/更新。
英文摘要：
Compositional reasoning capabilities are usually considered as fundamental skills to characterize human perception. Recent studies show that current Vision Language Models (VLMs) surprisingly lack sufficient knowledge with respect to such capabilities. To this end, we propose to thoroughly diagnose the composition representations encoded by VLMs, systematically revealing the potential cause for this weakness. Specifically, we propose evaluation methods from a novel game-theoretic view to assess the vulnerability of VLMs on different aspects of compositional understanding, e.g., relations and attributes. Extensive experimental results demonstrate and validate several insights to understand the incapabilities of VLMs on compositional reasoning, which provide useful and reliable guidance for future studies. The deliverables will be updated at https://vlms-compositionality-gametheory.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
