arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.08333cs.CV
收藏

走向人类全球语境:视觉语言模型真的像人类一样判断吗?

Towards the Human Global Context: Does the Vision-Language Model Really Judge Like a Human Being?

Testworks Inc.(Testworks公司)

Sangmyeong Woh, Jaemin Lee, Ho Joong Kim, Jinsuk Lee

摘要:

随着计算机视觉和自然语言处理技术的发展,视觉语言成为一个重要的研究领域。尽管重要性,评价指标的研究领域仍处于发展的初级阶段。在本文中,我们提出了一个定量度量“等方差分数”和评估数据集“人类困惑”来评估VL模型是否像人类一样理解图像。我们观察到,VL模型并不解释输入图像的整体上下文,而是显示出对形成局部上下文的特定对象或形状的偏见。我们的目标是定量测量模型在理解上下文方面的性能。为了验证当前现有VL模型的能力,我们将原始输入图像切片并随机放置它们,从而扭曲图像的全局上下文。我们的论文讨论了每一个VL模型的解释水平上的全球背景和地址的结构特征如何影响的结果。

英文摘要:

As computer vision and NLP make progress, Vision-Language(VL) is becoming an important area of research. Despite the importance, evaluation metrics of the research domain is still at a preliminary stage of development. In this paper, we propose a quantitative metric "Equivariance Score" and evaluation dataset "Human Puzzle" to assess whether a VL model is understanding an image like a human. We observed that the VL model does not interpret the overall context of an input image but instead shows biases toward a specific object or shape that forms the local context. We aim to quantitatively measure a model's performance in understanding context. To verify the current existing VL model's capability, we sliced the original input image into pieces and randomly placed them, distorting the global context of the image. Our paper discusses each VL model's level of interpretation on global context and addresses how the structural characteristics influenced the results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑