arXiv 2505.126602025-05-19cs.CV

利用中央凹场景理解图预测理解场景的反应时间

Predicting Reaction Time to Comprehend Scenes with Foveated Scene Understanding Maps

Department of Computer Science University of California Santa Barbara（计算机科学系加州大学圣芭芭拉分校）；Graduate Program in Dynamical Neuroscience University of California Santa Barbara（动态神经科学研究生项目加州大学圣芭芭拉分校）；Department of Psychological and Brain Sciences University of California Santa Barbara（心理学与脑科学系加州大学圣芭芭拉分校）

Ziqi Wen, Jonathan Skaza, Shravan Murlidaran, William Y. Wang, Miguel P. Eckstein

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出结合中央凹视觉与视觉语言模型的F-SUM指标，用于预测人类场景理解反应时间，其与人类反应时间、眼跳次数等的相关性显著优于传统图像指标，验证了中央凹视觉加工的重要性。

摘要：

尽管已有模型可预测人类在目标搜索、视觉辨别等任务中的反应时间（RT），但用于场景理解时间的图像可计算预测器的开发仍是一项开放挑战。视觉语言模型（VLM）近年来取得进展，能够为任意图像生成场景描述，再加上可用于比较语言描述的量化指标的出现，为建模人类场景理解提供了新契机。我们假设，人类场景理解的主要瓶颈，以及不同场景间反应时间差异的驱动来源，是人类视觉系统的中央凹特性与图像中任务相关视觉信息的空间分布之间的相互作用。基于这一假设，我们提出了一种新颖的图像可计算模型，该模型将中央凹视觉与VLM相结合，生成随注视位置变化的空间分辨场景理解图（中央凹场景理解图，简称F-SUM），以及一个聚合的F-SUM分数。该指标与理解场景所需的人类平均反应时间（N=17，r=0.47）和眼跳次数（r=0.51）相关（涵盖277个场景）。F-SUM分数还与限时呈现条件下的人类平均描述准确率（N=16，r=-0.56）相关。这些相关性显著超过了标准的基于图像的指标，如杂乱度、视觉复杂度以及基于语言熵的场景模糊度。综上，我们的工作提出了一种用于预测人类场景理解反应时间的新型图像可计算指标，并证明了中央凹视觉加工在影响理解难度方面的重要性。

英文摘要：

Although models exist that predict human response times (RTs) in tasks such as target search and visual discrimination, the development of image-computable predictors for scene understanding time remains an open challenge. Recent advances in vision-language models (VLMs), which can generate scene descriptions for arbitrary images, combined with the availability of quantitative metrics for comparing linguistic descriptions, offer a new opportunity to model human scene understanding. We hypothesize that the primary bottleneck in human scene understanding and the driving source of variability in response times across scenes is the interaction between the foveated nature of the human visual system and the spatial distribution of task-relevant visual information within an image. Based on this assumption, we propose a novel image-computable model that integrates foveated vision with VLMs to produce a spatially resolved map of scene understanding as a function of fixation location (Foveated Scene Understanding Map, or F-SUM), along with an aggregate F-SUM score. This metric correlates with average (N=17) human RTs (r=0.47) and number of saccades (r=0.51) required to comprehend a scene (across 277 scenes). The F-SUM score also correlates with average (N=16) human description accuracy (r=-0.56) in time-limited presentations. These correlations significantly exceed those of standard image-based metrics such as clutter, visual complexity, and scene ambiguity based on language entropy. Together, our work introduces a new image-computable metric for predicting human response times in scene understanding and demonstrates the importance of foveated visual processing in shaping comprehension difficulty.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。