他们所见，而不仅仅是他们看的地方：通过VLMs和NLP度量的语义扫视路径相似性
What They Saw, Not Just Where They Looked: Semantic Scanpath Similarity via VLMs and NLP metric
AI总结：
本文提出一种整合视觉语言模型的语义扫视路径相似性框架，通过控制视觉上下文生成文本描述，利用NLP度量计算语义相似性，揭示空间差异下的内容一致性。
摘要：
扫视路径相似性度量是眼动研究的核心，但现有方法主要评估空间和时间对齐，忽视了关注区域间的语义等价性。本文提出一个整合视觉语言模型（VLMs）的眼动分析框架。每个注视点在受控视觉上下文（基于补丁和标记策略）下进行编码，并转换为简洁的文本描述，这些描述被聚合为扫视路径级别的表示。然后使用基于嵌入和词汇NLP度量计算语义相似性，并与已有的空间度量（如MultiMatch和DTW）进行比较。在自由观看眼动数据上的实验表明，语义相似性捕捉了部分独立于几何对齐的方差，揭示了高内容一致性但空间差异的案例。进一步分析了上下文编码对描述准确性和度量稳定性的影响。我们的发现表明，多模态基础模型能够为经典扫视路径分析提供可解释的内容感知扩展，为ETRA社区内的目光研究提供了补充维度。
英文摘要：
Scanpath similarity metrics are central to eye-movement research, yet existing methods predominantly evaluate spatial and temporal alignment while neglecting semantic equivalence between attended image regions. We present a semantic scanpath similarity framework that integrates vision-language models (VLMs) into eye-tracking analysis. Each fixation is encoded under controlled visual context (patch-based and marker-based strategies) and transformed into concise textual descriptions, which are aggregated into scanpath-level representations. Semantic similarity is then computed using embedding-based and lexical NLP metrics and compared against established spatial measures, including MultiMatch and DTW. Experiments on free-viewing eye-tracking data demonstrate that semantic similarity captures partially independent variance from geometric alignment, revealing cases of high content agreement despite spatial divergence. We further analyze the impact of contextual encoding on description fidelity and metric stability. Our findings suggest that multimodal foundation models enable interpretable, content-aware extensions of classical scanpath analysis, providing a complementary dimension for gaze research within the ETRA community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
