arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.22416cs.HC
收藏

像人类一样看图:通过计算度量和MLLMs进行相似性评估的基准测试

Seeing Graphs Like Humans: Benchmarking Computational Measures and MLLMs for Similarity Assessment

Korea Advanced Institute of Science and Technology (KAIST)(韩国科学技术院);Seoul National University(首尔大学);University of Oklahoma(俄克拉荷马大学)

Seokweon Jung, Jeongmin Rhee, Seoyoung Doh, Hyeon Jeon, Ghulam Jilani Quadri, Jinwook Seo

AI总结:

本文通过实验比较了计算度量和MLLMs在图相似性评估中的表现,发现MLLMs,特别是GPT-5,在匹配人类感知方面表现优异,能提供可解释的推理理由。

摘要:

将图进行比较以识别相似性是图数据分析可视化分析中的基本任务。为了支持这一任务,可视化分析系统经常使用定量计算度量来提供自动指导。然而,这些度量与主观人类视觉感知的匹配程度仍然不清楚,从而提供与分析人员直觉判断相冲突的建议,可能导致混淆而不是减轻认知负荷。多模态大语言模型(MLLMs),能够视觉上解释图并用自然语言解释其推理,已成为解决这一挑战的潜在替代方案。本文通过使用1881个节点-链接图的数据集,通过三个相互关联的实验来弥合人类和机器对图相似性评估之间的差距。实验1收集了32名人类参与者对相对相似性判断和理由,揭示了关于图相似性的共识,同时优先考虑全局形状和边密度而非精确拓扑细节。实验2将16种计算度量与这些人类判断进行基准测试,发现Portrait divergence是表现最好的度量,尽管匹配程度中等。实验3评估了三种最先进的MLLMs(GPT-5、Gemini 2.5 Pro、Claude Sonnet 4.5)作为感知代理的潜力。结果表明,MLLMs,特别是GPT-5,在与人类图相似性感知的匹配上显著优于传统度量,并能提供可解释的推理理由,而Claude Sonnet 4.5则显示出最佳的计算效率。我们的发现表明,MLLMs不仅作为有效的、可解释的代理来模拟人类感知具有重要意义,而且作为智能引导,能够揭示在可视化分析系统中可能被人类分析人员忽视的细微差别。

英文摘要:

Comparing graphs to identify similarities is a fundamental task in visual analytics of graph data. To support this, visual analytics systems frequently employ quantitative computational measures to provide automated guidance. However, it remains unclear how well these measures align with subjective human visual perception, thereby offering recommendations that conflict with analysts' intuitive judgments, potentially leading to confusion rather than reducing cognitive load. Multimodal Large Language Models (MLLMs), capable of visually interpreting graphs and explaining their reasoning in natural language, have emerged as a potential alternative to address this challenge. This paper bridges the gap between human and machine assessment of graph similarity through three interconnected experiments using a dataset of 1,881 node-link diagrams. Experiment 1 collects relative similarity judgments and rationales from 32 human participants, revealing consensus on graph similarity while prioritizing global shapes and edge densities over exact topological details. Experiment 2 benchmarks 16 computational measures against these human judgments, identifying Portrait divergence as the best-performing metric, though with only moderate alignment. Experiment 3 evaluates the potential of three state-of-the-art MLLMs (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5) as perceptual proxies. The results demonstrate that MLLMs, particularly GPT-5, significantly outperform traditional measures in aligning with human graph similarity perception and provide interpretable rationales for their decisions, whereas Claude Sonnet 4.5 shows the best computational efficiency. Our findings suggest that MLLMs hold significant promise not only as effective, explainable proxies for human perception but also as intelligent guides that can uncover subtle nuances that might be overlooked by human analysts in visual analytics systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑