DiffSpot：VLM能发现网页界面中的细微视觉差异吗？
DiffSpot: Can VLMs Spot Fine-Grained Visual Differences in Web Interfaces?
AI总结：
提出DiffSpot基准，通过CSS属性突变生成可控图像对，评估视觉语言模型在网页界面中检测细微视觉差异的能力，发现最佳模型仅识别40.7%的真实变化。
摘要：
视觉语言模型（VLM）在高层次图像-文本对齐方面取得了显著进展，但其感知细微视觉差异的能力仍然有限。我们在渲染的网页界面中研究这一问题，其中局部视觉变化既是对细粒度感知的诊断测试，也是GUI代理和设计工具的实际需求。我们引入了 extbf{DiffSpot}，一个用于网页界面开放式找不同的代码驱动基准。DiffSpot通过突变自包含HTML中目标元素的单个CSS属性，重新渲染页面，并记录变化的属性、元素和突变幅度，从而构建受控图像对。一个接地门控仅保留渲染像素差异局限于目标元素的图像对。该基准包含4,400对图像，包括3,900对有差异对（平衡分布在13个CSS属性操作符和三个难度级别上）以及500对无差异对用于幻觉控制。对13个前沿VLM进行零样本评估，我们发现即使最佳模型也只能识别$40.7\%$的真实变化，所有模型在困难级别的召回率低于$23\%$。DiffSpot进一步表明，难度强烈依赖于属性：在CSS操作符中，像素幅度和CLIP距离都不能可靠预测召回率。
英文摘要：
Vision-language models (VLMs) have made strong progress on high-level image-text alignment, yet their ability to perceive subtle visual differences remains limited. We study this problem in rendered web interfaces, where localized visual changes are both a diagnostic test of fine-grained perception and a practical requirement for GUI agents and design tools. We introduce \textbf{DiffSpot}, a code-driven benchmark for open-ended spot-the-difference on web interfaces. DiffSpot constructs controlled image pairs by mutating a single CSS property of a target element in self-contained HTML, re-rendering the page, and recording the changed property, element, and mutation magnitude. A grounding gate retains only pairs whose rendered pixel difference is confined to the target element. The benchmark contains 4{,}400 pairs, including 3{,}900 has-diff pairs balanced across 13 CSS-property operators and three difficulty tiers, plus 500 no-diff pairs for hallucination control. Evaluating 13 frontier VLMs zero-shot, we find that even the best model identifies only $40.7\%$ of true changes, with Hard-tier Recall below $23\%$ for every model. DiffSpot further shows that difficulty is strongly property-dependent: across CSS operators, neither pixel magnitude nor CLIP distance reliably predicts Recall.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
