视觉语言模型对城市场景的认知与人类一致吗？一项城市感知基准研究
Do Vision-Language Models See Urban Scenes as People Do? An Urban Perception Benchmark
AI总结：
该研究构建了含100张蒙特利尔街景图的城市感知基准，基于人类标注评估7个零样本VLM的城市感知能力，发现模型在客观属性上对齐度优于主观评价，并公开基准以支持相关可复现评估。
摘要：
理解人类如何解读城市场景可为设计与规划提供参考。我们提出了一个用于测试视觉语言模型（VLM）城市感知能力的小型基准，使用100张蒙特利尔街道图像，均匀分为实拍照片和照片级真实感合成场景两类。来自7个社区群体的12名参与者提供了涵盖30个维度的230份标注表单，维度混合了物理属性与主观印象。法语回答被统一标准化为英语。我们采用结构化提示词与确定性解析器，在零样本设置下评估了7个VLM。单选题使用准确率、多标签题使用Jaccard重叠度作为指标；人类一致性采用Krippendorff's alpha与成对Jaccard系数衡量。结果表明，模型在可见的客观属性上的对齐度优于主观评价。表现最优的系统（claude-sonnet）在多标签题上的宏准确率为0.31，平均Jaccard系数为0.48。人类一致性越高的维度，模型得分也越高。合成图像会略微降低得分。我们公开了该基准、提示词与测试框架，以支持参与式城市分析中可复现、具备不确定性感知的评估。
英文摘要：
Understanding how people read city scenes can inform design and planning. We introduce a small benchmark for testing vision-language models (VLMs) on urban perception using 100 Montreal street images, evenly split between photographs and photorealistic synthetic scenes. Twelve participants from seven community groups supplied 230 annotation forms across 30 dimensions mixing physical attributes and subjective impressions. French responses were normalized to English. We evaluated seven VLMs in a zero-shot setup with a structured prompt and deterministic parser. We use accuracy for single-choice items and Jaccard overlap for multi-label items; human agreement uses Krippendorff's alpha and pairwise Jaccard. Results suggest stronger model alignment on visible, objective properties than subjective appraisals. The top system (claude-sonnet) reaches macro 0.31 and mean Jaccard 0.48 on multi-label items. Higher human agreement coincides with better model scores. Synthetic images slightly lower scores. We release the benchmark, prompts, and harness for reproducible, uncertainty-aware evaluation in participatory urban analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
