刻意设计的不适定问题：探究VLMs中的证据使用
Ill-Posed by Design: Probing Evidence Use in VLMs
AI总结：
提出单目物体尺寸估计作为不适定诊断任务，通过反事实分析分解六种视觉和语言证据通道，评估12个开源VLM，发现最大模型仍落后于纯文本LLM，且模型未有效利用场景几何信息。
摘要：
反事实分析被广泛用于研究视觉语言模型中的证据使用，但其诊断价值在适定任务上有限：当多个线索独立支持同一答案时，移除一个可能不会改变预测。我们提出单目度量物体尺寸估计作为证据选择的不适定诊断设置：因为物理尺寸无法从单个未校准图像确定，模型必须依赖不完美的线索——类别先验、目标外观、局部上下文、表观图像尺寸和场景几何。我们构建了Metric VQA（来自Objectron的10,813个尺寸查询和331个野外卷尺测量场景），并通过反事实分析分解六个视觉和语言证据通道，评估了12个开源VLM（3–397B参数）。即使测试的最大VLM（Qwen3-VL-235B、Qwen3.5-397B、InternVL3.5-241B）在野外分割上仍落后于纯文本前沿LLM。诊断分析显示：目标身份是最关键的线索，目标像素和局部上下文仅对部分模型有帮助，表观尺寸会改变预测但缺乏方向性解读，全局场景几何基本未被利用。我们分析了LoRA微调作为针对度量估计的可操作干预：虽然任务可学习，但模型并未学会利用场景几何。
英文摘要：
Counterfactual analysis is widely used to study evidence use in vision-language models, but its diagnostic value is limited on well-posed tasks: when several cues independently support the same answer, removing one may not change the prediction. We propose monocular metric object-size estimation as an ill-posed diagnostic setting for evidence selection: because physical size cannot be determined from a single uncalibrated image, models must rely on imperfect cues category priors, target appearance, local context, apparent image size, and scene geometry. We assemble Metric VQA ($10{,}813$ dimension queries from Objectron and $331$ tape-measured in-the-wild scenes) and evaluate $12$ open-weight VLMs ($3$--$397$\,B parameters) with counterfactual analysis decomposing six visual and language evidence channels. Even the largest VLMs tested (Qwen3-VL-235B, Qwen3.5-397B, InternVL3.5-241B) trail a text-only frontier LLM on the in-the-wild split. The diagnostic analysis shows: target identity is the most load-bearing cue, target pixels and local context help only some models, apparent size shifts predictions without a directional readout, and global scene geometry is largely unused. We analyze LoRA fine-tuning as an actionable intervention specific to metric estimation: while the task is learnable, the models do not learn to leverage scene geometry.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
