DO-Bench: 一种用于诊断视觉语言模型中物体幻觉的可归因基准
DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Models
AI总结：
本文提出DO-Bench，通过结构化多模态干预隔离错误来源，评估模型对文本先验和感知能力的鲁棒性，揭示物体幻觉的异质性失败模式。
摘要：
物体层面的幻觉仍是视觉语言模型（VLMs）的核心可靠性挑战，特别是在二元物体存在验证中。现有基准侧重于整体准确性，但很少区分错误源于感知限制还是上下文文本先验的影响，使底层失败机制模糊。我们引入DO-Bench，一种受控诊断基准，通过结构化多模态干预隔离这些来源。不同于在无约束环境中评估模型，DO-Bench探查两个互补维度：先验覆盖维度逐步增强上下文文本先验同时保持视觉证据不变以评估对先验压力的抵抗能力，感知受限维度逐步增强从全场景上下文到局部物体裁剪的视觉证据以测量感知接地强度。这种配对设计使错误可归因于先验压制、感知不足或其交互。我们进一步定义了两个诊断指标，PriorRobust和PerceptionAbility，以一致量化这些行为。在多样化的开源和闭源VLMs上的评估揭示了系统性的先验敏感性和感知可靠性差异，证明物体幻觉反映了超越整体准确性的异质性、机制依赖性失败模式。
英文摘要：
Object level hallucination remains a central reliability challenge for vision language models (VLMs), particularly in binary object existence verification. Existing benchmarks emphasize aggregate accuracy but rarely disentangle whether errors stem from perceptual limitations or from the influence of contextual textual priors, leaving underlying failure mechanisms ambiguous. We introduce DO-Bench, a controlled diagnostic benchmark that isolates these sources through structured multimodal interventions. Rather than evaluating models in unconstrained settings, DO-Bench probes two complementary dimensions: the Prior Override dimension progressively strengthens contextual textual priors while holding visual evidence constant to assess resistance to prior pressure, and the Perception-Limited dimension incrementally enhances visual evidence from full-scene context to localized object crops to measure perceptual grounding strength. This paired design enables attribution of errors to prior suppression, perceptual insufficiency, or their interaction. We further define two diagnostic metrics, PriorRobust and PerceptionAbility, to quantify these behaviors consistently. Evaluations across diverse open- and closed-source VLMs reveal systematic differences in prior sensitivity and perceptual reliability, demonstrating that object hallucination reflects heterogeneous, mechanism dependent failure patterns beyond aggregate accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
