下一步测试什么:驾驶视觉语言模型中可解释的覆盖缺口发现
What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs
AI总结:
提出 SliceScorer 和 SliceNav 方法,通过结合暴露先验和邻居失败先验的确定性评分规则,在驾驶视觉语言模型中有效发现高风险覆盖缺口,并支持可解释和可审计的验证流程。
摘要:
驾驶视觉语言模型必须准确理解由操作设计域定义的各种条件下的场景,然而验证仍然稀疏:许多切片缺失,使得经验故障率不可靠。我们提出 SliceScorer,一种用于缺失切片推荐的确定性评分规则,它结合了 (i) 基于暴露的覆盖先验,优先考虑罕见、测试不足的区域,以及 (ii) 邻居失败先验,从类似测试条件传播风险。SliceScorer 刻意简单——可解释、可审计且保守——这些属性对于安全关键验证至关重要。为了在声明的 ODD 之外进行压力测试,我们将 SliceScorer 嵌入 SliceNav,一个由 LLM 编排的验证流程,其中模型解释开发者查询以选择相关操作(分诊、评分、获取、评估)和词汇扩展,组合验证工作流,同时保持所有评分确定性和可审计性。在三个驾驶 VLM(WiseAD、DriveMM、Cosmos-Reason2-2B)上的实验表明,SliceNav 比先前的切片发现方法更有效地发现高风险覆盖缺口,同时在条件空间中保持多样化的推荐。消融实验证实了两个评分组件的贡献,定性分析展示了从开发者查询到目标评估的端到端工作流。
英文摘要:
Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs), yet verification remains sparse: many slices are missing, making empirical failure rates unreliable. We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation that combines (i) an exposure-based coverage prior to prioritize rare, under-tested regions, and (ii) a neighbor-failure prior that propagates risk from similar tested conditions. SliceScorer is deliberately simple - interpretable, auditable, and conservative - properties essential for safety-critical validation. For stress testing beyond the declared ODD, we embed SliceScorer within SliceNav, an LLM-orchestrated verification pipeline where the model interprets developer queries to select relevant operators (triage, scoring, acquisition, evaluation) and vocabulary extensions, composing verification workflows while keeping all scoring deterministic and auditable. Experiments on three driving VLMs (WiseAD, DriveMM, Cosmos-Reason2-2B) show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods while maintaining diverse recommendations across the condition space. Ablations confirm both scoring components contribute, and qualitative analysis demonstrates end-to-end workflows from developer query to targeted evaluation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。