GH-ESD：基于假设驱动的实例级视觉任务错误切片发现
GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks
AI总结：
提出GH-ESD框架，通过LLM生成假设与视觉语言模型验证，在实例级任务中自动发现空间关系错误切片，并构建GESD基准，显著提升检测和分割任务的错误切片发现精度。
摘要：
视觉模型在语义一致子集上的系统性失败（称为错误切片）揭示了鲁棒性和评估的局限性。现有的切片发现方法主要将切片建模为表示空间中的聚类或预定义属性的组合。虽然对图像级分类有效，但这种公式对于目标检测和分割等实例级任务不足，因为失败通常源于上下文关系性和空间定位的视觉模式。我们提出GH-ESD（基于假设驱动的实例级错误切片发现），一个生成与验证框架，将切片发现重新表述为基于假设的生成和统计验证。GH-ESD利用LLM先验和基于空间的视觉证据构建关系失败假设，通过视觉语言模型在实例级发现假设切片，并通过实例级错误的统计趋势分析进行验证。我们还引入了GESD（基于空间的错误切片数据集），一个用于实例级错误切片发现的新基准，提供由专家定义且基于空间的切片，这些切片源自检测和分割失败。大量实验表明，GH-ESD持续优于基线，在检测任务的GESD基准上Precision@10提高了0.10（0.73对比0.63），同时也支持分割场景。GH-ESD识别出可解释的切片，促进可操作的模型改进。GESD数据集将在接收后公开。
英文摘要：
Systematic failures of vision models on semantically coherent subsets, known as error slices, reveal limitations in robustness and evaluation. Existing slice discovery approaches largely model slices as clusters in representation space or combinations of predefined attributes. While effective for image-level classification, such formulations are insufficient for instance-level tasks such as object detection and segmentation, where failures often arise from contextual relational and spatially grounded visual patterns. We propose GH-ESD (Grounded Hypothesis-Driven Error Slice Discovery), a generate and verify framework that reformulates slice discovery as grounded hypothesis generation and statistical verification. GH-ESD constructs relational failure hypotheses using LLM priors and grounded visual evidence, discovers hypothesis slices at the instance level via Vision Language Models, and verifies them through statistical trend analysis over instance-level errors. We also introduce GESD (Grounded Error Slice Dataset), a new benchmark for instance-level error slice discovery, providing expert-defined and spatially grounded slices derived from detection and segmentation failures. Extensive experiments demonstrate that GH-ESD consistently outperforms baselines, improving Precision@10 by 0.10 (0.73 vs. 0.63) on the GESD benchmark for detection tasks, while also supporting segmentation scenarios. GH-ESD identifies interpretable slices that facilitate actionable model improvements.GESD is publicly available at https://github.com/apple/ml-fine-grained-error-slice-discovery-dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
