arXiv 2604.222742026-05-26cs.CV

CAGE-SGG：用于开放词汇场景图生成的反事实主动图证据

CAGE-SGG: Counterfactual Active Graph Evidence for Open-Vocabulary Scene Graph Generation

Institute of Intelligent Vision and Embodied Cognition（智能视觉与具身认知研究院）

Suiyang Guang, Chenyu Liu, Ruohan Zhang, Siyuan Chen

备注：: This manuscript has been withdrawn by the authors because we found a methodological flaw in the formulation and evaluation of the proposed approach. The issue affects the reliability of the experimental results and the conclusions drawn from them. Therefore, the authors consider the current version unsuitable for citation or further use

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出基于反事实关系验证的开放词汇场景图生成框架，通过分解谓词为软证据基并使用反事实验证器确保关系有视觉证据支持，从而提升可靠性、可解释性和泛化能力。

摘要：

开放词汇场景图生成（SGG）旨在用超出固定谓词词汇表的灵活且细粒度的关系短语描述视觉场景。虽然最近的视觉语言模型极大地扩展了SGG的语义覆盖范围，但它们也引入了一个关键的可信性问题：预测的关系可能由语言先验或对象共现驱动，而非基于视觉证据。在本文中，我们提出了一种基于反事实关系验证的证据充分的开放词汇SGG框架。我们的方法不是直接接受合理的关系提议，而是验证每个候选关系是否得到关系特定的视觉、几何和上下文证据的支持。具体来说，我们首先使用视觉语言提议器生成开放词汇关系候选，然后将谓词短语分解为软证据基，如支撑、接触、包含、深度和状态。关系条件证据编码器提取谓词相关线索，而反事实验证器测试当必要证据被移除时关系分数是否下降，并在无关扰动下保持稳定。我们进一步引入矛盾感知谓词学习和图级偏好优化，以改进细粒度区分和全局图一致性。在常规、开放词汇和全景SGG基准上的实验表明，我们的方法一致地改进了标准召回率指标、未见谓词泛化和反事实基础质量。这些结果表明，从关系生成转向关系验证可产生更可靠、可解释且基于证据的场景图。

英文摘要：

Open-vocabulary scene graph generation (SGG) aims to describe visual scenes with flexible and fine-grained relation phrases beyond a fixed predicate vocabulary. While recent vision-language models greatly expand the semantic coverage of SGG, they also introduce a critical reliability issue: predicted relations may be driven by language priors or object co-occurrence rather than grounded visual evidence. In this paper, we propose an evidence-rounded open-vocabulary SGG framework based on counterfactual relation verification. Instead of directly accepting plausible relation proposals, our method verifies whether each candidate relation is supported by relation-pecific visual, geometric, and contextual evidence. Specifically, we first generate open-vocabulary relation candidates with a vision-language proposer, then decompose predicate phrases into soft evidence bases such as support, contact, containment, depth and state. A relation-conditioned evidence encoder extracts predicate-relevant cues, while a counterfactual verifier tests whether the relation score decreases when necessary vidence is removed and remains stable under irrelevant perturbations. We further introduce contradiction-aware predicate learning and graph-level preference optimization to improve fine-grained discrimination and global graph consistency. Experiments on conventional, open-vocabulary, and panoptic SGG benchmarks show that our method consistently improves standard recall-based metrics, unseen predicate generalization, and counterfactual grounding quality. These results demonstrate that moving from relation generation to relation verification leads to more reliable, interpretable, and evidence-grounded scene graphs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。