RelWitness: 基于视觉-几何关系见证者的开放词汇3D场景图生成
RelWitness: Open-Vocabulary 3D Scene Graph Generation with Visual-Geometric Relation Witnesses
AI总结：
提出RelWitness框架，通过视觉-几何关系见证者从不完整关系监督中生成开放词汇3D场景图，解决关系标注稀疏和词汇扩展问题。
摘要：
开放词汇3D场景图生成旨在用灵活的自然语言谓词描述对象实例及其关系。核心难点不仅在于词汇扩展，还在于监督可靠性：3D场景图数据集中的关系标注具有选择性，许多有效的对象对关系未被标注。我们提出RelWitness，一个从带有位姿的RGB-D序列中生成开放词汇3D场景图的框架，可在不完整关系监督下工作。关键概念是关系见证者：一种具体的视觉-几何线索，使关系在捕获场景中可观察。支持关系需要接触和垂直排序；包含关系需要包围；邻近关系需要度量接近；朝向关系需要面对方向；稳定关系应在两个对象可见的视角间持续存在。RelWitness从RGB视图、深度图、重建的3D几何、角色敏感文本、对象先验空视图和多视角一致性构建关系见证记录。视觉-几何见证验证器将未标注的关系候选分配给验证的缺失正例、可靠负例或不确定未标注案例。然后，见证引导的正-无标记目标从不完整标注中学习，而不将每个缺失标签视为负例。我们进一步引入见证一致解码和RGB-D缺失关系审计协议。在3DSSG/3RScan和ScanNet派生的开放词汇分割上的模拟手稿规划实验显示了预期行为：改进的未见关系识别、更高的见证精度、更低的幻觉和减少的关系短语冗余。所有数值结果均为规划值，在提交前必须替换为复现的测量值。
英文摘要：
Open-vocabulary 3D scene graph generation seeks to describe object instances and their relations with flexible natural-language predicates. The central difficulty is not only vocabulary expansion, but supervision reliability: relation annotations in 3D scene graph datasets are selective, and many valid object-pair relations are unannotated. We propose RelWitness, a framework for open-vocabulary 3D scene graph generation from posed RGB-D sequences under incomplete relation supervision. The key concept is a relation witness: a concrete visual-geometric cue that makes a relation observable in the captured scene. Support relations require contact and vertical ordering; containment requires enclosure; proximity requires metric closeness; orientation requires facing direction; and stable relations should persist across views where both objects are visible. RelWitness constructs relation witness records from RGB views, depth maps, reconstructed 3D geometry, role-sensitive text, object-prior null views, and multi-view consistency. A visual-geometric witness verifier assigns unannotated relation candidates to verified missing positives, reliable negatives, or uncertain unlabeled cases. A witness-guided positive-unlabeled objective then learns from incomplete annotations without turning every missing label into a negative. We further introduce witness-consistent decoding and an RGB-D missing-relation audit protocol. Simulated manuscript-planning experiments on 3DSSG/3RScan and ScanNet-derived open-vocabulary splits show the intended behavior: improved unseen-relation recognition, higher witness precision, lower hallucination, and reduced redundant relation phrases. All numerical results are planning values and must be replaced by reproduced measurements before submission
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
