arXiv 2605.149232026-05-14cs.CV

SceneParser: 用于视觉语义理解的分层场景解析

SceneParser: Hierarchical Scene Parsing for Visual Semantics Understanding

Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Zhongguancun Academy（中关村学院）；Huazhong University of Science and Technology（华中科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；South China University of Technology（华南理工大学）；University of Oxford（牛津大学）；Peking University（北京大学）

Pengxin Xu, Xincheng Lin, Luping Xiao, Qing Jiang, Meishan Zhang, Hao Fei, Shanghang Zhang, Xingyu Chen

备注：: Preprint. Code, models, and dataset are provided in the manuscript

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出SceneParser，通过分层解析任务构建场景-对象-部件-功能的层次结构，利用结构完成伪标签和课程学习提升结构感知能力，实验表明其在复杂场景理解中表现更优。

摘要：

通用场景感知已从物体识别发展到开放词汇定位、部件定位和功能预测。然而，这些能力通常以孤立预测形式实现，无法捕捉交互导向的场景理解所需的结构依赖。为此，我们引入分层场景解析任务，通过显式场景->对象->部件->功能层次结构和跨层绑定表示物理场景。我们通过SceneParser，一种基于VLM的解析器，训练统一的层次生成，使用结构完成伪标签和课程学习。为支持训练和评估，我们构建了SceneParser-Bench大规模基准，包含110K训练图像、5K验证集、777K对象、1.14M部件、1.74M功能注解和1.74M有效的对象-部件-功能链实例。我们进一步引入Level-1到Level-3条件度量和ParseRate来评估定位、跨层绑定和层次完整性。实验表明，现有MLLMs和感知拼接管道在SceneParser-Bench上难以进行层次解析，而SceneParser在结构感知性能上更强。此外，消融实验、在COCO和AGD20K上的评估以及下游规划探针表明，我们的SceneParser兼容传统任务并为视觉理解提供可操作的表示。

英文摘要：

General scene perception has progressed from object recognition toward open-vocabulary grounding, part localization, and affordance prediction. Yet these capabilities are often realized as isolated predictions that localize objects, parts, or interaction points without capturing the structured dependencies needed for interaction-oriented scene understanding. To address this gap, we introduce Hierarchical Scene Parsing, an interaction-oriented parsing task that represents physical scenes as explicit scene -> object -> part -> affordance hierarchies with cross-level bindings. We instantiate this task with SceneParser, a VLM-based parser trained for unified hierarchical generation with structural-completion pseudo labels and curriculum learning. To support training and evaluation, we construct SceneParser-Bench, a large-scale benchmark built with a scalable hierarchical data engine, containing 110K training images, a 5K validation split, 777K objects, 1.14M parts, 1.74M affordance annotations, and 1.74M valid object-part-affordance chain instances. We further introduce Level-1 to Level-3 conditional metrics and ParseRate to evaluate localization, cross-level binding, and hierarchical completeness. Experiments show that existing MLLMs and perception-stitching pipelines struggle with hierarchical parsing on our SceneParser-Bench, while SceneParser achieves stronger structure-aware performance. Besides, ablations, evaluations on COCO and AGD20K, and a downstream planning probe demonstrate that our SceneParser is compatible with conventional tasks and provides an actionable representation for visual understanding.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。