TB-HSU：具有上下文可供性的层次化三维场景理解
TB-HSU: Hierarchical 3D Scene Understanding with Contextual Affordances
AI总结：
本文提出TB-HSU模型，利用Transformer构建层次化三维场景图，结合上下文可供性，提升三维场景理解性能，并公开代码和数据集。
摘要：
功能和可供性的概念是三维场景理解的一个关键方面，并支持面向任务的目标。在这项工作中，我们开发了一个模型，学习在表示场景空间组织的三维层次化场景图中构建和变化功能可供性。变化的功能可供性被设计为与图的变化空间上下文相集成。更具体地说，我们开发了一种算法，学习构建捕获场景空间组织的三维层次化场景图（3DHSG）。从分割的对象点云和对象语义标签开始，我们构建了一个3DHSG，其顶部节点标识房间标签，子节点定义房间内的局部空间区域并具有区域特定的可供性，孙节点指示对象位置和对象特定的可供性。为了支持这项工作，我们创建了一个自定义的3DHSG数据集，为局部空间区域提供具有区域特定可供性的地面真值数据，并为每个对象提供对象特定的可供性。我们采用基于Transformer的模型来学习3DHSG。我们使用多任务学习框架，同时学习房间分类和定义房间内具有区域特定可供性的空间区域。我们的工作改进了最先进的基线模型的性能，并展示了一种将Transformer模型应用于三维场景理解和生成捕获房间空间组织的3DHSG的方法。代码和数据集是公开可用的。
英文摘要：
The concept of function and affordance is a critical aspect of 3D scene understanding and supports task-oriented objectives. In this work, we develop a model that learns to structure and vary functional affordance across a 3D hierarchical scene graph representing the spatial organization of a scene. The varying functional affordance is designed to integrate with the varying spatial context of the graph. More specifically, we develop an algorithm that learns to construct a 3D hierarchical scene graph (3DHSG) that captures the spatial organization of the scene. Starting from segmented object point clouds and object semantic labels, we develop a 3DHSG with a top node that identifies the room label, child nodes that define local spatial regions inside the room with region-specific affordances, and grand-child nodes indicating object locations and object-specific affordances. To support this work, we create a custom 3DHSG dataset that provides ground truth data for local spatial regions with region-specific affordances and also object-specific affordances for each object. We employ a transformer-based model to learn the 3DHSG. We use a multi-task learning framework that learns both room classification and learns to define spatial regions within the room with region-specific affordances. Our work improves on the performance of state-of-the-art baseline models and shows one approach for applying transformer models to 3D scene understanding and the generation of 3DHSGs that capture the spatial organization of a room. The code and dataset are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
