用于室内场景紧凑3D表示的深度多长方体拟合
Deep Polycuboid Fitting for Compact 3D Representation of Indoor Scenes
AI总结：
该研究提出一种深度多长方体拟合框架，通过Transformer检测表面、图神经网络验证关系来重建多长方体，实现室内场景紧凑3D表示，在多个真实数据集上泛化良好，可支撑虚拟漫游等应用。
摘要：
本文提出了一种新颖框架，通过基于深度学习的拟合方法，利用一组多长方体（polycuboids）来紧凑表示3D室内场景。室内场景主要由家具等人造物体构成，这类物体通常呈现直线几何特征。这一特性使得室内场景可以用多长方体的组合来表示，提供了一种紧凑表示形式，可为家具重新布置等下游应用提供助力。我们的框架以含噪点云为输入，首先使用Transformer网络检测六类长方体表面。接着，利用图神经网络验证所检测表面的空间关系，以形成潜在的多长方体。最后，通过基于聚合的表面标签构建一组盒子，重建每个多长方体实例。为训练我们的网络，我们引入了一个合成数据集，其中包含多种多样的长方体和多长方体形状，能够反映室内场景的特征。我们的框架在真实世界室内场景数据集上表现出良好的泛化能力，包括Replica、ScanNet以及用iPhone采集的场景。我们通过虚拟房间漫游、场景编辑等实际应用验证了该方法的通用性。
英文摘要：
This paper presents a novel framework for compactly representing a 3D indoor scene using a set of polycuboids through a deep learning-based fitting method. Indoor scenes mainly consist of man-made objects, such as furniture, which often exhibit rectilinear geometry. This property allows indoor scenes to be represented using combinations of polycuboids, providing a compact representation that benefits downstream applications like furniture rearrangement. Our framework takes a noisy point cloud as input and first detects six types of cuboid faces using a transformer network. Then, a graph neural network is used to validate the spatial relationships of the detected faces to form potential polycuboids. Finally, each polycuboid instance is reconstructed by forming a set of boxes based on the aggregated face labels. To train our networks, we introduce a synthetic dataset encompassing a diverse range of cuboid and polycuboid shapes that reflect the characteristics of indoor scenes. Our framework generalizes well to real-world indoor scene datasets, including Replica, ScanNet, and scenes captured with an iPhone. The versatility of our method is demonstrated through practical applications, such as virtual room tours and scene editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
