利用全景映射和CAD模型对齐重建交互式三维场景
Reconstructing Interactive 3D Scenes by Panoptic Mapping and CAD Model Alignments
摘要：
在本文中，我们重新思考的问题，从一个具体的代理的角度来看，场景重建：而经典的观点侧重于重建的准确性，我们的新观点强调的基本功能和约束条件，使重建的场景提供\em{actionable}信息模拟\em{interactions}与代理。在这里，我们通过使用RGB-D数据流重建交互式场景来解决这个具有挑战性的问题，RGB-D数据流通过3D体积全景映射模块捕获（i）对象和布局的语义和几何形状，以及（ii）通过推理对象之间的物理常识来捕获对象示能表示和上下文关系，由基于图形的场景表示来组织。至关重要的是，这个重建的场景用基于零件的铰接式CAD模型替换了密集全景图中的对象网格，以实现更细粒度的机器人交互。在实验中，我们证明了（i）我们的全景映射模块优于以前的最先进的方法，（ii）一个高性能的物理推理过程，匹配，对齐，并取代对象的网格与最适合的CAD模型，和（iii）重建的场景是物理上合理的，自然提供可操作的互动;无需任何手动标记，它们就可以无缝导入到基于ROS的模拟器和虚拟环境中，以执行复杂的机器人任务。
英文摘要：
In this paper, we rethink the problem of scene reconstruction from an embodied agent's perspective: While the classic view focuses on the reconstruction accuracy, our new perspective emphasizes the underlying functions and constraints such that the reconstructed scenes provide \em{actionable} information for simulating \em{interactions} with agents. Here, we address this challenging problem by reconstructing an interactive scene using RGB-D data stream, which captures (i) the semantics and geometry of objects and layouts by a 3D volumetric panoptic mapping module, and (ii) object affordance and contextual relations by reasoning over physical common sense among objects, organized by a graph-based scene representation. Crucially, this reconstructed scene replaces the object meshes in the dense panoptic map with part-based articulated CAD models for finer-grained robot interactions. In the experiments, we demonstrate that (i) our panoptic mapping module outperforms previous state-of-the-art methods, (ii) a high-performant physical reasoning procedure that matches, aligns, and replaces objects' meshes with best-fitted CAD models, and (iii) reconstructed scenes are physically plausible and naturally afford actionable interactions; without any manual labeling, they are seamlessly imported to ROS-based simulators and virtual environments for complex robot task executions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
