arXiv 2401.022812024-07-15cs.CV

PEGASUS：面向6DoF物体位姿数据集生成的物理增强高斯泼溅仿真系统

PEGASUS: Physically Enhanced Gaussian Splatting Simulation System for 6DoF Object Pose Dataset Generation

Friedrich-Alexander-Universität Erlangen-Nürnberg（弗里德里希-亚历山大-埃尔兰根-纽伦堡大学）；National Institute of Advanced Industrial Science and Technology（日本先进工业科学技术研究院）；Industrial CPS Research Center（工业网络物理系统研究中心）

Lukas Meyer, Floris Erich, Yusuke Yoshiyasu, Marc Stamminger, Noriaki Ando, Yukiyasu Domae

备注：: Project Page: https://meyerls.github.io/pegasus_web

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出基于3D高斯泼溅的PEGASUS系统，可结合物理引擎生成6DoF物体位姿数据集，能实现合成数据到真实数据的迁移，同时发布了适配该系统的Ramen杯面数据集。

摘要：

我们提出了面向6自由度（6DoF）物体位姿数据集生成的物理增强高斯泼溅仿真系统（Physically Enhanced Gaussian Splatting Simulation System，PEGASUS），这是一款基于3D Gaussian Splatting的多功能数据集生成工具。\n使用普通消费级相机即可轻松获取环境与物体的表征，并通过Gaussian Splatting完成重建。PEGASUS支持将环境对应的底层Gaussian Splatting点云与一个或多个物体的对应点云融合，从而组合出新场景。借助物理引擎，系统通过为物体和环境提取的网格之间的交互，模拟物体在场景中的自然放置效果。由此，通过组合不同的环境和物体，可创建大量静态或动态的新场景。从不同视角渲染场景后，可提取RGB图像、深度图、语义掩码和6DoF物体位姿等多样化数据。\n研究表明，使用PEGASUS生成的数据训练的位姿估计网络，可成功实现从合成数据到真实世界数据的迁移。此外，我们还推出了Ramen数据集，包含30款日本杯面产品。该数据集包含从物体半球面采集图像的球形扫描数据，以及对应的Gaussian Splatting重建结果，可与PEGASUS兼容使用。

英文摘要：

We introduce Physically Enhanced Gaussian Splatting Simulation System (PEGASUS) for 6DOF object pose dataset generation, a versatile dataset generator based on 3D Gaussian Splatting. Environment and object representations can be easily obtained using commodity cameras to reconstruct with Gaussian Splatting. <i>PEGASUS</i> allows the composition of new scenes by merging the respective underlying Gaussian Splatting point cloud of an environment with one or multiple objects. Leveraging a physics engine enables the simulation of natural object placement within a scene through interaction between meshes extracted for the objects and the environment. Consequently, an extensive amount of new scenes - static or dynamic - can be created by combining different environments and objects. By rendering scenes from various perspectives, diverse data points such as RGB images, depth maps, semantic masks, and 6DoF object poses can be extracted. Our study demonstrates that training on data generated by PEGASUS enables pose estimation networks to successfully transfer from synthetic data to real-world data. Moreover, we introduce the Ramen dataset, comprising 30 Japanese cup noodle items. This dataset includes spherical scans that captures images from both object hemisphere and the Gaussian Splatting reconstruction, making them compatible with PEGASUS.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。