UnIRe:用于动态城市场景重建的无监督实例分解
UnIRe: Unsupervised Instance Decomposition for Dynamic Urban Scene Reconstruction
AI总结:
提出基于3DGS的无监督方法UnIRe,通过4D superpoints实现动态城市场景的实例分解与重建,支持灵活的实例级编辑。
摘要:
重建和分解动态城市场景对自动驾驶、城市规划与场景编辑至关重要。然而,现有方法无法在无人工标注下进行实例感知分解,而这对于实例级场景编辑是必不可少的。我们提出UnIRe,一种基于3D Gaussian Splatting (3DGS)的方法,仅使用RGB图像和LiDAR点云将场景分解为静态背景和各个动态实例。其核心是引入4D superpoints,这是一种在4D空间中聚类多帧LiDAR点的新表示,能基于时空相关性实现无监督实例分离。这些4D superpoints作为分解4D初始化的基础,即为训练动态3DGS提供空间和时间初始化,适用于任意动态类别且无需边界框或物体模板。此外,我们在2D和3D空间引入平滑正则化策略,进一步提升时间稳定性。在基准数据集上的实验表明,我们的方法在分解动态场景重建上优于现有方法,同时实现准确灵活的实例级编辑,成为实际应用的有效方案。
英文摘要:
Reconstructing and decomposing dynamic urban scenes is crucial for autonomous driving, urban planning, and scene editing. However, existing methods fail to perform instance-aware decomposition without manual annotations, which is crucial for instance-level scene editing.We propose UnIRe, a 3D Gaussian Splatting (3DGS) based approach that decomposes a scene into a static background and individual dynamic instances using only RGB images and LiDAR point clouds. At its core, we introduce 4D superpoints, a novel representation that clusters multi-frame LiDAR points in 4D space, enabling unsupervised instance separation based on spatiotemporal correlations. These 4D superpoints serve as the foundation for our decomposed 4D initialization, i.e., providing spatial and temporal initialization to train a dynamic 3DGS for arbitrary dynamic classes without requiring bounding boxes or object templates.Furthermore, we introduce a smoothness regularization strategy in both 2D and 3D space, further improving the temporal stability.Experiments on benchmark datasets show that our method outperforms existing methods in decomposed dynamic scene reconstruction while enabling accurate and flexible instance-level editing, making it a practical solution for real-world applications.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。