PE3R:感知高效的3D重建
PE3R: Perception-Efficient 3D Reconstruction
AI总结:
PE3R提出一种无需调优的框架,通过集成多视图几何与2D语义先验,实现跨场景的高效3D语义重建,提升推理速度并取得新的SOTA精度。
摘要:
近年来,2D到3D感知的进步使从未姿态图像中恢复3D场景语义成为可能。然而,现有方法常面临泛化能力有限、依赖场景特定优化以及视角间语义不一致的问题。为解决这些问题,我们引入PE3R,一种无需调优的高效且可推广的3D语义重建框架。通过在前馈管道中集成多视图几何与2D语义先验,PE3R在无需任何场景特定微调的情况下,实现了跨多样场景和物体类别的零样本泛化。在开放词汇分割和多视图深度估计的广泛评估中,PE3R不仅实现了高达9倍的推理速度提升,还在语义和几何指标上均取得了新的SOTA精度。我们的方法为可扩展的语言驱动3D场景理解铺平了道路。代码可在github.com/hujiecpp/PE3R获取。
英文摘要:
Recent advances in 2D-to-3D perception have enabled the recovery of 3D scene semantics from unposed images. However, prevailing methods often suffer from limited generalization, reliance on per-scene optimization, and semantic inconsistencies across viewpoints. To address these limitations, we introduce PE3R, a tuning-free framework for efficient and generalizable 3D semantic reconstruction. By integrating multi-view geometry with 2D semantic priors in a feed-forward pipeline, PE3R achieves zero-shot generalization across diverse scenes and object categories without any scene-specific fine-tuning. Extensive evaluations on open-vocabulary segmentation and multi-view depth estimation show that PE3R not only achieves up to 9$\times$ faster inference but also sets new state-of-the-art accuracy in both semantic and geometric metrics. Our approach paves the way for scalable, language-driven 3D scene understanding. Code is available at github.com/hujiecpp/PE3R.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。