arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.09965cs.CVcs.GR

ProteusNeRF:使用3D感知图像上下文的快速轻量级NeRF编辑

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

University College London(伦敦大学学院)

Binglun Wang, Niladri Shekhar Dutt, Niloy J. Mitra

AI总结:

提出ProteusNeRF,一种快速轻量的NeRF编辑架构,通过3D感知图像上下文和语义特征蒸馏实现视图一致的交互式外观与几何编辑,速度较现有文本引导方法提升10-30倍。

摘要:

Neural Radiance Fields (NeRFs) 近期已成为照片级真实物体捕捉的流行选择,因为即使仅凭手持视频输入,它们也能忠实捕捉高保真体积内容。尽管已有大量研究致力于通过高效优化实现实时训练和渲染,但交互式编辑 NeRFs 的选项仍然有限。我们提出了一种非常简单但有效的神经网络架构,它不仅快速高效,而且保持了较低的内存占用。该架构可以通过用户友好的基于图像的编辑进行增量式引导。我们的表示方法允许在训练阶段通过语义特征蒸馏直接进行物体选择。更重要的是,我们提出了一种局部 3D 感知图像上下文,以促进视图一致的图像编辑,随后可以通过几何和外观调整将其蒸馏到微调的 NeRFs 中。我们在各种示例上评估了我们的设置,以展示外观和几何编辑,并报告称,相较于专注于文本引导 NeRF 编辑的同期工作,我们实现了 10-30 倍的加速。视频结果可在我们的项目网页 https://proteusnerf.github.io 上查看。

英文摘要:

Neural Radiance Fields (NeRFs) have recently emerged as a popular option for photo-realistic object capture due to their ability to faithfully capture high-fidelity volumetric content even from handheld video input. Although much research has been devoted to efficient optimization leading to real-time training and rendering, options for interactive editing NeRFs remain limited. We present a very simple but effective neural network architecture that is fast and efficient while maintaining a low memory footprint. This architecture can be incrementally guided through user-friendly image-based edits. Our representation allows straightforward object selection via semantic feature distillation at the training stage. More importantly, we propose a local 3D-aware image context to facilitate view-consistent image editing that can then be distilled into fine-tuned NeRFs, via geometric and appearance adjustments. We evaluate our setup on a variety of examples to demonstrate appearance and geometric edits and report 10-30x speedup over concurrent work focusing on text-guided NeRF editing. Video results can be seen on our project webpage at https://proteusnerf.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑