NeuralEditor:通过操纵点云编辑神经辐射场
NeuralEditor: Editing Neural Radiance Fields via Manipulating Point Clouds
AI总结:
针对NeRF难以编辑场景形状的问题,提出以点云为底层结构、基于K-D树引导密度自适应体素确定性积分渲染的NeuralEditor,实现形状编辑并达到SOTA性能,支持零样本推理与微调。
摘要:
本文提出NeuralEditor,可让神经辐射场(NeRFs)原生支持通用形状编辑任务。尽管NeRF在新视角合成方面效果出色,但编辑场景形状仍是其核心挑战。我们的关键洞见是利用显式点云表示作为构建NeRF的底层结构,这一思路源于对NeRF渲染的直观理解:渲染是将关联的3D点云投影或“绘制”到2D图像平面的过程。为此,NeuralEditor提出一种基于K-D树引导的密度自适应体素内确定性积分的新型渲染方案,通过优化既能生成高质量渲染结果,又能得到精确点云,随后通过在点云之间映射关联点来执行形状编辑。大量评估表明,NeuralEditor在形状变形和场景渐变任务中均达到了SOTA(state-of-the-art,当前最优)性能。值得注意的是,NeuralEditor同时支持零样本推理,以及对编辑后场景的进一步微调。我们的代码、基准和演示视频可在https://immortalco.github.io/NeuralEditor获取。
英文摘要:
This paper proposes NeuralEditor that enables neural radiance fields (NeRFs) natively editable for general shape editing tasks. Despite their impressive results on novel-view synthesis, it remains a fundamental challenge for NeRFs to edit the shape of the scene. Our key insight is to exploit the explicit point cloud representation as the underlying structure to construct NeRFs, inspired by the intuitive interpretation of NeRF rendering as a process that projects or "plots" the associated 3D point cloud to a 2D image plane. To this end, NeuralEditor introduces a novel rendering scheme based on deterministic integration within K-D tree-guided density-adaptive voxels, which produces both high-quality rendering results and precise point clouds through optimization. NeuralEditor then performs shape editing via mapping associated points between point clouds. Extensive evaluation shows that NeuralEditor achieves state-of-the-art performance in both shape deformation and scene morphing tasks. Notably, NeuralEditor supports both zero-shot inference and further fine-tuning over the edited scene. Our code, benchmark, and demo video are available at https://immortalco.github.io/NeuralEditor.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。