学习统一的分解式与组合式 NeRF 以实现可编辑的新视角合成
Learning Unified Decompositional and Compositional NeRF for Editable Novel View Synthesis
AI总结：
本文提出统一的两阶段NeRF框架，通过one-hot物体辐射场正则化和修复伪监督实现场景分解与组合，同时提升新视角合成和场景编辑性能。
摘要：
隐式神经表示在建模真实世界三维场景方面展现出强大能力，在新视角合成中提供了优越性能。本文针对一个更具挑战性的场景，即基于隐式神经场景表示的联合场景新视角合成与编辑。该方向上的最先进方法通常考虑为这两个任务（即视角合成和编辑）分别构建网络。因此，对这两个任务之间交互和相关性的建模非常有限，而这对于学习高质量的场景表示至关重要。为了解决这一问题，本文提出一个统一的神经辐射场（NeRF）框架，以有效执行联合场景分解与组合来建模真实世界场景。分解旨在学习不同物体和背景的解耦三维表示，从而支持场景编辑，而场景组合则建模整个场景表示以用于新视角合成。具体而言，通过一个两阶段 NeRF 框架，我们学习一个粗阶段来预测全局辐射场作为点采样的引导，并在第二个细粒度阶段中，通过一种新颖的 one-hot 物体辐射场正则化模块以及通过修复进行的伪监督来处理被物体遮挡的模糊背景区域。分解得到的物体级辐射场进一步利用分解模块的激活进行组合。大量定量和定性结果表明我们的方法在场景分解和组合上的有效性，在新视角合成和编辑任务上均优于最先进方法。
英文摘要：
Implicit neural representations have shown powerful capacity in modeling real-world 3D scenes, offering superior performance in novel view synthesis. In this paper, we target a more challenging scenario, i.e., joint scene novel view synthesis and editing based on implicit neural scene representations. State-of-the-art methods in this direction typically consider building separate networks for these two tasks (i.e., view synthesis and editing). Thus, the modeling of interactions and correlations between these two tasks is very limited, which, however, is critical for learning high-quality scene representations. To tackle this problem, in this paper, we propose a unified Neural Radiance Field (NeRF) framework to effectively perform joint scene decomposition and composition for modeling real-world scenes. The decomposition aims at learning disentangled 3D representations of different objects and the background, allowing for scene editing, while scene composition models an entire scene representation for novel view synthesis. Specifically, with a two-stage NeRF framework, we learn a coarse stage for predicting a global radiance field as guidance for point sampling, and in the second fine-grained stage, we perform scene decomposition by a novel one-hot object radiance field regularization module and a pseudo supervision via inpainting to handle ambiguous background regions occluded by objects. The decomposed object-level radiance fields are further composed by using activations from the decomposition module. Extensive quantitative and qualitative results show the effectiveness of our method for scene decomposition and composition, outperforming state-of-the-art methods for both novel-view synthesis and editing tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
