GO-NeRF：在神经辐射场中生成物体用于虚拟现实内容创作
GO-NeRF: Generating Objects in Neural Radiance Fields for Virtual Reality Content Creation
AI总结：
GO-NeRF提出一种在NeRF场景中根据文本提示和用户区域生成并集成3D物体的流程，通过组合渲染和定制优化提升场景和谐度与新视角质量。
摘要：
虚拟环境（VEs）对于虚拟现实、增强现实和混合现实系统至关重要。尽管3D生成和重建技术取得了进展，但在已建立的3D场景（以NeRF表示）中直接创建3D物体以生成新颖虚拟环境，仍是一个相对未被探索的领域。该过程复杂，不仅需要生成高质量的3D物体，还需将其无缝集成到现有场景中。为此，我们提出了一种新颖的流程，配备直观界面，称为GO-NeRF。我们的方法以文本提示和用户指定区域为输入，利用场景上下文在场景内生成3D物体。我们采用组合式渲染公式，有效将生成的3D物体集成到场景中，利用优化的3D感知不透明度图避免对原始场景产生意外修改。此外，我们开发了定制优化目标和训练策略，以增强模型捕获场景上下文的能力，并减轻在场景内优化3D物体时可能出现的伪影（如漂浮物）。在前向场景和360度场景上进行的大量实验表明，我们提出的方法在生成与周围场景和谐一致的物体以及合成高质量新视角图像方面表现出优越性能。我们致力于公开我们的代码。
英文摘要：
Virtual environments (VEs) are pivotal for virtual, augmented, and mixed reality systems. Despite advances in 3D generation and reconstruction, the direct creation of 3D objects within an established 3D scene (represented as NeRF) for novel VE creation remains a relatively unexplored domain. This process is complex, requiring not only the generation of high-quality 3D objects but also their seamless integration into the existing scene. To this end, we propose a novel pipeline featuring an intuitive interface, dubbed GO-NeRF. Our approach takes text prompts and user-specified regions as inputs and leverages the scene context to generate 3D objects within the scene. We employ a compositional rendering formulation that effectively integrates the generated 3D objects into the scene, utilizing optimized 3D-aware opacity maps to avoid unintended modifications to the original scene. Furthermore, we develop tailored optimization objectives and training strategies to enhance the model's ability to capture scene context and mitigate artifacts, such as floaters, that may occur while optimizing 3D objects within the scene. Extensive experiments conducted on both forward-facing and 360o scenes demonstrate the superior performance of our proposed method in generating objects that harmonize with surrounding scenes and synthesizing high-quality novel view images. We are committed to making our code publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
