基于组合神经辐射场的语义3D感知人像合成与操控
Semantic 3D-aware Portrait Synthesis and Manipulation Based on Compositional Neural Radiance Field
AI总结：
提出组合神经辐射场（CNeRF），通过为各语义区域学习独立辐射场并解耦形状与纹理，实现细粒度语义操控与高质量3D一致人像合成。
摘要：
近年来，基于神经辐射场的3D感知GAN方法发展迅速。然而，当前方法将整个图像建模为一个整体的神经辐射场，这限制了合成结果的局部语义可编辑性。由于NeRF逐像素渲染图像，因此可以在空间维度上拆分NeRF。我们提出了一种用于语义3D感知人像合成与操控的组合神经辐射场（CNeRF）。CNeRF按语义区域划分图像，并为每个区域学习独立的神经辐射场，最后将它们融合并渲染出完整图像。因此我们能够独立操控合成的语义区域，同时保持其他部分不变。此外，CNeRF还设计为在每个语义区域内解耦形状和纹理。与最先进的3D感知GAN方法相比，我们的方法实现了细粒度的语义区域操控，同时保持了高质量的3D一致性合成。消融实验证明了我们方法所使用的结构和损失函数的有效性。此外，真实图像反演和卡通人像3D编辑实验展示了该方法的应用潜力。
英文摘要：
Recently 3D-aware GAN methods with neural radiance field have developed rapidly. However, current methods model the whole image as an overall neural radiance field, which limits the partial semantic editability of synthetic results. Since NeRF renders an image pixel by pixel, it is possible to split NeRF in the spatial dimension. We propose a Compositional Neural Radiance Field (CNeRF) for semantic 3D-aware portrait synthesis and manipulation. CNeRF divides the image by semantic regions and learns an independent neural radiance field for each region, and finally fuses them and renders the complete image. Thus we can manipulate the synthesized semantic regions independently, while fixing the other parts unchanged. Furthermore, CNeRF is also designed to decouple shape and texture within each semantic region. Compared to state-of-the-art 3D-aware GAN methods, our approach enables fine-grained semantic region manipulation, while maintaining high-quality 3D-consistent synthesis. The ablation studies show the effectiveness of the structure and loss function used by our method. In addition real image inversion and cartoon portrait 3D editing experiments demonstrate the application potential of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
