NeRF-VAE：一种几何感知的三维场景生成模型
NeRF-VAE: A Geometry Aware 3D Scene Generative Model
摘要：
我们提出了NeRF-VAE，一个3D场景生成模型，通过NeRF和可微分体绘制结合几何结构。与NeRF相比，我们的模型考虑了场景之间的共享结构，并且能够使用摊销推理来推断新场景的结构-而无需重新训练。NeRF-VAE的显式3D渲染过程进一步将以前的生成模型与缺乏几何结构的基于卷积的渲染进行了对比。我们的模型是一个VAE，学习辐射场的分布，通过调节它们的潜在场景表示。我们表明，一旦训练，NeRF-VAE能够使用很少的输入图像从以前看不见的3D环境中推断和渲染几何一致的场景。我们进一步证明了NeRF-VAE可以很好地推广到分布式相机，而卷积模型则不能。最后，我们介绍和研究了NeRF-VAE的解码器，它提高了模型的性能基于注意力的条件反射机制。
英文摘要：
We propose NeRF-VAE, a 3D scene generative model that incorporates geometric structure via NeRF and differentiable volume rendering. In contrast to NeRF, our model takes into account shared structure across scenes, and is able to infer the structure of a novel scene -- without the need to re-train -- using amortized inference. NeRF-VAE's explicit 3D rendering process further contrasts previous generative models with convolution-based rendering which lacks geometric structure. Our model is a VAE that learns a distribution over radiance fields by conditioning them on a latent scene representation. We show that, once trained, NeRF-VAE is able to infer and render geometrically-consistent scenes from previously unseen 3D environments using very few input images. We further demonstrate that NeRF-VAE generalizes well to out-of-distribution cameras, while convolutional models do not. Finally, we introduce and study an attention-based conditioning mechanism of NeRF-VAE's decoder, which improves model performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
