SparseGNV：利用稀疏输入视图生成室内场景的新视角
SparseGNV: Generating Novel Views of Indoor Scenes with Sparse Input Views
AI总结：
SparseGNV提出一种结合3D神经点云与Transformer图像生成模型的框架，利用稀疏输入视图生成室内场景新视角，在真实与合成场景中均超越现有方法。
摘要：
我们研究在给定稀疏输入视图的情况下生成室内场景的新视角。挑战在于同时实现照片真实感和视角一致性。我们提出了SparseGNV：一个结合3D结构和图像生成模型的学习框架，通过三个模块生成新视角。第一个模块构建一个神经点云作为底层几何，为目标新视角提供上下文信息和指导。第二个模块利用基于Transformer的网络将场景上下文和指导映射到共享潜在空间，并以离散图像令牌的形式自回归解码目标视角。第三个模块将令牌重建为目标视角的图像。SparseGNV在大型室内场景数据集上进行训练，以学习可泛化的先验。一旦训练完成，它可以以前馈方式高效生成未见过的室内场景的新视角。我们在真实世界和合成室内场景上评估了SparseGNV，并证明它优于基于神经辐射场或条件图像生成的最先进方法。
英文摘要：
We study to generate novel views of indoor scenes given sparse input views. The challenge is to achieve both photorealism and view consistency. We present SparseGNV: a learning framework that incorporates 3D structures and image generative models to generate novel views with three modules. The first module builds a neural point cloud as underlying geometry, providing contextual information and guidance for the target novel view. The second module utilizes a transformer-based network to map the scene context and the guidance into a shared latent space and autoregressively decodes the target view in the form of discrete image tokens. The third module reconstructs the tokens into the image of the target view. SparseGNV is trained across a large indoor scene dataset to learn generalizable priors. Once trained, it can efficiently generate novel views of an unseen indoor scene in a feed-forward manner. We evaluate SparseGNV on both real-world and synthetic indoor scenes and demonstrate that it outperforms state-of-the-art methods based on either neural radiance fields or conditional image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
